Quality · Full time
QA & Evaluation Engineer
Own how we prove an AI system works: evaluation sets, regression suites and the reporting clients see every release.
- Experience
- 2-5 years
- Type
- Full time
- Location
- Whitefield, Bengaluru (hybrid)
What you will do
- —Build evaluation datasets with client subject-matter experts
- —Automate regression suites for prompts, retrieval and applications
- —Report accuracy, latency and cost per release
- —Investigate production failures and turn them into test cases
What we need from you
- —Test automation experience with Python or TypeScript
- —Understanding of statistical evaluation basics
- —Meticulous about reproducibility
- —Clear reporting to non-technical readers
Nice to have
- ·LLM evaluation tooling experience
- ·Performance testing
- ·Accessibility testing
We read every application ourselves and reply within two weeks. If you would rather write to a person, email admin@a2z-aitech.com.