Lead AI Quality Engineer & Test Automation Architect

About the role

We are looking for a senior professional to design and execute end‑to‑end testing strategies for Machine Learning models, Generative AI systems, Retrieval‑Augmented Generation (RAG) architectures, and autonomous agents. The role ensures model accuracy, fairness, robustness and compliance with AI ethics and regulatory guidelines.

Key responsibilities

• Design and run testing pipelines for model accuracy, bias detection, explainability, and performance on diverse datasets.
• Conduct adversarial testing, prompt‑injection, jailbreaking and red‑team exercises to evaluate robustness.
• Validate multi‑step agentic workflows, state transitions, tool execution and fallback behaviours.
• Assess LLM output quality, vector store behavior, semantic search accuracy and API performance (latency, throughput, concurrency).
• Define AI quality KPIs, establish test governance and integrate automated tests into CI/CD pipelines.

Required profile

• Proven senior/lead experience in software quality engineering with a focus on AI/ML and Generative AI.
• Strong background in Python for test automation and custom AI testing scripts.
• Hands‑on experience with AI ethics, compliance testing and security of data pipelines.

Required skills

• Python
• Postman or REST Assured (API testing)
• CI/CD pipelines
• Docker & Kubernetes
• Cloud platforms (AWS/Azure/GCP)
• Vector databases and RAG frameworks
• LLM evaluation tools (Ragas, DeepEval, TruLens)