AI Quality & Automation Engineer
Location: Gurugram, Haryana
Employment Type: Full-Time
Experience: 4–7 Years
Role Overview
We are looking for an AI Quality & Automation Engineer to drive quality across our web platform, APIs, AI agents, recommendation engines, and AI-generated content workflows.
You will build scalable automation, evaluate AI outputs, establish quality benchmarks, and create robust testing frameworks for an AI-first product.
Key Responsibilities
🔹 Test Automation & Platform Quality
- Build and maintain scalable UI, API, regression, integration, and E2E automation.
- Develop automation using Playwright / Cypress and integrate tests into CI/CD pipelines.
- Improve test coverage, reduce manual effort, and proactively identify production risks.
- Perform API, database, and integration testing.
🔹 AI & Agent Quality
- Test LLM applications, AI agents, RAG systems, recommendation engines, and AI-generated content.
- Create AI evaluation criteria, benchmarks, and regression frameworks.
- Identify hallucinations, reasoning failures, inconsistencies, edge cases, and quality degradation.
- Test prompt-driven workflows, agent orchestration, multi-agent workflows, and tool calling.
🔹 API & Third-Party Integrations
- Test integrations with Meta Ads, Shopify, Stripe, Chargebee, and Google Services.
- Validate REST APIs, OAuth flows, webhooks, API contracts, and data synchronization.
- Test failure scenarios and integration edge cases.
🔹 Quality Strategy
- Define QA processes, testing standards, release quality gates, and quality KPIs.
- Partner with Product, Engineering, AI, Design, and Data teams to drive a quality-first culture.
Required Skills
- 4+ years in Quality Engineering / Test Automation.
- Strong experience with Playwright / Cypress; Selenium is a plus.
- Strong API testing experience with Postman / REST APIs.
- Experience with CI/CD, SQL, database validation, and Agile methodologies.
- Strong programming skills in JavaScript/TypeScript, Python, or Java.
- Hands-on experience testing LLM-based applications, AI Agents, Prompt workflows, RAG, or AI-generated content.
- Experience with AI evaluation tools such as LangSmith, Promptfoo, DeepEval, or OpenAI Evals.
- Strong hands-on use of AI tools such as ChatGPT, Claude, Cursor, Playwright MCP, or Browser Use.
Nice to Have
- Experience with AI, MarTech, AdTech, or SaaS products.
- Exposure to Meta Ads, Shopify, Google Ads, or marketing analytics.
- Familiarity with AWS and cloud-native applications.
- Experience with CloudWatch, Datadog, Grafana, or Sentry.
- Understanding of ML systems and model evaluation.