Senior Software Test Automation Engineer
Location: Bengaluru – Hybrid
Experience: 7+ Years
Role Overview
We are looking for a Senior Software Test Automation Engineer to drive automation and QA standards for a SaaS-based AI Platform. This is a hands-on individual contributor role focused on Python/Pytest automation, API & UI testing, backend/platform quality, and CI/CD quality gates, with an opportunity to build expertise in LLM and Agent testing.
Key Responsibilities
- Design and maintain robust Pytest-based automation frameworks for Python backend services and agent logic.
- Build automated tests for REST APIs, contracts, schemas, and service-level behaviour using HTTPX, Postman or equivalent tools.
- Develop Playwright-based UI automation for Angular applications.
- Validate SaaS platform capabilities including multi-tenancy, APIs, authentication, event-driven workflows, and database state.
- Integrate automated testing and quality gates into Azure DevOps CI/CD pipelines.
- Define and promote QA automation standards, framework best practices, and reusable testing approaches across multiple engineering squads.
- Contribute to testing strategies for LLM/AI agents, including non-deterministic outputs, multi-step workflows, tool calls, intermediate states, and model-version regressions.
- Collaborate with engineering teams to identify systemic quality gaps and improve automation coverage and test effectiveness.
Must-Have Skills
- 7+ years of QA/Test Automation experience.
- Strong hands-on Python + Pytest experience, preferably with framework design/architecture.
- Strong Playwright experience; Selenium experience may also be considered.
- Experience in REST API automation/testing using HTTPX, Postman or equivalent.
- Experience testing SaaS, backend services, APIs or microservices.
- Strong understanding of SQL/NoSQL for test data and database validation.
- Experience integrating automation into Azure DevOps CI/CD pipelines.
- Understanding of LLM/Agent testing concepts, including non-deterministic outputs and multi-step agent workflows.
Good to Have
- Langfuse or equivalent LLM observability experience.
- Ragas or other LLM evaluation frameworks.
- Experience with event-driven/distributed systems and multi-tenant platforms.
- Performance/load testing experience.
- Experience defining QA standards or mentoring automation engineers.