We are looking for a QA / Automation Engineer
This role involves building and maintaining automated tests for an agentic, LLM-driven application, focusing on creating a new Python-based framework for evaluating agentic systems.
Responsibilities:
- Build and maintain automated tests for an agentic, LLM-driven application, extending beyond conventional UI test automation.
- Develop a new, Python-based framework for evaluating agentic systems, including LLM-as-judge style evaluators and state checks for conversational/agentic flows.
- Utilize Playwright for front-end automation where applicable, while focusing on the Python-based evaluation layer for AI-driven behavior.
- Collaborate closely with developers in a pod structure, as QA is distributed and collaborative rather than siloed.
Requirements:
- Comfort operating with a light, UAT-style formal QA process, where quality ownership is shared with developers.
- Ability to work in an undefined, evolving testing landscape as the framework is being built and expanded.
Required Skills:
- Proficiency in Python, as it is the primary language for the role's evaluation/automation framework.
- Experience or strong aptitude for testing AI/LLM-based systems, including evaluators, LLM-as-judge techniques, and state/behavior validation for agentic or conversational systems.
- Working knowledge of Playwright for the front-end automation layer, though it is not the core skill being hired for.
Preferred Skills:
- Industry domain knowledge in HCM.