⚡ Member of Technical Staff (QA Engineer - Agentic Systems)
💊 AI-Powered Pharmaceutical Marketing
📍 New York City (on-site)
💸 $160k - $300k + equity
Join a fast-growing AI startup transforming how pharmaceutical companies create and launch marketing campaigns. You'll be the first dedicated QA hire, owning quality across a platform powered by LLMs, agentic systems, and AI workflows. This is a highly technical role where you'll work directly with leadership and engineering to build the testing, evaluation, and quality infrastructure that keeps the platform reliable, compliant, and production-ready.
Responsibilities
- Build and own evaluation frameworks for LLM and agentic systems
- Create automated testing infrastructure across AI, backend, and frontend systems
- Design tests that identify agent failures, regressions, and compliance risks
- Own end-to-end testing using Playwright and manual exploratory QA
- Implement CI/CD quality gates, test automation, and release sign-off processes
- Partner with Engineers and leadership to define the company's quality standards
Requirements
- Strong experience with Python and building testing infrastructure
- Hands-on expertise with Playwright or modern end-to-end automation frameworks
- Experience testing LLM, ML, or non-deterministic systems
- Knowledge of evaluation methodologies including golden datasets, LLM-as-judge, and regression testing
- Strong understanding of quality metrics, statistical analysis, and error investigation
- Ability to operate independently and build processes from 0 → 1
- Agentic AI, LangChain, LangGraph, or LLM tooling experience strongly preferred
Why apply?
- First QA hire with ownership over the entire quality function
- Work on the cutting edge of AI and agentic systems
- Partner closely with an elite team backed by top healthcare and AI operators
- Competitive compensation with meaningful equity upside
- Health, dental, vision, 401(k), relocation support, and unlimited PTO
- Help shape a platform already trusted by 50+ pharmaceutical brands
📍 On-site, New York City
📧 Interested in applying? Please click on the 'Easy Apply' button.
⚡ Storm3 is a HealthTech recruitment firm partnering with some of the most innovative healthcare and AI companies across North America. To discuss open opportunities or career options, visit storm3.com and follow the Storm3 LinkedIn page for the latest jobs and market insights.