AI Jobs Map

Delta Labs · Zurich, Zurich, Switzerland

Senior Full Stack Developer — Evaluation & Model Quality

seniorfull timePosted today
Apply on LinkedInOpens the original posting. AI Jobs Map never asks for your details.

Stack mentioned

observabilityllmsystem-designetla/b-testingtypescriptnext.jsreactpostgresqlpythonmicroservicesopenaianthropicgeminiazure

ABOUT THE ROLE

The most important question anyone can ask about a simulated population is whether it's right. It's the question our clients ask, and the one we hold ourselves to. Answering it rigorously, repeatably and at scale is what this role exists to do.

You'd be building the measurement layer for behavioural simulation: the systems that decide whether a model change improved the thing we care about, whether a population behaves like the people it represents, and whether an output is good enough to put in front of a client making a real decision. You'd work alongside a research team with unusual depth in exactly this problem — survey methodology, experimental design, psychometrics, computational models of decision-making — turning measurement judgment that today lives in their heads into infrastructure that runs on every release.

An increasing share of that work is validating simulated behaviour against real behavioural outcomes, not just self-reported response. That's the frontier of this role.

RESPONSIBILITIES

– Build the measurement layer: evaluation execution infrastructure, versioned datasets, provenance, holdout integrity, regression suites that run inside the development loop rather than as a gate at the end

– Build the pipelines comparing simulated responses against real behavioural data — transaction records, panel data, client validation studies

– Turn research judgment into running code: take a validity criterion and make it a metric, a rubric, an automated grader, a release criterion

– Own model observability — extend our LLM tracing so a regression can be traced to the prompt version, model or population that caused it

– Build tools that let a behavioural scientist inspect outputs, label data and investigate a regression without asking an engineer

YOU MAY BE A FIT IF

– 4+ years building production software, with sound judgment on system design, testing and maintainability — this is an engineering role

– Backend services, data pipelines, relational modelling, automation workflows

– Statistical literacy: sampling, distributions, uncertainty, calibration, confidence intervals. Enough to build the right thing and argue with a researcher productively — you needn't be a statistician, you need to not be intimidated by one

– Enough LLM fluency to reason about whether a change improved the thing we care about or just moved a number

– AI tools as standard development practice

– Hands-on ownership: build the first version, inspect the data, find the metric was wrong, revise it — repeatedly

STRONG CANDIDATES MAY ALSO HAVE

– LLM or ML evaluation systems, benchmark platforms, regression suites, experiment tracking, model-quality dashboards

– LLM-as-judge systems, rubric design, grader calibration, human-in-the-loop labelling

– Survey methodology, psychometrics, causal inference or A/B testing infrastructure

– Internal tooling for researchers or data scientists

– Sensitive human data: consent, access control, auditability

This role suits someone who finds the honest version of a result more interesting than the flattering one. If you've ever killed a metric because you worked out how it could be gamed, we want to hear about it.

ABOUT DELTA LABS

Delta Labs uses AI to simulate and predict consumer behaviour at scale. We build Elaiia, a simulation engine that generates AI Twins — intelligent synthetic agents that mirror real consumer populations. Our clients use Elaiia to simulate customer decisions before committing to them: pricing strategies, product launches, campaign messaging, channel allocation. We replace surveys, focus groups, and intuition with simulation-based evidence.

We're a small, focused team and we intend to stay that way. We give people ownership, trust, and the autonomy to do their best work. We work with urgency and intellectual honesty and expect new team members to match our pace. We seek individuals who are curious, rigorous, and want their work to have demonstrable impact. If you're drawn to the idea of a small team building something that hasn't existed before, let's build together.

THE STACK

TypeScript frontend (Next.js/React), PostgreSQL, Python microservices, durable background jobs, integrated LLM APIs (OpenAI, Anthropic, Google Gemini), LangGraph agents with tracing. Deployed on Vercel and Microsoft Azure.

LOCATION

This role is based in Zürich, Switzerland. Delta Labs is an in-person company. Candidates are expected to be located in the Zürich area or open to relocation.

BENEFITS

Ownership of systems rather than tickets. Direct work with a research team of unusual depth — behavioural scientists, experimental economists, psychometricians and cognitive scientists whose methods you'd be building. A product global enterprises use for decisions that matter. The chance to shape an early-stage company.

Delta Labs is an equal opportunity employer, welcoming applicants of all backgrounds.

More jobs at Delta Labs