About The Role
Senior data annotator jobs in Madrid at Rex.zone focus on high-precision data labeling and evaluation workflows that improve large language model (LLM) training pipelines. This is a remote, full-time role emphasizing training data quality, auditability, and consistent guideline-driven execution.
What You Will Do
- Deliver high-accuracy data labeling across NLP tasks (e.g., named entity recognition, classification, ranking) and computer vision annotation when needed
- Run QA evaluation, peer review, and adjudication to resolve disagreements and improve inter-annotator agreement
- Enforce annotation guidelines compliance; document edge cases and propose guideline refinements
- Execute QA sampling plans, error analysis, and root-cause reporting to improve training data quality
- Support RLHF and prompt evaluation by ranking responses, writing preference rationales, and validating reward-model training sets
- Collaborate with project leads and stakeholders to track throughput, accuracy, rework rate, and audit pass rates
Required Qualifications
- Mid-Senior experience in data annotation, data labeling, or QA evaluation for ML datasets
- Strong command of annotation guidelines compliance and consistent decision-making in ambiguous cases
- Familiarity with LLM training pipelines, large language model evaluation, prompt evaluation, and RLHF workflows
- Comfort working in a remote, metrics-driven delivery environment
Quality Standards
- Maintain targets for accuracy, inter-annotator agreement, audit pass rates, and low rework rates
- Support traceable and reproducible pipelines through gold set verification and documented decision logs
Apply
Apply through Rex.zone and highlight your data labeling depth, QA evaluation rigor, and experience improving training data quality for LLM training pipelines.