Senior Data Annotator (Remote, Full-Time)
Join Rex.zone to support AI/ML training workflows by producing and reviewing high-quality labeled data used for model training and evaluation. You will execute guideline-driven annotation, QA evaluation, RLHF preference judgments, and prompt evaluation to strengthen LLM training pipelines across NLP, computer vision, and content safety.
Key Responsibilities
- Execute senior-level data labeling and review for text, image, and multimodal datasets
- Perform RLHF preference ranking and large language model evaluation using defined rubrics
- Run QA evaluation, audits, and calibration rounds to ensure training data quality
- Identify guideline gaps, document edge cases, and propose rubric clarifications
- Monitor inter-annotator agreement and resolve disagreements through calibration
- Support prompt evaluation for helpfulness, harmlessness, and policy compliance
- Contribute to content safety labeling and sensitive-topic handling processes
- Partner with engineering and data ops to improve tooling, throughput, and error detection
Required Qualifications
- Experience in data annotation, data labeling, or dataset QA for ML systems
- Comfort applying detailed rubrics and maintaining annotation guidelines compliance
- Familiarity with RLHF concepts, preference data, or LLM evaluation workflows
- Strong written reasoning for judgment-based tasks and edge-case documentation
- Experience with NLP (e.g., named entity recognition) and/or CV annotation (e.g., bounding boxes, segmentation)
How Quality Is Measured
Quality is assessed via audits, inter-annotator agreement checks, rubric adherence, calibration rounds, and error-rate monitoring to protect downstream model performance.
Pay
$30–$50 per hour (USD)