Remote Data Annotation Jobs Amsterdam (Remote)
Rex.zone connects experienced annotators with real-world AI/ML training workflows, including data labeling, RLHF evaluation, prompt evaluation, and training data quality improvements for large language models. This is a full-time remote role supporting distributed teams and rubric-driven annotation operations.
About The Role
As a mid-senior Data Annotation professional, you will deliver high-precision labels and human judgments used to train, fine-tune, and evaluate AI systems. Work spans NLP classification, named entity recognition, prompt evaluation for LLMs, RLHF preference ranking, computer vision annotation, and content safety labeling.
Key Responsibilities
- Annotate and review datasets for NLP, CV, and content safety tasks while following detailed rubrics
- Perform RLHF workflows (preference ranking, pairwise comparisons, justification capture)
- Execute QA evaluation: sampling plans, discrepancy analysis, and rework loops to reduce label noise
- Document edge cases and propose guideline updates to improve inter-annotator agreement
- Support prompt evaluation and model evaluation sets measuring helpfulness, correctness, and safety
- Maintain secure handling of sensitive content and adhere to confidentiality requirements
Required Qualifications
- 3+ years in data annotation, data labeling, QA evaluation, or model evaluation for AI/ML
- Experience with annotation guidelines compliance and quality calibration methods
- Familiarity with RLHF, prompt evaluation, or LLM evaluation concepts
- Strong analytical writing for issue reports and edge-case documentation
- Comfort working remotely with distributed teams and structured throughput/quality metrics
Compensation
Hourly rate: $30–$50/hr (USD), depending on scope and evaluation responsibilities.
How To Apply
Apply through Rex.zone and keep your profile updated with labeling domains (NLP, CV, content safety), QA evaluation experience, and any RLHF or prompt evaluation work.