Remote Data Annotation Specialist (Milan) — Remote (US-listed)
Rex.zone connects Milan-based talent with full-time remote AI data work supporting LLM training pipelines, RLHF, and model evaluation. You will label, review, and QA training data using clear rubrics and annotation guidelines to improve model quality and safety.
What You Will Do
- Produce labeled examples for NLP and computer vision annotation tasks
- Perform prompt evaluation, rubric scoring, and RLHF preference ranking/response ranking
- Execute QA evaluation using gold sets, peer review, audits, and disagreement resolution
- Apply named entity recognition (span labeling) and taxonomy-based labeling
- Support content safety labeling (policy-driven categories) and escalate ambiguous cases
- Document annotation decisions and ensure annotation guidelines compliance
Core Workflows
- RLHF ranking and instruction-following grading
- Inter-annotator agreement checks and training data quality audits
- Adversarial/edge-case prompt evaluation
- Computer vision annotation (bounding boxes, polygons, keypoints)
Requirements
- 3+ years in data annotation, data labeling, QA evaluation, or ML data operations
- Strong written English and ability to follow detailed rubrics
- Familiarity with LLM evaluation concepts (preference ranking, instruction tuning)
- Comfort with annotation tools and structured labeling schemas
- Ability to maintain high accuracy under throughput targets
- Security-minded approach to handling potentially sensitive content
Employment Notes
This is a FULL_TIME Remote role aligned to Milan-based availability; projects may span multiple time zones depending on client needs.
Pay
Competitive hourly rate: $30–$50 per hour (USD)