Remote Data Labeling Specialist (Vancouver) — Full-Time
Rex.zone is hiring a Remote Data Labeling Specialist to deliver consistent, policy-aligned annotations used to train and evaluate AI/ML models. You will work across NLP labeling, RLHF preference ranking, prompt evaluation, computer vision annotation, and content safety labeling to improve training data quality and downstream model performance.
Key Responsibilities
- Apply annotation guidelines accurately across NLP and computer vision tasks
- Perform RLHF preference ranking and rubric-based evaluations to align model behavior with human preferences
- Execute prompt evaluation and response quality scoring for LLM evaluation workflows
- Conduct QA evaluation, sampling audits, guideline compliance checks, and error categorization to reduce label noise
- Label content safety categories (e.g., harassment, self-harm, hate) using policy taxonomies
- Collaborate with project leads to resolve edge cases, refine rubrics, and document decisions
- Track productivity and accuracy metrics to support model performance improvement
Workstreams You May Support
- NLP data labeling: named entity recognition (NER), intent classification, sentiment/topic labeling, summarization evaluation
- LLM training pipelines: instruction-following evaluation, pairwise preference ranking, rubric grading
- Computer vision annotation: bounding boxes, polygons, keypoints, segmentation, OCR validation
- Content safety labeling: policy-based moderation labels and severity scoring
- Multimodal tasks: text-image relevance, caption validation, VQA-style evaluation
Required Qualifications
- Experience applying structured guidelines to data annotation and/or QA evaluation work
- Strong written communication for consistent labeling rationale and disagreement resolution
- Comfort working with ambiguity, edge cases, and evolving rubrics
- Familiarity with NLP, RLHF, or LLM evaluation concepts (training data quality, preference ranking, prompt evaluation)
- Ability to maintain accuracy at production scale and meet full-time delivery expectations
Preferred Qualifications
- Experience with annotation platforms and audit workflows
- Exposure to computer vision annotation tools (boxes, polygons, segmentation)
- Experience with content safety labeling or policy taxonomy interpretation
- Experience documenting guidelines, building QA checklists, or running inter-annotator agreement checks
Remote Work Notes
This is a Remote role. You can be based in Vancouver while working with distributed teams. Projects may require overlap with US business hours, and task availability can vary by domain.
Pay: $30–$50 per hour (base salary).