Role Overview: We are seeking an experienced Query Classification & Vetting Specialist with strong expertise in commerce ,Business Services & Healthcare to support an AI Data Annotation and LLM evaluation project. The role involves reviewing and vetting queries/prompts, validating AI-generated domain and sub-domain classifications, and identifying and correcting inaccurate classifications.
Business & Commerce
Education: Bachelor’s/Master’s in Business Administration OR Management OR Commerce OR Economics OR a related business field from Top Colleges
Experience: 3+ years in business operations OR commercial operations OR consulting OR strategy OR general management OR professional services from good companies.
Healthcare
Education: Bachelor’s/Master’s in Medicine OR Nursing OR Pharmacy OR Healthcare Administration OR Public Health OR a related field from Top colleges.
Experience: 3+ years in healthcare delivery OR hospital/clinical operations OR healthcare services OR pharma from good companies.
Role Details:
Identify ambiguous or incorrectly classified queries and apply appropriate domain judgment.
Determine whether the predicted classification accurately reflects the query's intent and context.
Correct inaccurate domain and sub-domain classifications wherever required.
Maintain high accuracy and consistency across large datasets and perform quality checks on annotated data
Preferred / good-to-have
Working or conceptual knowledge of data labeling / annotation workflows
Working or conceptual knowledge of prompt engineering / LLM evaluation
Familiarity with following classification taxonomies, rubrics, and annotation guidelines