Agentic Judge - Backoffice Helpdesk

About the role

This hybrid backoffice helpdesk role supports customer experience operations for independent software vendors and high-tech clients. It covers day-to-day case management documentation and coordination, using strong spoken English to resolve inquiries, keep accurate records and work with internal teams so that reliable digital services can be delivered to global customers.

Key responsibilities

• AI annotation: review structured and unstructured data outputs from AI models and flag inaccuracies, inconsistencies or hallucinations
• Summarise AI-generated summaries against source material for accuracy, completeness and appropriate length
• Review AI-generated responses and give structured written feedback on failures
• Review pre-annotated images and maintain quality benchmarks across high-volume batches
• Review AI content about US real estate (listings, transaction summaries, market analyses, home inspection reports, valuations) and identify factual errors, outdated terms or missing context
• Add corrective commentary to AI responses and evaluate property feature descriptions (floor plans, room types, fixtures, materials, appliances)
• Give culturally nuanced feedback on whether content fits a US homeowner or homebuyer audience

Required profile

• Full professional English proficiency, written and spoken
• Strong reading comprehension and attention to detail, able to apply written criteria consistently across many samples
• Comfortable working with structured review interfaces, spreadsheets or annotation platforms
• Reliable internet connection and ability to work independently and meet deadlines with minimal supervision
• Able to maintain throughput targets while keeping quality
• Prior experience in data annotation, content moderation, QA, transcription or a related field is a plus