Remote | Generalist – English & Italian – $36.16/hr

We are sharing a specialised part-time consulting opportunity for bilingual professionals fluent in English and Italian who are interested in contributing to the evaluation and improvement of advanced AI systems. This role supports collaborations with leading AI teams focused on improving how conversational AI models respond to real-world user questions across a wide range of topics. Experts will evaluate AI-generated responses and provide structured feedback to ensure accuracy, clarity, reasoning quality, and natural communication aligned with human expectations. Key Responsibilities Evaluate AI-generated responses to determine how effectively they answer user queries Conduct fact-checking using trusted public sources and external tools Annotate responses by identifying strengths, weaknesses, and factual inaccuracies Assess reasoning quality, tone, clarity, and completeness of responses Ensure model responses align with expected conversational behavior and system guidelines Apply consistent evaluation standards using structured taxonomies and benchmarks Ideal Profile Strong candidates will have: Bachelors degree or higher Language Requirements: Native or C2-level proficiency in Italian Fluent English Additional qualities include: Experience working with large language models (LLMs) Excellent writing and feedback articulation skills Strong analytical and critical thinking ability High attention to detail and ability to detect subtle reasoning issues Comfort working across diverse topics and domains Additional Background That Helps Candidates often come from fields requiring structured analytical thinking, such as: Research Policy analysis Data analytics Linguistics Engineering Strong college-level mathematics and reasoning skills are also beneficial for the role. Nice-to-Have Experience Experience with RLHF, model evaluation, or AI data annotation Experience producing high-quality written content Experience comparing multiple outputs and making qualitative judgments Familiarity with evaluation rubrics or structured scoring frameworks Location Requirements Candidates should be based in one of the following regions: Europe United States What Success Looks Like Identifying factual inaccuracies and reasoning errors in model outputs Producing clear and consistent evaluation feedback Delivering reproducible annotations that improve AI system performance Helping ensure AI systems communicate clearly and reliably Why This Opportunity Contribute directly to improving how AI systems interact with real users Work with cutting-edge AI labs and research teams Flexible remote work with competitive compensation Play a direct role in shaping next-generation AI systems Contract Details Independent contractor role Fully remote with flexible scheduling Compensation: $36.16/hour Weekly payments via Stripe or Wise Projects may extend depending on performance and project needs About the Platform This opportunity is available through a leading AI-driven work platform.