AI Evaluator

We are looking for highly skilled AI Evaluators to support the development of advanced AI tools for grammar correction and proofreading. This project focuses on creating a rich and diverse dataset of written Italian text, annotated and corrected by humans.

Responsibilities:

• Respond to a set of prompts simulating realistic business or casual communication.
• Write clear, plausible short texts (3–4 sentences for business settings, 1–2 sentences for casual chat).
• Generate responses with intentional but realistic grammar, spelling, and structural errors.
• Review annotated message pairs (input text + corrected/proofread version) to ensure:
• All grammar/spelling/preposition/punctuation errors are corrected appropriately
• Meaning is preserved between input and corrected text
• Error types are classified correctly
• Explanations are clear and linguistically sound

Qualifications:

• Italian as a first language - C1 Italian proficiency 
• English as a second language - B2 English proficiency.
• Ability to follow linguistic and situational writing prompts.
• Good written communication and creativity.
• 2 years working experience in a corporate/ business environment.
• 1 year working experience working as an AI Evaluator, data annotator, or similar role 
• Strong computer literacy and fast, accurate typing skills are essential for navigating annotation tools and maintaining efficiency in text-based tasks.
• Strong logic, intuitive problem-solving and critical thinking; adaptability to the needs of the task. The candidate needs to be open-minded and welcome different viewpoints or solutions.
• Proactiveness, confidence to work independently without constant support. Reliability and being able to make good decisions on their own.
• Be a thought partner to delivery managers, input into the project delivery strategies, and share quality best practices to aid in achieving the project goals.
• Adapt quickly to evolving guidelines, new task types, and AI model updates in a fast-paced, iterative environment. Access to your own laptop for work