Position: Russian Audio Generalist Evaluator Expert (San Francisco Bay Area)
Type: Hourly contract
Compensation: $50 per hour
Location: Remote
Commitment: 10 20 hours/week
Role Responsibilities
- Listen to, analyse, and transcribe audio and video content in Russian following detailed constraints and instructions
- Produce high-quality written outputs in Russian, with supporting work in English when required
- Ensure clarity, accuracy, and adherence to formatting and stylistic guidelines
- Capture nuances such as tone, intent, register, regional expressions, dialectal variations, and contemporary Russian usage
- Establish expectations for high-quality responses in general consumer audio contexts
- Develop evaluation rubrics and grading guidelines in Russian and English
- Document standards to ensure consistency across reviewers and model evaluations
- Identify linguistic nuances, grammatical complexities, colloquialisms, and edge cases specific to Russian
- Run prompts through language models and assess generated outputs
- Evaluate responses for accuracy, completeness, fluency, and instructional clarity
- Provide structured feedback to improve model performance in Russian audio tasks
- Participate in QA and review cycles to ensure tasks, rubrics, and outputs meet quality standards
- Maintain consistency and reliability before datasets are integrated into benchmarks
- Collaborate with project leads to resolve ambiguities and improve task design
Requirements
- Strong writing, editing, and critical thinking skills
- Ability to work independently, manage time effectively, and meet deadlines
- Strong familiarity with spoken Russian, regional vocabulary, dialects, and contemporary language usage
- Ability to accurately transcribe and analyse Russian audio content across general consumer contexts
- Based in the San Francisco Bay Area
- Available to commit 10 20 hours per week
- College student or recent graduate preferred
- Prior experience with transcription, annotation, localisation, evaluation, or research workflows in Russian preferred
- Familiarity with regional variations of Russian and contemporary digital language usage
- Interest in AI, language models, or applied research environments
Application Process
- Complete the application form and submit the required information.
- Applications will be reviewed based on the role requirements.
- Eligible candidates will receive an inmail with instructions to continue the application process.
- Follow the instructions in the inmail to complete the remaining application steps.
#LI-DNI