Refine system prompts to improve LLM output accuracy, tone, and style.
Identify recurring error patterns and translate them into actionable instructions for the LLM.
Manage the feedback loop between LLM output and desired target language standards.
- Error Root Cause Analysis (ERCA)
Perform a deep-dive analysis on rejected translations to distinguish between “linguistic preference” and “systemic LLM hallucination.”
Categorize errors by source (e.g., glossary gaps, ambiguity, cultural misalignment) to guide iterative model improvements.
- Quality Metrics & Auditing
Implement and track key performance indicators (KPIs) such as percentage of words changed (Human Edit Distance, HED), Error Density, and Multidimensional Quality Metrics (MQM) scores.
Drive the transition from 100% manual review to exception-based auditing, focusing on high-risk and high-visibility content.
Conduct quarterly quality health checks to ensure the system maintains parity as it gains autonomy.
- Data Curation & Asset Management
Maintain and evolve localized glossaries, style guides, and gold standard datasets to serve as training inputs for the LLM.
Collaborate with the localization lead and Cisco’s regional teams to ensure terminology consistency and brand voice.
Required Competencies
Linguistic Expertise: Native-level fluency in one of the target languages (German, French, or Japanese) with high proficiency in English.
Analytical Mindset: Ability to identify systemic patterns rather than focusing solely on individual string corrections.
Prompt Engineering Awareness: Interest in or experience with LLM prompting techniques and data annotation.
Strategic Thinking: Ability to balance immediate quality needs with long-term goals of system automation.
Adaptability: Experience or interest in evolving from a translator role to a system architect role within a fast-paced technical environment.
Success Metrics (KPIs)
HED Reduction: Steady decrease in Human Edit Distance for standard content.
Error Density: Maintaining critical error rates at near-zero levels.
Automation Velocity: Successfully migrating content categories from human-review to automated-audit workflows.
System Stability: Ensuring consistent terminology and style application across localized sites. #LI-DNI