ISO 5060: The New International Standard for Translation Quality Evaluation

ISO 5060:2024 is the first international standard specifically focused on evaluating translation quality. Published in 2024, it consolidates decades of industry best practices—particularly from the MQM framework—into an official ISO standard that organizations worldwide can adopt.

In this guide, you'll learn what ISO 5060 covers, how it relates to other translation standards, and how to implement it in your organization.

What is ISO 5060?

ISO 5060, officially titled "Translation and interpreting — Evaluation of translation output — General guidance," provides a comprehensive framework for assessing translation quality. Unlike ISO 17100 (which covers translation service requirements) or ISO 18587 (post-editing requirements), ISO 5060 focuses specifically on how to evaluate the quality of translated content.

Key Features of ISO 5060

The standard introduces several important concepts:

Why ISO 5060 Matters

Before ISO 5060, organizations used various quality evaluation approaches—LISA QA, J2450, custom frameworks, or MQM. This fragmentation made it difficult to:

• Compare quality across vendors
• Establish industry-wide benchmarks
• Train evaluators consistently
• Demonstrate compliance to clients

ISO 5060 provides a unified reference point that the entire industry can adopt.

ISO 5060 and MQM: The Connection

The Multidimensional Quality Metrics (MQM) framework served as the primary foundation for ISO 5060. The standard's error typology is explicitly harmonized with MQM's first-level categories:

Error Categories in ISO 5060

Based on MQM, ISO 5060 defines these primary error types:

Severity Levels

ISO 5060 adopts the three-tier severity model:

1. Critical - Errors that could cause legal liability, safety risks, or severe misunderstanding. Example: incorrect medication dosage in medical translation.

2. Major - Errors that significantly impact comprehension or user experience. Example: mistranslated key product feature.

3. Minor - Errors with minimal impact on understanding. Example: minor punctuation inconsistency.

How ISO 5060 Differs from Other Standards

ISO 5060 vs. ISO 17100

ISO 17100 tells you how to produce quality translations. ISO 5060 tells you how to measure translation quality.

ISO 5060 vs. ISO 11669

ISO 11669:2024, released alongside ISO 5060, focuses on translation specifications—how to define requirements before translation begins. The two standards work together:

• ISO 11669 → Define quality requirements upfront
• ISO 5060 → Evaluate whether requirements were met

ISO 5060 vs. ISO 18587

ISO 18587 covers post-editing of machine translation output. ISO 5060 can be applied to evaluate the quality of any translation—human, machine, or post-edited.

The Three Phases of ISO 5060 Evaluation

Phase 1: Pre-Evaluation

Before evaluation begins, establish:

Quality specifications

• Error categories to assess
• Severity weights for each category
• Passing threshold (e.g., MQM score ≥95)
• Sample size and selection method

Evaluator selection

• Native target language speakers
• Subject matter expertise (for specialized content)
• Trained in the evaluation methodology
• Independent from original translators

Evaluation setup

• Tools and templates
• Reference materials (glossaries, style guides)
• Calibration process

Phase 2: Evaluation

During evaluation, evaluators:

1. Compare source and target segments
2. Identify potential errors
3. Classify errors by type and severity
4. Document errors with annotations
5. Assign penalty points based on severity

Best Practice: Calibration

ISO 5060 emphasizes evaluator calibration. Before production evaluation:

• Have multiple evaluators assess the same content
• Compare results and discuss discrepancies
• Update guidelines based on alignment
• Document calibration decisions

Phase 3: Post-Evaluation

After evaluation:

Score calculation

Quality Score = 100 - (Total Penalty Points / Word Count × 100) 

Typical penalty weights:

• Critical: 25 points
• Major: 5 points
• Minor: 1 point

Reporting

• Overall quality score
• Error breakdown by category
• Error breakdown by severity
• Specific annotations with examples
• Trend analysis (if historical data exists)

Feedback loop

• Share findings with translators
• Identify systematic issues
• Update training materials
• Adjust workflows as needed

Implementing ISO 5060 in Your Organization

Step 1: Assess Current State

Evaluate your existing quality processes:

• What evaluation methodology do you use currently?
• How well does it align with ISO 5060 requirements?
• What gaps need to be addressed?
• Do you have trained evaluators?

Step 2: Define Quality Tiers

Not all content requires the same evaluation rigor. Create content tiers:

Step 3: Train Your Team

Ensure evaluators understand:

• ISO 5060 error categories and definitions
• Severity criteria and examples
• Scoring methodology
• Calibration processes
• Tool usage

According to ISO 5060, qualified evaluators should be able to "compare paired source and target language segments and judge translation quality based on MQM Error Typology criteria."

Step 4: Select Tools

Choose evaluation tools that support ISO 5060:

• Error annotation and categorization
• Severity assignment
• Score calculation
• Report generation
• Trend tracking

AI-powered tools like KTTC can automate much of the evaluation process while maintaining ISO 5060 compliance.

Step 5: Establish Calibration Processes

Regular calibration ensures consistency:

• Weekly: Quick alignment checks
• Monthly: Full calibration sessions
• Quarterly: Methodology review and updates

Step 6: Document Everything

Create and maintain:

• Evaluation guidelines
• Error examples database
• Calibration records
• Evaluator certifications
• Quality reports archive

ISO 5060 and AI-Powered LQA

The rise of AI in translation quality evaluation creates new opportunities and questions around ISO 5060 compliance.

Can AI Perform ISO 5060 Evaluation?

AI LQA tools can:

• Identify potential errors using NLP analysis
• Classify errors according to MQM categories
• Calculate quality scores automatically
• Generate reports at scale

However, AI currently cannot:

• Make nuanced cultural judgments
• Evaluate creative/marketing content reliably
• Serve as sole evaluator for critical content
• Replace human calibration oversight

Best Practice: Hybrid Approach

Combine AI efficiency with human expertise:

1. AI first pass - Automated evaluation at scale
2. Human verification - Review AI findings, especially critical errors
3. Random sampling - Human spot-checks on AI-evaluated content
4. Continuous calibration - Compare AI and human results, adjust AI models

This hybrid approach maintains ISO 5060 compliance while dramatically reducing evaluation time and cost.

Demonstrating ISO 5060 Compliance

For Language Service Providers

If you're an LSP, ISO 5060 compliance can be a competitive differentiator:

• Document your evaluation methodology alignment
• Train and certify evaluators
• Maintain calibration records
• Provide ISO 5060-compliant quality reports to clients

For Enterprises

If you're a buyer of translation services:

• Include ISO 5060 requirements in RFPs
• Require quality reports in ISO 5060 format
• Audit vendor compliance periodically
• Benchmark vendors using consistent metrics

FAQ

What is ISO 5060 in translation?

ISO 5060:2024 is the first international standard specifically focused on evaluating translation quality. It provides a framework for assessing translated content, including error categorization (aligned with MQM), severity levels, evaluation phases, and scoring methodologies. Organizations use it to standardize how they measure and report translation quality.

How is ISO 5060 different from MQM?

ISO 5060 is based on MQM (Multidimensional Quality Metrics) but is an official ISO standard. MQM is a flexible, open framework that organizations can customize. ISO 5060 formalizes key MQM concepts into an international standard, providing official guidance on implementation, evaluator qualifications, and evaluation processes. They're complementary—you can use MQM tools while following ISO 5060 methodology.

Is ISO 5060 certification available?

As of 2025, ISO 5060 itself doesn't have a certification program like ISO 17100. However, organizations can document their ISO 5060 compliance and include it in quality management systems. Some certification bodies may develop ISO 5060-aligned evaluator certification programs in the future.

What's the minimum score to pass ISO 5060 evaluation?

ISO 5060 doesn't mandate a specific passing score—that's determined by your organization based on content type and risk. Common thresholds are 98+ for critical content (legal, medical), 95+ for standard business content, and 90+ for low-risk internal content. The standard provides guidance on setting appropriate thresholds.

Can AI tools be used for ISO 5060 evaluation?

Yes, AI tools can assist with ISO 5060 evaluation, particularly for initial screening and scale. However, the standard emphasizes qualified human evaluators for final quality decisions, especially for critical content. A hybrid approach—AI-assisted evaluation with human oversight—is the 2025 best practice that maintains compliance while improving efficiency.

Conclusion

ISO 5060 represents a significant step forward for the translation industry. For the first time, there's an international standard that provides clear, consistent guidance on how to evaluate translation quality.

Whether you're an LSP looking to differentiate your services, an enterprise seeking to standardize vendor evaluation, or a quality manager building evaluation processes, ISO 5060 offers a solid foundation.

The key is to approach implementation practically: start with your highest-priority content, train your team on the fundamentals, and gradually expand coverage. Combined with AI-powered tools, ISO 5060-compliant evaluation can be both rigorous and efficient.

Ready to implement ISO 5060-compliant quality evaluation? Try KTTC for AI-powered LQA with MQM-based error categorization and automated quality scoring.