The obvious way to improve a machine translation is to run it past the model again and ask for a better version. It is one line of code and it feels like free quality.

It is not free, and recent measurement suggests it is often not quality either: a blanket second LLM pass over an entire document costs disproportionately more than it returns (arXiv:2601.19410). The sensible version of the idea is selective — edit only the segments a quality estimate says are weak. This post is about how to choose that threshold, what to measure, and the two ways the selective approach fails in practice.

Why the blanket pass disappoints

Three reasons, and the third is the one that surprises people.

Most segments do not need it. In a typical document the majority of machine-translated segments are fine. A second pass over them is spend with no possible upside.

The pass is not free of risk. A model asked to improve a correct translation will change it, because that is what it was asked to do. Some of those changes are neutral, some are worse, and on official documents "different" is itself a cost — a reviewer who has already approved a phrasing now sees a new one.

It hides where the problems are. After a blanket rewrite you cannot tell which segments were weak, because they all changed. You have destroyed the signal you would have used to improve the upstream step — the prompt, the glossary, the model choice.

Selective post-editing avoids all three, and produces a by-product worth as much as the edits: a list of which segments your pipeline struggles with.

The shape of a targeted pipeline

Four stages, and only the third costs real money:

Translate. Whatever your normal path is.
Estimate quality per segment. Cheap heuristics first, an LLM judgement only on what the heuristics flag. (We wrote about the two-tier setup separately; the same structure applies here.)
Post-edit the segments below the threshold. With the source, the current translation, and — this matters — the specific complaint from the estimator.
Re-estimate the edited segments only, so you can tell whether the edit helped.

Stage 3 is where the design decision lives. Giving the editor the reason the segment was flagged is what separates targeted post-editing from an expensive rephrasing service. "This is machine-translated, improve it" produces a rewrite. "The glossary term for this concept is X and the translation uses Y" produces a fix.

Setting the threshold

The threshold is not a quality question, it is a cost-asymmetry question. Write down two numbers before you pick it:

The cost of a false flag. One unnecessary edit call, plus the risk that a correct segment is changed for the worse.
The cost of a missed flag. A defect ships. On a customs declaration or a certificate, that is a document the recipient rejects.

For official documents the second cost dominates by a wide margin, which argues for a generous threshold — flag more, edit more, accept the waste. For high-volume marketing content the balance is the other way.

Then measure the thing most teams never look at: the escalation rate. What fraction of segments cross the threshold?

If the answer is 0%, your second stage is decorative and you are paying for an estimator that changes nothing. If the answer is close to 100%, you have re-invented the blanket pass with extra steps.

We hit the first failure. Our rule-based prefilter scored a batch of real machine-translated segments between 75 and 85 and escalated none of them. The LLM tier, when we forced it to look, scored thirteen of those fourteen segments at 100. The prefilter was conservative rather than wrong — it under-rated good work and passed nothing bad — but the practical effect was an expensive tier that never ran and users seeing pessimistic numbers on correct translations.

That is the failure mode to watch for: a threshold that is safe and inert at the same time.

What to measure, in order

1. Escalation rate. Per document type, not overall. Declarations and certificates behave differently.

2. Edit acceptance. Of the segments you post-edited, how many did the edit actually improve? Re-estimating only the edited segments gives you this almost free. If the number is low, your editor prompt is the problem, not your threshold.

3. Human agreement on the flagged set. Take a sample of flagged segments to a reviewer. If they disagree with most of the flags, the estimator is miscalibrated and no threshold will save it.

4. What the flags cluster on. This is the by-product, and over time it is the most valuable output. If 60% of your flags are glossary mismatches, the fix is not more post-editing — it is putting the glossary in the first prompt.

Two failure modes of the selective approach

The editor without a reason. If stage 3 receives only "this segment scored low", it will rewrite rather than repair. Pass the estimator's specific complaint through. This is also why an auditable estimator — one that emits error categories and spans rather than a bare number — is worth more than a slightly better-correlated one.

The threshold that drifts. Change the model, the prompt or the glossary, and the score distribution moves under you. A threshold set against last quarter's distribution can silently go inert, which looks exactly like "quality improved". Re-check the escalation rate whenever an upstream component changes, and alert on it going to zero.

Where post-editing is the wrong tool entirely

For official documents, some defects should never reach a post-editor:

Missing numbers. Check digit runs deterministically. A comparison costs nothing and does not have opinions.
Untranslated labels. Sweep the target for source-script characters. Also free.
Glossary mismatches. Check the terms directly rather than asking a model whether the terminology feels right.

Every one of these is cheaper, more reliable and more explainable as a deterministic check than as an LLM pass. Reserve post-editing for what genuinely needs judgement: awkward phrasing, register, ambiguity the first pass resolved badly.

The general principle: do not spend a model call on a question a comparison can answer.

Key Takeaways

A blanket second pass costs disproportionately more than it returns, risks changing correct segments, and destroys the signal about where your pipeline is weak.
Give the editor the reason, not just the score. Without the specific complaint, stage 3 rewrites instead of repairing.
Set the threshold from cost asymmetry, not from a quality target — on official documents a missed defect dominates a wasted call.
Measure the escalation rate per document type. Zero means your expensive tier is decorative; near-100% means you rebuilt the blanket pass.
Deterministic checks first. Numbers, untranslated labels and glossary terms are comparisons, not judgements, and should never consume a model call.

FAQ

What is targeted post-editing?

Running an automatic post-editing pass only on the segments a quality estimator flags as weak, rather than over the whole document. It captures most of the benefit at a fraction of the cost, and produces a list of where the pipeline struggles as a by-product.

How do I choose the quality threshold?

From the asymmetry between a false flag (one wasted edit, plus the risk of changing a correct segment) and a missed flag (a defect ships). For official documents the second cost dominates, so a generous threshold is right. Then verify by measuring what fraction actually escalates.

What escalation rate should I expect?

There is no universal number, but 0% and near-100% are both wrong. Zero means the estimator changes nothing and your expensive tier is decorative; near-100% means you have rebuilt the blanket pass with extra steps. Measure it per document type.

Can post-editing fix missing numbers?

It can, but it should not have to. Numeric integrity is a deterministic comparison of digit runs between source and target — it costs nothing, cannot hallucinate, and catches the defect class that matters most on declarations and certificates. Post-editing is for judgement calls.

What if the estimator and the reviewers disagree?

Then the estimator is miscalibrated, and no threshold will fix it. Sample flagged segments, take them to a reviewer, and measure agreement before tuning anything else — a threshold on top of a bad signal just picks a different subset of noise.

Conclusion

Selective post-editing is a cheaper architecture than the blanket pass, but that is not its main advantage. The main advantage is that it forces you to have a quality estimate you trust — and once you have one, the list of what it flags tells you what to fix upstream.

The teams that get the most from this are the ones who treat the flags as a bug report on their own pipeline rather than as a work queue. Editing a glossary mismatch is a fix for one segment. Putting the glossary in the prompt is a fix for all of them.

If you want document translation where quality estimation drives what gets a second look, try KTTC.