The pitch is easy to like. One model translates, a second reviews, a third edits, and quality goes up because that is how human translation teams work. It is intuitive, it demos well, and it is being sold hard.

The arithmetic is less popular. A sequential translator→reviewer→editor pipeline costs roughly five times the tokens of a single call; an iterative loop that keeps going until it converges costs about fifteen (arXiv:2505.01560). That is the price. This post is about what the price buys, according to people who measured it — and what happened when we looked at our own agentic pipeline and decided against it.

The three shapes, and what each multiplies

Single call. One prompt, one output. Baseline: 1×.

Sequential agents. Translate, then review the translation, then edit per the review. Each stage re-reads the source and the previous stage's output, so the input tokens compound faster than the stage count suggests. Measured at roughly 5×.

Iterative agents. The review-and-edit loop repeats until some convergence test passes or a cap is hit. Roughly 15×, and — importantly — the variance is high, because a document that never satisfies the reviewer costs the cap every time.

The multiplier is on tokens, not on wall clock, and latency multiplies too. For a document pipeline where a customer is watching a progress bar, three sequential model calls per segment is a different product than one.

What the multiplier buys, measured

Here the results get uncomfortable for the pitch.

Against mature neural machine translation, agentic systems lost on automatic metrics in 7 of 12 language-pair-and-domain combinations. Not "won by less than expected" — lost, to a specialised NMT system costing a fraction as much.

Against a single LLM call, agentic systems lost on human evaluation in 5 of 6 comparisons. That is the finding that should give a buyer pause, because human evaluation is the metric the agentic pitch implicitly appeals to: machines miss nuance, so add a reviewer.

The honest reading is not "agents are bad." It is narrower and more useful:

The gains are real on long-form literary text, where consistency across a whole work matters and a reviewer with the full context can catch drift a segment-level translator cannot.
The gains are real on legal nuance, where a second pass specifically instructed to check for altered obligations catches things a single pass does not.
Everywhere else, the multiplier buys variance.

Why a reviewer often makes things worse

The counter-intuitive part deserves an explanation, because "add a checker" sounds like it can only help.

A reviewer stage is not a validator. It is another generation, and it is rewarded — by its prompt — for finding something to say. Give a model a good translation and ask what is wrong with it, and it will tell you what is wrong with it. The editor then applies that advice. Two of the three stages are actively pushing the output away from a translation that was already correct.

This is the same failure shape as an over-eager linter: the cost of a false positive is not zero, because something downstream acts on it.

The fix, where the pattern does pay, is to make the reviewer's job falsifiable: check that these specific numbers appear, that these terms match the glossary, that no obligation changed polarity. A checklist, not an opinion. That is a much cheaper thing than an agent, which is rather the point.

What we did with ours

We had one. agent_pipeline.py plus its prompts and types came to 668 lines, wired into the translation service with an import, a constructor line, a call and two methods.

It ran when translation_mode == "agentic". The API never accepted that value.

So: 668 lines of agentic translation, in the repository for months, executed exactly zero times in production. Nobody noticed, because nothing depended on it and nothing tested it.

We deleted it. The research pass that preceded the deletion recommended salvaging the reviewer loop for a different purpose — verifying extracted fields against a schema. We read the code to do that and found nothing to salvage: the reviewer compared a fuzzy translation-memory match against a source sentence, which is a different problem than checking a value against a declared field. The idea survives as a recommendation in our own research notes. The code was not needed for it.

That is the most common real state of an agentic pipeline, in our experience: not "expensive", but unmeasured and unreached, kept because deleting it feels like admitting something.

How to decide, in four questions

If someone is selling you an agentic translation layer, or you are considering building one:

1. What is the baseline, measured on your content? Not on WMT. On your documents, with your glossary, scored the way you actually judge quality. Most agentic wins evaporate against a well-prompted single call with a glossary in it.

2. What is the reviewer allowed to say? If the answer is "anything", expect regressions. If the answer is a checklist of falsifiable properties, you may not need an agent at all — you need a validator.

3. What does a failed convergence cost? For iterative loops, price the worst case, not the average. The documents that loop are the hard ones, which is to say the expensive ones, which is to say the ones you have a lot of.

4. Is the win concentrated? If the gain is real on 5% of your documents, run the expensive path on that 5%. Selective escalation captures most of the benefit at a fraction of the multiplier — and unlike an agent framework, it is a routing rule you can write this afternoon.

The cheaper thing that usually wins

For document translation specifically, the interventions that measurably paid, in our own numbers, were all boring:

A glossary in the prompt, which is one extra input block, not one extra model call.
Turning off a reasoning model's thinking for mechanical work, which is a flag.
A deterministic numeric check after translation, which is a comparison of digit runs and costs nothing.
Not re-extracting a document already extracted, which is a cache.

None of those are agents. All of them are cheaper than one extra pass, and each of them fixes a defect class rather than nudging an average.

Key Takeaways

Sequential agents cost about 5× the tokens; iterative loops about 15×, with high variance on the documents that fail to converge.
The measured results are not kind to the pitch: agentic systems lost to mature NMT on automatic metrics in 7 of 12 combinations, and to a single LLM call on human evaluation in 5 of 6.
A reviewer stage is another generation, not a validator. It is rewarded for finding something to say, and the editor then acts on it.
Where the pattern pays, make the reviewer falsifiable — a checklist of properties, which is cheaper and more reliable than an agent.
Escalate selectively. If the win is on 5% of documents, spend the multiplier on that 5%.

FAQ

How much more does agentic translation cost?

Roughly five times the tokens for a sequential translator-reviewer-editor pipeline and about fifteen times for an iterative loop, per published measurements. Latency multiplies similarly, which matters for any pipeline a user is watching.

Does a reviewer agent improve translation quality?

Sometimes, and not by default. In published comparisons a single LLM call beat agentic systems on human evaluation in five of six cases. A reviewer prompted to find problems will find problems in a correct translation, and the editor stage then acts on that advice.

When is agentic translation actually worth it?

On long-form literary text, where whole-work consistency matters, and on legal text where a second pass can be pointed at specific risks like altered obligations. Both are cases where the gain is concentrated, which argues for routing those documents to the expensive path rather than running it on everything.

What is a cheaper alternative to a reviewer agent?

A validator: a deterministic check for the properties you actually care about — numbers present, glossary terms matched, no polarity flips. It costs nothing per document, cannot hallucinate a complaint, and catches the defect classes that matter on official documents.

How do we know an agentic pipeline is being used at all?

Instrument it. Ours was 668 lines gated behind a mode value the API never accepted, so it ran zero times in production and nobody noticed. Token telemetry per call path answers the question in a day.

Conclusion

"Add more model calls" is a hypothesis, not an architecture, and it is testable for the price of one afternoon: run your own content through a single well-prompted call and through the agentic path, and score both the way you actually judge quality.

Our own experience is a small, embarrassing version of the same lesson. We had the pipeline, we never ran it, and when we finally read it in order to reuse its best part, that part turned out to be solving a different problem. The measurement that mattered was not "is it better" — it was "is it running", and we could not answer that either.

If you want document translation where the expensive steps are the ones that were measured, try KTTC.