The model read the document correctly. It found the address, the area, the registration number. Then it returned them under the keys location, area, and reg_no — and your pipeline, which was looking for legal_address, area_sqm and registration_number, rendered three blank fields.

This is the shape of most LLM extraction failures. Not comprehension — format. The model knew the answer and described it in its own words, and your code has no way to tell that location was meant to be legal_address.

This guide covers three layers of defence against that, in increasing order of cost, and where each one stops being worth it.

The three ways format breaks

Invented key names. The most common by a wide margin. Your prompt describes the fields in prose — "extract the legal address, the area in square metres" — and the model picks plausible JSON keys on its own. They are never the same twice, and they never match the names your template expects.

Almost-JSON. A trailing comma, a stray markdown fence, a comment, a truncated object because the response ran out of budget. Your parser raises, and the retry produces almost-JSON with a different flaw.

Silently missing fields. The model omits a key rather than reporting that the document has no value for it. An absent key and an empty value look identical downstream, so nobody can tell a genuine blank from a miss.

All three are format problems with format solutions.

Layer 1: Pin the key names in the prompt

This is the cheapest and by far the highest-yield fix. Stop describing fields in prose and start listing the exact keys.

The instruction that worked for us appends an explicit key contract to whatever the prompt already says:

Return a JSON object that uses these EXACT key names
(do not rename, translate, or abbreviate them):
legal_address, area_sqm, registration_number, cadastral_value.

Spell each key exactly as listed even when the document labels the value
differently (e.g. use the listed key, not a synonym). Omit a key only when
the document genuinely has no value for it.

Three details matter more than they look.

"Do not translate." Extraction runs on documents in one language and reports in JSON, and a model reading a Russian form will happily produce Russian key names unless told otherwise.

"Even when the document labels the value differently." This is the sentence that stops synonym invention. Without it the model reasonably concludes that a field labelled «Местоположение» in the form should be keyed location, because it is reading the document, not your schema.

"Omit a key only when there is genuinely no value." This converts a silent miss into an honest absence — and makes the two distinguishable when you validate.

Generate this list from your schema, not by hand. Ours is built from the same field_definitions the output is mapped onto, which means the prompt cannot drift away from what the pipeline expects, no matter how the human-written part of the prompt is phrased.

For array fields, pin the per-item keys too, or repeating rows fill unevenly:

Array fields:
  - "items" is an array; every element must use EXACTLY these keys:
    description, quantity, unit_price, total_price

Layer 2: Provider-native structured outputs

Most providers now accept a JSON schema alongside the request and constrain decoding so the response conforms. When available, this eliminates the almost-JSON class entirely — not by validating afterwards, but by making the malformed output unreachable.

Support is uneven and worth checking rather than assuming. In our testing OpenAI and Anthropic both honour a schema properly; DeepSeek's support is weaker. One thing worth knowing if you extract from scans: schema constraints work alongside image inputs, so vision extraction is not excluded from this layer.

Do not type your values unless you are certain you want to.

This is the trap, and it cost us a careful afternoon to see clearly. Our own field definitions declare unit_price and total_price as number. Our prompt requires them as strings — "49.0000", explicitly not 49.0 — because a customs declaration's trailing zeros are part of the value, and a JSON number silently discards them. 49.0000 becomes 49.0, and a figure that must match a paper form to four decimal places no longer does.

The same conflict appeared on quantity, declared as a scalar but required by the prompt to be an array of strings, duplicates included, because a form cell can genuinely hold three lines.

So our schema pins key names and object shapes but does not type the values. Every scalar crosses the wire as a string. Two tests guard it: no numeric type appears anywhere in the generated schema, and every scalar position also accepts an array of strings. Loosening either is a direct path to corrupted sums.

The general rule: a schema is a format contract, not a validation layer. Use it to guarantee the shape. Validate the values yourself, where you can decide what to do about a bad one.

Layer 3: Re-ask about the fields that failed

Even with both layers, some documents come back with a field missing or malformed. The reflex is to retry the whole extraction. That is the wrong move on both cost and quality: you pay for the entire document again, and a blind retry on a reasoning model tends to hit the same wall it hit the first time.

Better: validate the result, collect the specific fields that failed, and ask about only those.

The following required fields were missing or invalid in your previous answer:
  - export_date (required, not present)
  - total_value (present but not a string)

Look at the document again and return a JSON object containing ONLY these
keys. If the document genuinely has no value for a field, return null for it
rather than guessing.

The last sentence is essential. A re-ask without it is an invitation to fabricate: you have told the model it got something wrong, and the path of least resistance is to invent something plausible. Explicit permission to answer "not present" is what makes the second pass safe.

Then merge under a rule you write down before you run it. Ours: the merge is accepted only if wrong answers do not increase and missing answers decrease. A second pass that trades one miss for one error is not an improvement, and without a stated rule it is very easy to convince yourself it is.

Measure before you switch it on

We built this loop, and then we measured it, and the measurement said: not yet.

On the one document we could reach, the baseline extraction was already 85 correct out of 85 leaf values — both company codes, a three-line quantity cell with a duplicate, every price with its trailing zeros. The verification pass produced exactly the same 85. It flagged one field, export_date, and the corresponding box on the form is genuinely empty. The loop asked its one question, got a correct silence, and applied nothing. One extra call, zero changes.

By our own merge rule, that arm does not pass. So the code is written, tested, and switched off behind a flag until we have documents with real misses to measure it on.

Two things worth taking from that:

Establishing the baseline took four rounds of counting, and every discrepancy turned out to be an error in our reference data, not in the extraction. We had mis-mapped which form box a field corresponded to, missed that a code sat on the second line of the same cell, overlooked a printed prefix. Check your yardstick as strictly as the thing you are measuring.

A validator's count is an upper bound, not an estimate. Ours reports missing values on 10 of 27 stored documents — but without the source files, a genuine miss is indistinguishable from an empty box on the form. The one case we could check against its source turned out to be an empty box.

Key takeaways

Extraction fails on format, not comprehension. The model usually found the value; it just filed it under a name your code does not know.
Pin exact key names in the prompt, generated from your schema. This is the cheapest layer and it removes the largest failure class.
Use provider-native schemas for shape, not for types. Typing a value as number will silently eat the trailing zeros that make a customs figure match its paper form.
Re-ask only about the fields that failed, and explicitly permit "not present." A blind full retry costs more and invites fabrication.
Write the merge rule before the measurement, and check your reference data first. Four of our first four discrepancies were errors in the reference, not the extraction.

FAQ

Does a JSON schema make prompt-level key pinning unnecessary?

No. Keep both. Schemas are not universally supported, support varies in quality, and the prompt instruction is nearly free. When both are present they reinforce each other; when the schema is unavailable, the prompt is all you have.

Why not just type the numeric fields as numbers?

Because a JSON number discards formatting the document treats as meaningful. 49.0000 becomes 49.0, and a figure that must match a form to four decimals no longer does. Carry values as strings and validate them yourself.

What should I do about a field that is missing from the response?

First determine whether it is missing from the document. Those are different facts and only one of them is a bug. Instruct the model to return null for a genuine absence, so your validator can tell them apart.

Is a verification pass worth the extra call?

Only if you measured that it helps. Ours produced zero corrections on a document where the baseline was already perfect, so it stays off. Measure on documents with real misses — and be sure they are real misses and not empty boxes.

How do I test extraction without paying for every run?

Render the same page images and the same prompt to disk, and drive the arms from there rather than through the live API. Our harness stores what the production call would send, which makes the comparison reproducible and keeps the paid calls down to the handful that actually establish something.

Conclusion

Structured extraction is mostly a contract problem. The model is usually reading the document correctly; the failure is in the handshake between what it returns and what your code expects.

Fix that handshake in the order the layers cost: pin the keys from your schema, add a native schema for shape where the provider supports it, and add a targeted re-ask only once you have measured that it recovers something real. And whatever you build, establish the baseline honestly first — the most expensive extraction bug we investigated turned out, four times running, to be a mistake in how we were counting.

KTTC extracts fields from customs declarations, registry extracts and civil records, where a dropped code is a rejected document. Try KTTC and see what comes back from your own forms.