Your LLM Extracted 10,000 Numbers. Which Ones Are Wrong?
I tested a way to find out without labelling a single document — on 535 real receipts
You point an LLM at 10,000 documents and ask it to pull out a number.
It returns 10,000 numbers. Every one of them confident. Some of them wrong.
You don’t know which ones. And you can’t check by hand, because checking by hand is the entire thing you automated away.
This is the most uncomfortable fact about shipping LLM extraction in production, and most teams handle it the same way: they don’t think about it too hard. They spot-check twenty documents, the twenty look fine, and the pipeline goes live.
The textbook answer is to build a labelled test set. Pay humans to annotate a few thousand documents, measure accuracy, ship with a number. That works. It also costs weeks and real money, it goes stale the moment your input distribution shifts, and it tells you nothing about the weird new document arriving tomorrow.
There’s an older idea that sidesteps this entirely. I wanted to know whether it survives contact with real, messy data — not whether it fires, but whether what it fires on is genuinely wrong.
So I ran it on 535 real scanned receipts and measured exactly that.
The trick: you don’t need the truth to catch a contradiction
Ask someone the same question three different ways. If the story changes, they’re unreliable — and notice that you worked that out without knowing what actually happened.
You can do this to a model.
Take a receipt. Shuffle the order of the lines. Add a “THANK YOU, PLEASE COME AGAIN” footer. Strip the currency symbols. None of that changes what the customer actually paid.
Run your extractor on each version.
If it returns two different totals for what is plainly the same receipt, it just caught itself. No answer key. No labelling budget. No reference data of any kind.
This is metamorphic testing, a technique built for exactly this situation — formally, the oracle problem: you can’t tell whether a single output is correct, so instead you assert relations that must hold between outputs when the input changes in a known way.
- Reorder an invoice’s line items → the total must not change
- Add an irrelevant sentence to a support ticket → the priority must not change
- Double every quantity → the total must double
- Rephrase a clause without changing its meaning → the extracted terms must not change
Every one of those is checkable with zero ground truth.
This is not a new idea, and I want to be upfront about that, because the interesting part of what follows isn’t the technique. Hyun et al. built METAL, a metamorphic testing framework for LLMs. Cho, Ruberto and Terragni went further: they reviewed 1,024 papers, catalogued 191 distinct metamorphic relations across 24 NLP tasks, and released LLMorph, which implements 36 of them. Their evaluation ran over 561,000 test executions against GPT-4, Llama3 and Hermes 2, finding an average failure rate of 18%. Giskard, a commercial ML-testing company, markets the approach too.
So the method is established and the tooling exists. Here’s the question I couldn’t find anyone answering:
When one of these tests fires, how often is the output actually wrong?
Failure rates tell you how often a relation is violated. They don’t tell you whether a violation predicts a real error. That gap matters enormously in practice — because if you’re going to hand a human a list of flagged documents to review, you need to know what fraction of that list is worth their time.
That’s the number I set out to measure.
The setup
I used the ICDAR-2019 SROIE dataset: 626 real scanned receipts with OCR text and ground-truth totals. Public, downloadable, genuinely messy. After filtering to receipts with a clean numeric total, 535 remained.
These are harder than they sound. The average receipt carries 13 dollar-shaped numbers — one had 49 — and exactly one is the real total. The rest are line prices, discounts, tax lines, cash tendered, change.
And the traps are vicious:
SUB-TOTAL (EX)
TOTAL TAX
TOTAL QTY: 1.00
TOTAL SALES (EXCLUDING GST) : 31.03
TOTAL SALES (INCLUSIVE OF GST) : 32.70
ROUNDING ADJUSTMENT
TOTAL : 31.03
Six lines containing the word “TOTAL.” One is what the customer paid. The bottom TOTAL : line — the one that looks most authoritative — is a reprint of the pre-tax figure.
I wrote a receipt extractor. Not a strawman: the kind of cue-matching heuristic a competent team actually ships, and the kind of thing an LLM prompt converges to on its own. Then I ran the experiment in two strictly separated stages.
Stage 1 — blind. Look at zero correct answers. Apply the transformations. Flag every receipt where the extractor contradicts itself.
Stage 2 — audit. Only now open the answer key. Not to find errors — purely to score whether the blind flags were actually wrong.
That separation is the whole experiment. If any label leaks into Stage 1, the result means nothing. (The test suite enforces it: strip the ground-truth field entirely and Stage 1 produces identical flags. Any label read would throw.)
The part where I was wrong
My first run flagged 60% of receipts. Sixty percent! Massive detection! Ship it!
It was garbage.
My shuffle was too violent. I was scattering every line independently, which destroyed the natural adjacency between a label and its value. On a receipt where TOTAL: sits directly above 32.70, ripping those apart doesn’t test robustness — it destroys information any correct extractor legitimately needs.
I was penalising the model for something no correct system could survive.
So I rebuilt it. Instead of shuffling all lines, permute semantically independent blocks: header, line items, totals section, footer. A receipt’s totals block genuinely could appear before or after its item list — block order carries no meaning. Local structure stays intact.
Flags collapsed from 72 to 4.
Sixty-eight of my seventy-two “findings” were artifacts of my own unfair test.
I thought this was my personal embarrassment until I read the LLMorph evaluation. They hand-analysed 937 metamorphic violations and found false positive rates ranging from 0% to 70% depending on which relation was used. Same phenomenon, measured at scale: a badly-chosen relation generates mostly noise.
Which is the lesson I’d keep if you forget everything else here:
A test that any correct system would fail isn’t a bug detector. It’s a broken test.
The engine is about fifty lines of code. It’s trivial. The hard part — the only hard part — is knowing which transformations are genuinely harmless. That’s not software engineering. That’s domain knowledge, and there’s no shortcut to it.
One filter, 38% of all failures
While fixing the extractor I hit something every engineer will recognise.
I had a list of “trap” strings — lines containing the word TOTAL that aren’t the payable total. SUBTOTAL. TOTAL QTY. And GST (the sales tax), because tax lines are not totals.
That last one was silently destroying the correct answer on 45 of 62 failures.
Why? Because the real total on these receipts is frequently printed as:
TOTAL SALES (INCLUSIVE OF GST) : 32.70
My defensive filter saw GST, decided “tax line, skip it,” and threw away the exact line I was looking for. A guard rule, added to prevent errors, causing 73% of them.
Fixing that one rule took accuracy from 48% to 87%.
But then three receipts that had been correct broke. My new rule preferred the GST-inclusive line — and on these three, the inclusive figure was pre-rounding:
TOTAL INCL GST 64.13
ROUNDING 0.02
TOTAL 64.15 ← what was actually paid
The fix isn’t a special case. It’s a magnitude rule: a stale pre-tax reprint differs from the inclusive figure by a full tax delta (1–6 currency units), while a rounded final differs by at most one rounding step (0.05). Different by an order of magnitude, cleanly separable.
I mention this because it’s the difference between publishing 87% and publishing a number that quietly embarrasses you. Check whether your fix broke something that used to work. Three receipts is a rounding error until someone runs your code on their data.
The results
Here’s where I have to be careful, because this is exactly where articles like this usually start lying.
I developed my extraction rules against the first 120 receipts. So I split the results:
Ignore the dev row. Two flags is not a result — 100% precision on n=2 is noise wearing a suit, and I’d be embarrassed to headline it. It’s in the table because leaving it out would be worse.
The held-out row is the real one. Accuracy drops from 87% to 72% on receipts I never iterated against. That 15-point gap is the honest generalisation cost, and any single blended number would have hidden it.
And the method still works on data it never saw:
Flagged receipts were 2.7× more likely to be wrong. 73% precision — roughly three in four flags were real errors. Found with zero labels.
That’s the number I went looking for. Not “the relation fired 18% of the time,” but “when it fired, three quarters of the time something was genuinely broken.”
Now the caveat that matters most:
Recall is low — about 7%. It flagged 13 of 535 receipts and caught 10 of roughly 130 total errors. This is not a comprehensive net. It’s a high-precision spot-checker.
That sounds like a weakness. In practice it’s the useful shape. Nobody can hand-review 10,000 outputs. But a team absolutely can review the 2% that get flagged — and when three in four are genuine errors, that review is worth someone’s afternoon.
The pitch isn’t “AI finds all your bugs.” It’s “here are the 2% worth a human’s attention, and most of them are real.”
What it actually caught
The survivors are the interesting part. The strongest catches were columnar receipts — where OCR reads the label column and the value column as separate runs of text.
Receipt 026, from a stationery shop:
SUB-TOTAL (EX)
TOTAL TAX
ROUNDING
TOTAL
CASH
CHANGE
144.68
144.68
8.68
-0.01
153.35
153.35
0.00
The word TOTAL and the number 153.35 never appear on the same line. They’re not even close together. The extractor has to know that the fourth label maps to the fifth value — and it doesn’t. It returns 144.68 in one block order and 8.68 in another.
Two different answers for the same receipt. That’s a contradiction, and it took no answer key to find it.
The ground truth, when I finally looked, was 153.35. Wrong and inconsistent — but the inconsistency is what surfaced it, blind.
And one it got wrong. Receipt 225 was flagged, but the base extraction was correct — the extractor happened to land on the right number and was merely fragile about it. That’s a false positive, it’s why precision is 73% and not higher, and it’s visible in the tool’s own output. If it weren’t there, you should trust the other numbers less.
Using it on your own system
I packaged the engine as wobbly — because that’s what it detects. A system that gives different answers to the same question is wobbly, and wobbly is a bug even when the answer happens to land right.
pip install wobbly
Code, data, and the full experiment: github.com/tarunagarwal1981/wobbly
It knows nothing about receipts and never looks inside your system. You give it two things:
- Your system — any function that takes an input and returns an answer. An LLM call, a parser, a whole pipeline. Black box.
- A list of things that shouldn’t change the answer.
Here’s a complete, runnable example — a deliberately naive extractor that takes the last “total”-ish line it sees, which makes it order-dependent:
import random
from wobbly import check, Relation, unchanged
def extract_total(receipt):
total = None
for line in receipt["lines"]:
if "total" in line.lower():
amounts = [w for w in line.split() if w.replace(".", "").isdigit()]
if amounts:
total = float(amounts[-1])
return total
rng = random.Random(0)
def shuffle_lines(receipt):
lines = list(receipt["lines"])
rng.shuffle(lines)
return {"lines": lines}
reorder = Relation(
name="reorder lines => total unchanged",
transform=shuffle_lines,
assertion=unchanged(),
)
receipt = {"lines": [
"Flat White 4.50",
"Muffin 3.00",
"SUBTOTAL 7.50",
"TOTAL 7.95",
"CASH 10.00",
]}
report = check(extract_total, receipt, [reorder], samples=20)
print(report.summary())
Output:
BROKE (1 of 5 trials):
[reorder lines => total unchanged] expected 7.95 to be preserved, got 7.5
It found the bug in five trials. The extractor picks up SUBTOTAL when the lines arrive in a different order — and nothing in that code ever knew the right answer was 7.95.
There’s a built-in pack too, if you’re working with receipt-shaped documents:
from wobbly import check, default_pack
report = check(lambda r: extract_total(r), receipt, default_pack())
One contract to know: base_input, your system, and every transform must all agree on the same input shape, because checkfeeds each transformed input straight back into your system.
Crucially, it modifies the input before your code ever sees it. Nothing is injected into your prompt. Your function receives what looks like an ordinary document and does what it always does. That’s why it works on anything — LangChain pipeline, raw API call, fine-tuned model, regex parser. It doesn’t care.
Swap receipts for support tickets, contracts, or lab reports and the same code works unchanged. What changes is the list of harmless transformations — and that list is where your domain expertise lives.
If you want breadth of pre-built relations rather than a minimal engine, use LLMorph — 36 relations across four NLP tasks, plus a catalogue of 191 more in their paper. I built something smaller because I wanted to test document extraction pipelines, not NLP benchmarks, and I wanted the whole thing auditable in one sitting.
One honest cost: each relation costs at least one extra call to your system. Randomised relations (like reordering) run several times to explore different permutations; deterministic ones (adding a footer, stripping currency symbols) run once. You wouldn’t run this on every production document. Run it in CI on 50 representative documents whenever you change a prompt or a model version, or sample 2% in production. It’s a testing cost, not a runtime cost — the same way you don’t run your unit tests on every user request.
Where this doesn’t apply
The boundary matters more than the technique.
If you can compute the right answer, compute it. If a field must be a valid date, check it’s a valid date. If line items must sum to the total, sum them. If physics gives you an expected value, use the physics. Metamorphic testing is strictly worse than a direct check — it only tells you the system disagreed with itself, not what the right answer was.
Self-consistency is what you reach for when no formula exists. Which, for anything involving reading a document, is most of the time. There is no equation that tells you whether an LLM read a scanned receipt correctly. The only ground truth is the paper, and if you had someone checking every piece of paper you wouldn’t need the model.
That’s the gap this fills — and only that gap.
The takeaway
Metamorphic flags predict real errors — but only some of them. 2.7× enrichment, 73% precision, 7% recall, on held-out data. High-precision spot-checking, not comprehensive coverage. That’s a useful tool and a modest claim, and I’d rather publish the modest version.
Your test is a hypothesis too. My first version was 94% wrong and looked like a triumph. The academic work found false positive rates up to 70% depending on the relation. Sanity-check the test before you trust its output.
The engine is the easy part. Fifty lines. What’s hard is knowing which changes are genuinely harmless in your domain — and that’s exactly the thing you already know and a generic tool never will.
Everything here is reproducible:
git clone https://github.com/tarunagarwal1981/wobbly
cd wobbly
python scripts/run_blind.py
No API key, no network — you get the exact numbers in that table. pip install -e “.[test]” && pytest runs the suite, including a test that strips the ground-truth field entirely and confirms the blind stage produces identical flags. A second script rebuilds the dataset from the original ICDAR source and asserts byte-identity against the committed slice, so you can verify I didn’t cherry-pick which receipts to include.
References
- Chen et al., Metamorphic Testing: A Review of Challenges and Opportunities, ACM Computing Surveys, 2018
- Barr et al., The Oracle Problem in Software Testing: A Survey, IEEE TSE, 2014
- Cho, Ruberto & Terragni, LLMorph: Automated Metamorphic Testing of Large Language Models, arXiv:2603.23611
- Hyun, Guo & Babar, METAL: Metamorphic Testing Framework for Analyzing Large-Language Model Qualities, 2024
What invariants hold in your domain? I’m collecting the ones that actually catch production bugs — if you’ve got one, I’d like to hear it.
Your LLM Extracted 10,000 Numbers. Which Ones Are Wrong? was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.
