When cheque fields disagreed, AI sometimes repeated one amount—how serious is the risk?
Two multimodal models were tested on 841 selected cheques. Under one redaction condition, predicted-field agreement rose sharply, but the study does not prove a live banking error or a universal copying mechanism.
The short version
- What happened
- The study evaluated GPT-4.1 and Claude 4.5 Sonnet on 243 primary cheques plus 598 strict mismatches from two Indian datasets.
- Who it affects
- A mistaken amount extraction can delay a payment, create a costly exception or move the wrong sum unless the two fields and the underlying image are checked separately.
- What remains unproven
- The unreviewed study tested two models on selected public datasets, used a blank-as-zero convention for part of the primary cohort and did not measure live bank losses, fraud or workflow outcomes.
Keep in mind
The unreviewed study tested two models on selected public datasets, used a blank-as-zero convention for part of the primary cohort and did not measure live bank losses, fraud or workflow outcomes.
Source analysis · 1 linked source. Primary analysed source date: 19 October 2026.
Read the full report · 3 min The evidence, methods, implications and limitations
Visual ambiguity changed whether the two extracted amounts agreed
BNY-affiliated researchers tested whether multimodal models confuse the numeric courtesy amount on a cheque with the written legal amount. Their primary cohort contained 243 selected cheques: 124 had two observed unequal amounts and 119 had one blank annotation represented as zero. Two additional Indian datasets supplied 598 strict mismatches, for 841 evaluated cheques in total.
On full images, unexpected matching predictions occurred in 4.0% to 9.7% of the 722 strict two-field mismatches across models and datasets. When the courtesy field was redacted while the legal field remained visible, agreement rose to 73.7% for GPT-4.1 and 49.0% for Claude on the broader 243-case cohort. That sensitivity is a warning about visual context, but matching predictions do not prove that one field was literally copied into the other.[1]
A small prompt intervention helped unevenly
The researchers also supplied small example tables as a single image. Claude's best condition improved joint extraction accuracy by 11.5 percentage points on all 243 cheques, but its gain on the stricter 124-case subset was not statistically significant after correction. GPT's effects depended on the example set. Model-reported confidence could remain high on wrong or agreeing outputs, so confidence alone was not a reliable safety check.
For banks and payment processors, the practical response is field-level validation: compare numeric and written amounts, preserve image context and route disagreements or obscured fields to human review. The public cohorts are small, selected and handwriting-specific; the primary cases came from a training split, and confidential production data were unavailable. Independent tests on held-out operational documents and measured payment errors would materially change the assessment.[1]
The distinction that matters
Why two cheque fields need separate validation
A cheque expresses one amount twice, but each field can be ambiguous for a different reason.
Courtesy amount
Numeric field is read
Digits may be obscured or stylized
Legal amount
Written field is read
Words provide a second signal
Cross-check
Predictions and image regions are compared
Mismatch triggers review