Can an AI explanation survive model compression?
A study of three image models found that INT8 conversion could preserve predictions while moving the pixels highlighted as important. A poor target-layer choice also created an apparent explanation collapse that disappeared when corrected.
The short version
- What happened
- The study used a 10-class, 10,407-image rice-leaf dataset, three image architectures, four explanation methods and 3,240 paired saliency comparisons per operating point on a stratified 320-image subset.
- Who it affects
- People may see a rationale that changed during deployment even when the visible prediction stayed the same.
- What remains unproven
- The main evidence comes from one rice-leaf benchmark and three convolutional architectures.
Keep in mind
The main evidence comes from one rice-leaf benchmark and three convolutional architectures.
The explanation analysis used fake quantization rather than every physical accelerator and runtime.
Similarity between saliency maps does not establish that either map is faithful, causal or useful.
Full-validation per-image records were unavailable, limiting independent verification beyond the primary subset.
No end user was studied and no real-world outcome was measured.
Source analysis · 1 linked source. Primary analysed source date: 11 October 2026.
Read the full report · 6 min The evidence, methods, implications and limitations
The answer: often partly, but accuracy does not prove it
A peer-reviewed study finds that converting image classifiers from 32-bit floating-point arithmetic to simulated 8-bit integer arithmetic can change the regions that explanation tools mark as important, even when the model keeps the same prediction. After the researchers corrected a target-layer problem, average overlap between the most salient pixels in the full-precision and quantized models ranged from 0.366 to 0.790 across architecture-and-method combinations. Median rank correlation across saliency maps was 0.850. Those figures indicate meaningful preservation in some settings, not identity.
This matters because compact models are commonly optimised before they reach a phone, camera or farm device. Teams often validate file size, latency and classification accuracy, then assume an explanation generated after conversion has the same meaning as one produced during development. The paper provides evidence against that shortcut. It also shows why explanation audits need controls: an apparently catastrophic failure came from selecting the wrong internal layer, not from quantization itself.[1]
What the researchers compared
The experiment used the public Paddy Disease Classification dataset, containing 10,407 images across 10 classes. EfficientNetV2-S, ResNet-50 and MobileNetV3-Large represented different convolutional design families. The team fine-tuned each model, made a full-precision version and a fake-quantized INT8 counterpart, then generated explanations with Grad-CAM, Grad-CAM++, Integrated Gradients and LIME. Drift was measured using top-k intersection-over-union, Dice overlap and Spearman rank correlation, with explicit random-overlap baselines.
The primary explanation analysis used a stratified subset of 320 validation images and yielded 3,240 paired saliency comparisons per operating point. That is stronger than inspecting a handful of attractive heatmaps, but remains one dataset in one domain. Full-validation summary checks differed from the primary subset by no more than 1.4 percentage points in intersection-over-union. Matching per-image records for those summaries were unavailable, limiting independent verification of which cases moved and why.[1]
The target-layer error is the most transferable finding
Grad-CAM-style methods require a spatial feature layer. A naive rule choosing the final module selected a one-by-one spatial tensor in all three architectures. That tensor cannot express a useful location: after expansion it produces a constant map. Under the faulty setup, apparent attribution collapse occurred in as many as 61.7% of images. Selecting layers with an explicit spatial-dimension criterion removed collapse in all 3,240 primary comparisons.
For engineers, this is more than a coding footnote. An explanation dashboard can produce smooth output even when the chosen hook makes localisation mathematically uninformative. Validation should test the explanation implementation before comparing models. Teams should record the layer, preprocessing, interpolation and quantization path; include random and constant-map controls; and verify that explanations respond when relevant image evidence changes. A heatmap that renders successfully is not yet an explanation that works.[1]
Calibration changed predictions as well as explanations
Quantization needs calibration data to map floating-point activations into a narrower integer range. On the full validation split, MinMax calibration reduced full-precision-to-INT8 prediction agreement to 0.644 for EfficientNetV2-S and 0.641 for MobileNetV3-Large. Percentile calibration raised those figures to 0.927 and 0.915. A model could therefore retain a respectable aggregate score yet disagree with its original version on many images because a deployment setting was poorly chosen.
Preservation varied by architecture and method. Integrated Gradients was least preserved overall, while MobileNetV3-Large was most affected among the architectures. Explanation-aware quantization training improved MobileNetV3-Large top-k overlap by 16.3%, but worsened EfficientNetV2-S by 8.5% to 26.0% relative to post-training quantization. Against a standard quantization-aware-training control, the consistency term improved overlap in all nine tested architecture-method cells, although only six remained significant after Holm correction. There is no universal switch that makes compressed explanations safe.[1]
What this means for people using edge AI
A plant-disease tool might show a grower which part of a leaf influenced a classification. If compression moves that highlight while leaving the label unchanged, the interface can offer a different rationale from the laboratory model. That could direct attention to a harmless edge, background object or imaging artefact. The study does not test farmers, treatment choices, crop loss or field conditions, so it cannot show that explanation drift changes real decisions. It identifies a technical risk that teams can measure before release.
The same principle could apply to industrial inspection or medical imaging, but these numbers do not automatically transfer. Explanation similarity is also not explanation quality. Two models can agree on the same irrelevant area, while a changed heatmap could move closer to the true object. Auxiliary chest-radiograph localisation summaries in the paper found low overlap with expert masks and had incompletely verified execution provenance, reinforcing the need to separate consistency, faithfulness and meaningful localisation.[1]
Limits and the evidence that would change the assessment
The work was conducted by one author at Daffodil International University, reports no external funding and declares no competing interests. Its evidence comes from one rice-leaf benchmark, three convolutional architectures, four post-hoc methods and simulated INT8 arithmetic. It does not cover vision transformers, lower precision, hardware-specific kernels, real-device latency, field image shift or whether people interpret the maps correctly.
Confidence would rise with independent replications on physical edge hardware, multiple domains and prespecified per-image records. Studies should compare explanations against causal interventions and relevant ground truth rather than only map-to-map similarity. User research should test whether changed explanations alter appropriate decisions. For now, compression validation should include prediction agreement, calibration sensitivity and explanation-specific tests, while teams treat a stable heatmap as one piece of evidence rather than proof of a trustworthy rationale.[1]