Finding FAULTs in Architectural Backdoors: Why Trigger Detection Fails Under Real-World Conditions

Architectural backdoors (ABs) embed trigger-detection logic directly in a deep learning model's inference code instead of its weights, letting them survive retraining and fine-tuning and evade every weight-vetting defense. They've been positioned as a serious emerging supply-chain threat — but their viability under real-world deployment conditions had never been characterized. That's the question this work answers.

One formalism for every AB. Our key insight is that all five existing AB detector families (convolution, pooling, slicing, concatenation, masking) reduce to the same operation: an inner-product similarity between an input region and a fixed reference pattern derived from the trigger, thresholded by a sensitivity parameter. That unification means the entire AB design space is governed by just two distributions — the trigger response and the benign-input response — and lets us formalize three fundamental limitations that no implementation choice can escape:

1. Transformations corrupt triggers. Ordinary preprocessing that real pipelines already apply — contrast adjustment, cropping, noise, blurring, denoising — shifts the trigger response below the detection threshold. On CIFAR-10, a simple contrast adjustment alone drops attack success from 1.0 to 0.01.
2. Sensitivity tuning is self-defeating. The natural attacker response is to relax the detection threshold — but the same relaxation that recovers attack success also admits benign inputs as false positives, producing observable accuracy degradation that destroys the stealth the attack depends on.
3. Complex triggers collide with benign inputs. As trigger patterns grow more intricate, the space of benign inputs that falsely activate the detector expands. On the most complex triggers we tested, benign accuracy collapses from ~90% to as low as 9.9%, with thousands of clean inputs firing the detector.

FAULT (Framework for Assessing Unreliability and Limitations of AB Trigger-detection) operationalizes this analysis: it takes any AB construction and any set of preprocessing transformations, sweeps the sensitivity parameter, and traces the full attack-success / accuracy / false-positive operating curve — exposing exactly the region an optimally-tuned attacker can actually reach.

Findings. Across GTSRB, CIFAR-10, and Mapillary with ResNet18, MobileNetV2, and YoloV11 — 30 configuration–transformation combinations in total — no choice of sensitivity achieves both meaningful attack success and baseline accuracy. Where transformations meaningfully impede the trigger, the attacker's reachable ceiling sits between roughly 10% and 20% attack success, and reaching even that costs measurable benign accuracy (~2.2% on average, up to 9.4%). The practical takeaway for defenders is cheap and concrete: apply one preprocessing transformation from each of three orthogonal families (geometric, photometric, stochastic) before inference, and watch validation accuracy for unexplained single-percent drops. ABs, as currently constructed, cannot be both effective and stealthy in realistic deployments.

Paper currently under review — official publication coming soon.