Learned vs Innate Tolerance for Incorrect Perspective

A factorial framework decomposing perspective tolerance in vision architectures into innate (architectural) and learned (training) components across 288 experimental conditions.

Category: Computer Vision Track: Research 6 Architectures 4 Distortion Types 288 Conditions

0.6432

Highest innate tolerance (ResNet-50) -- retains ~64% of base accuracy without perspective-diverse training

r = -0.9937

Near-perfect negative correlation between spatial invariance score and learned fraction

22.59%

MLP-Mixer-B learned fraction -- highest reliance on training data diversity among all architectures

288

Total experimental conditions: 6 architectures x 2 regimes x 4 distortions x 6 severity levels

Problem Statement

Modern vision architectures vary dramatically in their ability to recognize objects under perspective distortions, yet the source of this tolerance remains poorly understood.

Two broad sources of perspective tolerance exist: innate tolerance arising from architectural design choices (e.g., convolutional weight sharing, pooling hierarchies) and learned tolerance acquired through training on perspective-diverse data. Prior studies typically conflate these two sources, making it difficult to attribute observed robustness. This work disentangles these contributions through a controlled factorial experimental design.

Methodology

Factorial design with two training regimes (diverse vs. restricted) across calibrated perspective distortions.

τ_total = τ_innate + τ_learned

Tilt

Rotation around horizontal axis up to 60 degrees, simulating looking up/down at objects.

Pan

Rotation around vertical axis up to 60 degrees, simulating lateral viewpoint change.

Off-axis

Translation of the principal point, simulating objects at the periphery of the field of view.

Combined

Composition of all three primitives, representing worst-case compound distortion.

Architectures Evaluated

Architecture	Family	Spatial Inv. Score (σ)	Depth	Params (M)
ResNet-50	Convolutional	0.72	50	25.6
ConvNeXt-T	Convolutional	0.68	28	28.6
ViT-B/16	Attention	0.45	12	86.6
DeiT-S	Attention	0.48	12	22.1
Swin-T	Attention	0.61	24	28.3
MLP-Mixer-B	MLP	0.35	12	59.9

Tolerance Decomposition

Stacked innate and learned components per architecture, averaged across all severity levels and distortion types.

Innate vs Learned Tolerance

Learned Fraction by Architecture

Spatial Invariance vs Learned Fraction

A near-perfect negative correlation (r = -0.9937) reveals a fundamental capacity-data tradeoff.

Correlation: σ vs φ (r = -0.9937)

Family Comparison: Mean Tolerance

Analysis by Distortion Type

Off-axis distortions are best tolerated innately. Combined distortions yield the highest learned fraction.

Innate vs Learned by Distortion Type

Severity-Dependent Gap (Combined Distortion)

Data Tables

Complete numerical results from all experimental conditions.

Tolerance Decomposition by Architecture

Architecture	Family	σ	τ_innate	τ_learned	φ (Learned Frac.)
ResNet-50	Conv	0.72	0.6432 ± 0.1307	0.1197 ± 0.0357	16.46%
ConvNeXt-T	Conv	0.68	0.6322 ± 0.1300	0.1208 ± 0.0313	16.80%
ViT-B/16	Attention	0.45	0.5658 ± 0.1431	0.1377 ± 0.0432	20.78%
DeiT-S	Attention	0.48	0.5759 ± 0.1394	0.1335 ± 0.0337	19.90%
Swin-T	Attention	0.61	0.6171 ± 0.1422	0.1310 ± 0.0421	18.50%
MLP-Mixer-B	MLP	0.35	0.5405 ± 0.1383	0.1475 ± 0.0417	22.59%

Tolerance Decomposition by Distortion Type

Distortion	τ_innate	τ_learned	φ (Learned Frac.)
Tilt	0.6148 ± 0.1344	0.1329 ± 0.0366	18.67%
Pan	0.5823 ± 0.1367	0.1324 ± 0.0389	19.57%
Off-axis	0.6509 ± 0.1299	0.1299 ± 0.0428	17.40%
Combined	0.5352 ± 0.1422	0.1317 ± 0.0390	21.04%

Base Accuracy (Severity = 0, Diverse Training)

Architecture	Base Accuracy (a₀)
ResNet-50	0.7640
ConvNeXt-T	0.8173
ViT-B/16	0.8101
DeiT-S	0.7980
Swin-T	0.8211
MLP-Mixer-B	0.7480

Detailed Results: Combined Distortion

Architecture	Severity	Acc. (Diverse)	Acc. (Restricted)	τ_innate	τ_learned	φ