Anchored, Not Graded: Vision-Language Models Fail at Slant-from-Texture Perception¶

Conference: ECCV 2026
arXiv: 2606.06714
Code: None
Area: Multimodal VLMs
Keywords: Vision-Language Models, texture perception, slant perception, psychophysics, anchoring effect

TL;DR¶

This work systematically evaluates the performance of various VLMs on texture-driven slant perception tasks. It reveals that VLM zero-shot predictions are heavily anchored to a few discrete values (\(0^\circ\), \(\pm 25^\circ\), \(\pm 45^\circ\)), behaving completely differently from the continuous yet biased perception exhibited by humans and self-supervised CNNs. Furthermore, this anchoring originates from a readout bottleneck at the language output side rather than a lack of geometric information in the vision encoder.

Background & Motivation¶

Background: Human texture-driven slant perception (slant-from-texture) has a well-established foundation in psychophysics. Classic experiments by Todd et al. demonstrate that humans can judge the slant angle of 3D surfaces using texture gradient cues, but these judgments exhibit systematic, biased patterns: convex surfaces appear steeper than concave ones of the same physical angle (B1), larger fields of view (FOV) yield greater perceived slant (B2), and the ability to distinguish curvature signs drops to random levels under small FOVs (B3). These biases are typically quantified with a perceptual gain (perceived slant / physical slant) of approximately 0.56, indicating that humans systematically underestimate surface slants.

Key Challenge: Prior research shows that self-supervised CNNs (autoencoders trained for reconstruction) can replicate human-like texture slant perception biases, whereas supervised CNNs (regressing directly to physical slant angles) achieve unbiased performance. This suggests that human-like biases are not purely architecture-dependent but rather determined by whether the learning objective emphasizes texture statistics instead of explicit geometric labels. However, how these models perform on this classic psychophysics task as they evolve into larger Vision-Language Models (VLMs) — which combine large-scale vision encoders and language models trained via image-text alignment — remains unexplored.

Goal: To evaluate the behavior of various VLMs on standard texture slant perception tasks and answer two questions: (1) At the behavioral level, do VLMs exhibit systematic biases similar to humans and self-supervised CNNs? (2) At the intervention level, can prompt engineering, in-context learning, and supervised fine-tuning induce human-like continuous perceptual behavior?

Key Insight: The authors ingeniously introduce the psychophysical experimental paradigm to VLM evaluation, using synthetic polka-dot texture stimuli identical to those used in human/CNN experiments. These stimuli contain no shading, color, or contour cues, making texture gradients the sole source of 3D orientation information. This highly controlled setup eliminates semantic distractors, directly probing the model's ability to process low-level geometric cues.

Core Idea: Under zero-shot settings, VLMs exhibit a unique anchoring failure mode in texture slant perception — their outputs collapse to a few discrete values rather than continuously varying with stimulus parameters. This failure stems not from deficient visual encoding, but from a readout bottleneck at the language output side.

Method¶

Overall Architecture¶

This work establishes a psychophysics-inspired evaluation paradigm for VLMs. The input consists of synthetic polka-dot textured surface images — two flats with identical randomly distributed dots forming a dihedral angle, conveying 3D orientation information purely via texture gradients. The output consists of the model's estimations of the surface slant angle (\(-90^\circ\) to \(+90^\circ\)) and curvature sign (convex/concave). The entire pipeline is divided into three stages: stimulus generation and different prompt template designs, zero-shot/in-context learning/fine-tuning evaluation of multiple VLM families, and localizing the root cause of failures via linear probes and attention head ablation. Stimulus parameters include optical slant angle (\(25^\circ\) to \(60^\circ\), 10 values), FOV (\(5^\circ\) to \(60^\circ\), 10 values), curvature sign (convex/concave), and texture random seed (12 jitters), forming a complete stimulus set of 200 conditions \(\times\) 12 texture jitters. Each stimulus-prompt pair is queried once with a low temperature (\(T=0.1\)). The main experiment covers 6 open-source VLMs (Gemma3, LLaMA4, LLaVA, Mistral, Moondream, Qwen2.5-VL) as well as validation on 12 leading closed-source models (GPT-5.4, Gemini3.1, etc.).

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Synthetic Polka-Dot Texture Stimuli<br/>No shading/color/contours"] --> B["Optical Slant Angle (25-60°)<br/>FOV (5-60°)<br/>Curvature Sign (convex/concave)"]
    B --> C["VLM Vision Encoder<br/>ViT/SigLIP/VQ-VAE"]
    C --> D["Language Model Readout<br/>(Zero-shot/ICL/SFT)"]
    D --> E["Output Discrete Anchored Values<br/>{0°, ±25°, ±45°}<br/>or Continuous Estimates (Post-SFT)"]
    C --> F["Linear Probe<br/>(Layer-wise R² Analysis)"]
    F --> G["Visual Features Encode Geometric Info<br/>but Language Head Discards It"]

Key Designs¶

1. Controlled Psychophysical Stimulus Paradigm: Isolating Texture Gradient as the Sole Cue

The experiments employ synthetic polka-dot textured surfaces, strictly following the stimulus generation protocols of Todd et al. and Wang et al. Each stimulus is formed by two flat surfaces with identical randomly scattered dots creating a dihedral angle, conveying 3D information purely through texture gradients — no shading, color, or contour cues are present. Stimulus parameters follow a full-factorial design: optical slant angle \(\sigma_{\text{cen}}\) (10 levels, \(25^\circ\) to \(60^\circ\)), field of view \(\text{FOV}\) (10 levels, \(5^\circ\) to \(60^\circ\)), and curvature sign (convex/concave), yielding 200 conditions. The physical slant angle \(\rho\) is determined by both \(\sigma_{\text{cen}}\) and \(\text{FOV}\):

\[\rho = \sigma_{\text{cen}} - s \cdot \frac{\text{FOV}}{4}, \quad s = -1 (\text{concave}), \; s = +1 (\text{convex})\]

The crucial distinction here is that while the optical slant \(\sigma_{\text{cen}}\) is the experimental match parameter corresponding to the direct "readout" of texture gradients, the physical slant \(\rho\) integrates curvature and FOV information, requiring geometric reasoning to understand the 3D structure. Thus, predicting the physical slant \(\rho\) is far more difficult than simply reading local texture features, allowing the authors to distinguish whether a VLM "cannot see" the texture or "cannot speak." Each texture random jitter (12 seeds) independently generates new projections to ensure that geometric reasoning must generalize beyond texture appearance noise. The dataset comprises 2,000 training images and 400 test images.

2. Systematic Prompt Engineering and the Revelation of the Anchoring Phenomenon

The authors designed 7 prompt variants for the slant regression task and 3 for the binary classification (convex/concave) task, systematically varying task type, instruction detail, language style (natural vs. technical), output format (free text vs. JSON), and the presence of labeled examples for in-context learning (ICL). The experiments revealed that regardless of the prompt style, all zero-shot VLM predictions surprisingly anchored to a small set of discrete values — most commonly \(0^\circ\), \(\pm 25^\circ\), and \(\pm 45^\circ\). For many models, over 50% of the predictions collapsed onto a single value, indicating that this is a systematic anchoring effect rather than random noise. Three pieces of key evidence support the anchoring explanation over noise: (1) Error types include instruction-following errors (outputs without numerical values), exemplar anchoring (Moondream replicating values from the prompt examples), and zero-degree anchoring (predicting a flat surface despite reasoning text describing steep slants); (2) Two-way ANOVA showed no significant main effect of prompt style on slant error (\(p=0.157\)), while only model type had a significant effect (\(p=0.000291\)), and prompt \(\times\) model interaction was significant (\(p=0.002\)) but yielded no consistent improvement; (3) Variance across temperature shifts and repeated queries (10 repetitions) was extremely low, and averaging predictions did not mitigate the issue. Providing 4 labeled examples for ICL was similarly ineffective, with median errors remaining above \(40^\circ\) and curvature sign accuracy hovering near chance (50%).

3. Supervised Fine-Tuning (SFT) Partially Resolves but Leaves Residual Anchoring

Supervised fine-tuning (SFT) was performed on Qwen2.5-VL-3B using LoRA (updating only q_proj and v_proj) with masked token loss. SFT dramatically improved slant estimation (MAE dropped from \(45.1^\circ\) to \(15.3^\circ\), STD from \(34.9^\circ\) to \(26.2^\circ\), paired t-test \(p=3.4 \times 10^{-106}\)), and curvature sign identification rose from chance (50%) to 86.10%, numerically approaching human performance (86%) but remaining below self-supervised CNNs (96.4%). However, anchoring was not fully eliminated; instead, it occurred at a finer scale. Post-SFT predictions formed horizontal band-like distributions around the ground-truth values rather than forming a continuous linear mapping. Curvature sign judgments revealed an interesting human-like but more extreme asymmetric error pattern: concave surfaces had almost zero flipping errors (near 0%) at large FOVs, whereas convex surfaces maintained a substantial error rate of 0.3-0.4 across all optical slants. This aligns with the direction of the human B1 bias but is much more pronounced. SFT introduced a monotonic relationship between predictions and ground truths but failed to root out the language model's discretization tendency.

4. Visual Module Probing Pinpoints the Readout Bottleneck

To locate whether the zero-shot failure stems from inadequate visual encoding or language readout failure, the authors applied layer-wise linear probing (ridge regression, \(\alpha=1.0\)) on Qwen2.5-VL-3B's 32-layer vision encoder. Probing localized features (post-projector layer) yielded \(R^2 = 0.826\) for physical slant, \(R^2 = 0.988\) for optical slant, \(R^2 = 0.884\) for FOV, and \(98.3\%\) accuracy for curvature sign. This proves that visual features contain rich geometric details, matching the performance of a standalone ViT-MAE-86M. Layer-wise analysis revealed that optical slant decoding saturated almost immediately from the first layer (\(R^2 = 0.992\)), while FOV, physical slant, and curvature reached peak performance at intermediate layers (Layers 17–19, approximately 56%–59% deep) before declining in the back-end layers — FOV fell from 0.920 to 0.844 and physical slant from 0.879 to 0.769. This back-end decline indicates that the final transformer layers reorganize features to optimize downstream language modeling objectives, sacrificing the accessibility of some geometric information. The patch merger layer partially recovered some information (\(FOV = 0.884, PS = 0.826\)) by re-introducing local structure through \(2 \times 2\) spatial merging. This pattern was consistently validated across four different VLM architectures (LLaVA-1.5-7B/CLIP-ViT-L/14, PaliGemma-3B/SigLIP-So400M, and Chameleon-7B/VQ-VAE). Chameleon was particularly striking: as the only model where vision and text tokens pass through the same shared transformer, its last layer (layer 31) had almost zero geometric correlation (\(OS \; R^2 = 0.033, FOV \; R^2 = 0.005, PS \; R^2 = 0.065\), curvature accuracy = 55.2% [chance level]), meaning the geometric information was completely transformed away in the final layer.

Furthermore, readout tests provided direct evidence of a readout bottleneck: average pooling the frozen post-projector visual tokens (projected into the LM embedding space) and feeding them to a simple linear regression head (fitted only on the 2,000 training set) yielded continuous predictions on the 400 test set, resulting in \(R^2 = 0.696\) (MAE = \(21.4^\circ\)) for Qwen2.5-VL-3B, \(R^2 = 0.857\) for LLaVA, and \(R^2 = 0.880\) for PaliGemma, with \(342+/400\) unique predicted values — demonstrating that the underlying predictions are inherently continuous. This proves that geometric information is decodable at the linear readout level, but the VLM's language head discards it during token-by-token generation.

A Complete Example: Stimulus with Optical Slant = \(40^\circ\), FOV = \(35^\circ\), Convex Surface¶

Generating a synthetic image with an optical slant of \(40^\circ\), FOV = \(35^\circ\), a convex surface, and a specific texture seed. Feeding this image to Qwen2.5-VL-3B (zero-shot) with the natural language prompt "Analyze polka-dot deformations to estimate the surface slant angle" yields the output "(25, 0.7); dots are stretched near the fold center", giving a predicted value of \(25^\circ\) (ground truth: \(\rho = 40 - (+1) \times 35/4 = 31.25^\circ\)). Although the predicted direction (convex) is correct, the model's numerical output lands squarely on the \(\pm 25^\circ\) anchor point, failing to provide a continuous estimate for \(31.25^\circ\). If evaluated with a different texture jitter (same geometry, different random seed), the model consistently outputs \(25^\circ\), proving the prediction is driven by anchoring rather than variations in texture appearance. After SFT, the same model's prediction for a similar stimulus shifts from \(25^\circ\) to \(32^\circ\), closer to the ground truth of \(31.25^\circ\), though predictions still form a band-like distribution around ground truths rather than varying independently and continuously across individual stimuli.

Key Experimental Results¶

Main Results¶

Model	Slant MAE (deg)	Curvature Accuracy	Dominant Anchored Values	Mode Proportion
Qwen2.5-VL-3B	~45	~50% (chance)	0°, ±25°, ±45°	>50%
LLaVA-1.5-7B	~45	~50%	0°, ±25°, ±45°	>50%
PaliGemma-3B	~45	~50%	0°, ±25°, ±45°	>50%
Gemma3 (various sizes)	~45	~50%	0°, ±25°, ±45°	>50%
Qwen2.5-VL-3B (Post-SFT)	15.3	86.1%	Fine-grained anchoring (banded distribution)	Decreased but still significant
Humans (Todd et al.)	Biased (gain ~0.56)	~86%	Continuous biased	—
Self-supervised CNN (Wang et al.)	Biased (human-like)	96.4%	Continuous biased	—

Note: Differences between zero-shot models were not significant (two-way ANOVA on prompt styles \(p=0.157\)); all VLMs were equally anchored. An additional 12 closed-source models (GPT-5.4, Gemini3.1, Claude Opus, etc.) also displayed the same anchoring patterns, with \(R^2\) correlation to physical slant \(\le 0.16\) (mean 0.04) and curvature accuracy between 0.46 and 0.58 (chance level).

Ablation Study¶

Configuration	Key Metrics (Slant MAE / Curvature Acc)	Description
Qwen2.5-VL-3B Zero-shot	45.1° / 50%	Baseline: Discrete anchoring
+ Natural language prompt variants	~45° / ~50%	No significant improvement across 7 prompt styles
+ Technical language prompts	~45° / ~50%	Technical jargon does not help
+ JSON output format	~45° / ~50%	Schema constraints do not affect anchoring
+ In-context learning (4-shot)	~45° / ~50%	Labeled exemplars do not improve performance
+ SFT 10 epochs (LoRA)	15.3° / 86.1%	MAE drops substantially; curvature accuracy reaches human levels
+ SFT 100 epochs (LoRA)	~15° / ~86%	Limited further improvements; residual anchoring persists
+ Per-attention-head ablation	~45° / ~50%	None of the 504 single-head ablations eliminated anchoring
Vision token linear probe	\(R^2 = 0.696\) (continuous predictions)	Proves geometric information is present up to the language input side

Key Findings¶

Vision encoder is not the bottleneck: Linear probes show that VLM vision encoders represent rich geometric information in their deep-middle layers (Layers 17–19, physical slant \(R^2 \approx 0.88\)), but the final layer reorganizes the feature space for language modeling objectives, losing some geometric readability — providing the neural basis for "knowing but not being able to say."
SFT introduces monotonicity but does not eliminate discretization: SFT establishes a monotonic relationship between predictions and ground truths, but outputs still cluster into discrete horizontal bands instead of changing continuously. This suggests that language models naturally tend to map continuous quantities to a few "strong" numerical tokens — a behavior highly consistent with the "round-number bias" in LLM decoding.
Readout head test provides the strongest evidence: Directly predicting physical slant using a single linear regression layer on the post-projector visual tokens produces continuous outputs (\(R^2 = 0.696\text{-}0.880\), over 342 unique values out of 400 stimuli). This directly proves that the VLM "knows" but "cannot speak" — geometric information is already present in the visual tokens passed to the language model, but the language model fails to decode it into continuous numerical estimates.
Replicability across model architectures: From Qwen's Native ViT to CLIP-ViT-L/14, SigLIP-So400M, and VQ-VAE, the layer-wise probing patterns across four heterogeneous vision encoders are highly consistent — geometric information peaks in middle layers and decreases in back-end layers. Chameleon (where vision and text tokens share a single transformer) is the most extreme example, with the last layer (layer 31) degrading near-perfect geometric encoding (\(R^2 > 0.9\)) to nearly zero.

Highlights & Insights¶

Precise diagnosis of "knowing but not being able to say": The most brilliant insight of this paper is not just finding that VLMs perform poorly, but precisely isolating the failure node through layer-wise probing and readout tests. It reveals that the bottleneck is not the vision encoder's capacity, but rather the language model's "readout" phase discarding continuous geometric information. This encoding-readout dissociation provides a crucial piece of evidence for understanding multimodal information flow in VLMs.
Ingenious adaptation of psychophysical experimental paradigms: By strictly recreating classic human visual experiments (slant-from-texture) for VLM evaluation, the authors exploit highly controlled, semantics-free stimuli to expose low-level visual processing gaps that standard high-level VLM benchmarks (VQA, COCO captioning, etc.) completely fail to capture. This "dimensionality-reduction" evaluation strategy is highly worthy of extension to other perception tasks.
Speculative yet profound analysis of cross-entropy loss bottlenecks: The authors hypothesize that since VLM language models are trained with cross-entropy loss over discrete token sequences, a continuous value (such as \(42.5^\circ\)) must be encoded as a sequence of discrete tokens ("4-2-.-5"). As a result, the model has no intrinsic incentive to preserve covariation between the text output and the continuous geometric variables latent in the visual tokens. This observation offers a concrete case study of the "discrete nature of language models," hinting at the need for new training objectives or architectures to bridge the gap between continuous perception and discrete expression.
Behavioral implications of SFT's partial fix: Post-SFT, the asymmetric error pattern in curvature sign judgments (convex surfaces are more easily misclassified as concave, while concave surfaces stay error-free under large FOVs) aligns with the direction of the human B1 bias but is more extreme. This suggests that the VLM is learning heuristic processing strategies similar to humans, but its limited expressive capacity prevents smooth, continuous output.

Limitations & Future Work¶

Evaluated on polka-dot textures only: The experiments were conducted solely on synthetic polka-dot textures and did not extend to other texture types (regular textures, natural textures). While polka-dot textures are well-grounded in psychophysics, VLMs might perform differently on natural image textures, which are much more "in-distribution" for VLM training data.
SFT limited to a single model: Fine-tuning was only evaluated on Qwen2.5-VL-3B, without validating whether SFT is similarly or more effective on other architectures (e.g., LLaVA, PaliGemma). Exploring SFT behavioral differences across different VLMs could reveal more generalized mechanisms.
Future Directions: (1) Validating the anchoring effect across a broader range of texture types; (2) Adding extra geometric cues (e.g., shading, contours) to test whether VLMs can recover continuous predictions under cue-rich conditions; (3) Exploring alternative architectural designs, such as directly regressing continuous information from visual tokens via a continuous output head (instead of discrete token sequences) or designing auxiliary losses that encourage continuous expression; (4) Tracing the transformation of geometric information within the language model's hidden states more granularly — mapping the complete flow from visual tokens through transformer execution layers to output embeddings; (5) Extending this diagnostic paradigm to other low-level perception tasks, such as perspective, depth, and shape-from-shading.

vs. Wang et al. (2023, slant perception in self-supervised CNNs): Wang et al. found that self-supervised CNNs replicate human-like biased perception, while supervised CNNs are unbiased. This paper extends this inquiry to VLMs and shows a more fundamental failure mode: instead of behaving continuously with a bias, VLMs suffer from anchoring. These three paradigms form an intriguing spectrum: self-supervised CNNs = human-like, biased, but continuous; supervised CNNs = unbiased and continuous; VLMs = anchored, discrete, and decoupled from stimulus parameters. This indicates that the learning objective (reconstruction vs. classification vs. language modeling) dictates the nature of perceptual representation.
vs. Rudman et al. (2026, VLM failure mechanisms): Rudman et al. discovered that specific attention heads lead to prompt copying (replicating values given in the prompt) and demonstrated that ablating these heads can remove the copying. This work tests similar hypotheses at a larger scale (1,288 ablation experiments across two models) but finds that anchoring is not an attention-head-interpretable phenomenon — no single-head ablation could eliminate anchoring, proving that continuous geometric estimation anchoring is a distributed property, fundamentally different from discrete classification tasks.
vs. Tseng et al. / Lou et al. (numerical anchoring in LLMs): Existing literature highlights anchoring effects and round-number biases in LLMs during numerical reasoning tasks. This work extends these findings to VLM perceptual tasks, showcasing a more fundamental class of anchoring — one that does not depend on reasoning context but directly drives perceptual estimates from physical texture stimuli, suggesting this discretization might be an inherent property of language modeling.
vs. Standard VLM Visual Benchmarks: This work stands in stark contrast to the success of VLMs on high-level tasks like VQA, captioning, and scene understanding. It exposes gaps in "low-level perceptual gradients" missing from current benchmarks, suggesting future evaluations should incorporate more psychophysically grounded perceptual tasks.

Rating¶

Novelty: ⭐⭐⭐⭐⭐ [Systematically introduces classic psychophysical paradigms to VLM evaluation; identifies a unique anchoring failure mode and pinpoints the readout bottleneck via probing. The approach is clear and solid, filling a crucial gap in evaluating low-level geometric perception in VLMs.]
Experimental Thoroughness: ⭐⭐⭐⭐⭐ [Covers 6 open-source + 12 closed-source VLMs, multiple prompt variations, SFT interventions, layer-wise linear probing, 1,288 attention head ablations, and generalization across 4 heterogeneous vision encoders. The experimental design is comprehensive and rigorous.]
Writing Quality: ⭐⭐⭐⭐⭐ [The logical flow is highly coherent: from phenomenon discovery (anchoring) \(\rightarrow\) intervention (prompts/ICL/SFT ineffective or partially effective) \(\rightarrow\) finding the root cause (probing demonstrates healthy visual encoding) \(\rightarrow\) mechanism (readout bottleneck + negative results from head ablation). Diagrams and tables are well-crafted, and ANOVA and probing results are clearly presented.]
Value: ⭐⭐⭐⭐⭐ [Provides a crucial diagnostic tool for safe, reliable deployment of VLMs (meeting human-AI alignment requirements for perceptual consistency) and exposes fundamental limitations of the current language modeling paradigm in representing continuous perception, offering key guiding insights for future VLM architectures and evaluation methodologies.]