Count Me If You Can: Geometric Failure Modes in Language Model Counting

Large language models often count accurately at small magnitudes but fail systematically as the target count grows. We study this failure mode in two 30B-scale models and find that high-count errors are not explained by missing information: count remains linearly recoverable from internal activations even when the model predicts incorrectly.

Instead, counting failure coincides with a geometric transition in the residual stream. Small counts form clean, well-separated trajectories, while large counts become compressed, higher-dimensional, and unstable near readout — we observe adjacent high-count centroids are 3–6× closer than low-count centroids. This exposes a gap between internal recoverability and native decodability.

Causal ablations further implicate late-layer MLP sub-blocks as a source of high-count degradation: suppressing these components improves high-count accuracy by up to 20% while preserving low-count performance.

These results reframe symbolic counting failures as failures of representational geometry and readout compatibility.