Attention Heads Exhibit Cognitive Functional Specialization
Researchers publish on December 3, 2025 a new interpretability framework and CogQA dataset that decomposes complex questions into chain-of-thought subquestions labeled by cognitive function such as retrieval or logical reasoning. Using multi-class probing across multiple LLM families, they identify specialized 'cognitive heads' that are sparse, distributed, and hierarchically interactive. Ablations and augmentations show these heads materially affect reasoning performance.
Key Points
- 1Identify attention heads as 'cognitive heads' specialized for functions like retrieval and logical reasoning
- 2Show cognitive heads are sparse, variably distributed, and form interactive hierarchical structures
- 3Demonstrate removing them degrades reasoning while augmenting them improves model reasoning accuracy
Scoring Rationale
Strong cross-model interpretability and actionable ablation findings drive high impact, limited by single preprint source and pending peer review.
Sources
Public references used for this report.
Practice with real Ad Tech data
90 SQL & Python problems · 15 industry datasets
250 free problems · No credit card
See all Ad Tech problems