QV May Be Enough: Toward the Essence of Attention in LLMs

Computer Science > Artificial Intelligence [Submitted on 11 Mar 2026] QV May Be Enough: Toward the Essence of Attention in LLMs Zhang Edward Starting from first principles and a linguistic perspective centered on part-of-speech (POS) and syntactic analysis, this paper explores and derives the underlying essence of the Query-Key-Value (QKV) mechanism within the Transformer architecture. Based on this theoretical foundation, we provide a unified explanatory framework for the efficacy of contemporary architectures, including MQA, GQA, and MLA, while identifying their inherent trade-offs and potential optimization trajectories. We introduce the QV paradigm and provide empirical evidence for its validity. Building upon this, we propose the QV-Ka optimization scheme, which is further substantiated through experimental validation. The interpretable theoretical analysis of the QKV mechanism presented in this work establishes a robust foundation for the future evolution of large language model architectures. Subjects: Artificial Intelligence (cs.AI) Cite as: arXiv:2603.15665 [cs.AI]   (or arXiv:2603.15665v1 [cs.AI] for this version)   https://doi.org/10.48550/arXiv.2603.15665 Focus to learn more Submission history From: Edward Zhang [view email] [v1] Wed, 11 Mar 2026 14:08:53 UTC (4,275 KB) Access Paper: HTML (experimental) view license Current browse context: cs.AI < prev   |   next > new | recent | 2026-03 Change to browse by: cs References & Citations NASA ADS Google Scholar Semantic Scholar Export BibTeX Citation Bookmark Bibliographic Tools Bibliographic and Citation Tools Bibliographic Explorer Toggle Bibliographic Explorer (What is the Explorer?) Connected Papers Toggle Connected Papers (What is Connected Papers?) Litmaps Toggle Litmaps (What is Litmaps?) scite.ai Toggle scite Smart Citations (What are Smart Citations?) Code, Data, Media Demos Related Papers About arXivLabs Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?)