Context-Length Robustness in Question Answering Models: A Comparative Empirical Study

Computer Science > Artificial Intelligence [Submitted on 16 Mar 2026] Context-Length Robustness in Question Answering Models: A Comparative Empirical Study Trishita Dhara, Siddhesh Sheth Large language models are increasingly deployed in settings where relevant information is embedded within long and noisy contexts. Despite this, robustness to growing context length remains poorly understood across different question answering tasks. In this work, we present a controlled empirical study of context-length robustness in large language models using two widely used benchmarks: SQuAD and HotpotQA. We evaluate model accuracy as a function of total context length by systematically increasing the amount of irrelevant context while preserving the answer-bearing signal. This allows us to isolate the effect of context length from changes in task difficulty. Our results show a consistent degradation in performance as context length increases, with substantially larger drops observed on multi-hop reasoning tasks compared to single-span extraction tasks. In particular, HotpotQA exhibits nearly twice the accuracy degradation of SQuAD under equivalent context expansions. These findings highlight task-dependent differences in robustness and suggest that multi-hop reasoning is especially vulnerable to context dilution. We argue that context-length robustness should be evaluated explicitly when assessing model reliability, especially for applications involving long documents or retrieval-augmented generation. Comments: Accepted for Oral Presentation at Math AI 2026 conference Subjects: Artificial Intelligence (cs.AI) Cite as: arXiv:2603.15723 [cs.AI]   (or arXiv:2603.15723v1 [cs.AI] for this version)   https://doi.org/10.48550/arXiv.2603.15723 Focus to learn more Submission history From: Trishita Dhara [view email] [v1] Mon, 16 Mar 2026 17:14:05 UTC (760 KB) Access Paper: HTML (experimental) view license Current browse context: cs.AI < prev   |   next > new | recent | 2026-03 Change to browse by: cs References & Citations NASA ADS Google Scholar Semantic Scholar Export BibTeX Citation Bookmark Bibliographic Tools Bibliographic and Citation Tools Bibliographic Explorer Toggle Bibliographic Explorer (What is the Explorer?) Connected Papers Toggle Connected Papers (What is Connected Papers?) Litmaps Toggle Litmaps (What is Litmaps?) scite.ai Toggle scite Smart Citations (What are Smart Citations?) Code, Data, Media Demos Related Papers About arXivLabs Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?)