Understanding LLM Performance Degradation in Multi-Instance Processing: The Roles of Instance Count and Context Length

Computer Science > Artificial Intelligence [Submitted on 23 Mar 2026] Understanding LLM Performance Degradation in Multi-Instance Processing: The Roles of Instance Count and Context Length Jingxuan Chen, Mohammad Taher Pilehvar, Jose Camacho-Collados Users often rely on Large Language Models (LLMs) for processing multiple documents or performing analysis over a number of instances. For example, analysing the overall sentiment of a number of movie reviews requires an LLM to process the sentiment of each review individually in order to provide a final aggregated answer. While LLM performance on such individual tasks is generally high, there has been little research on how LLMs perform when dealing with multi-instance inputs. In this paper, we perform a comprehensive evaluation of the multi-instance processing (MIP) ability of LLMs for tasks in which they excel individually. The results show that all LLMs follow a pattern of slight performance degradation for small numbers of instances (approximately 20-100), followed by a performance collapse on larger instance counts. Crucially, our analysis shows that while context length is associated with this degradation, the number of instances has a stronger effect on the final results. This finding suggests that when optimising LLM performance for MIP, attention should be paid to both context length and, in particular, instance count. Subjects: Artificial Intelligence (cs.AI); Computation and Language (cs.CL) Cite as: arXiv:2603.22608 [cs.AI]   (or arXiv:2603.22608v1 [cs.AI] for this version)   https://doi.org/10.48550/arXiv.2603.22608 Focus to learn more Submission history From: Jingxuan Chen [view email] [v1] Mon, 23 Mar 2026 22:13:15 UTC (638 KB) Access Paper: view license Current browse context: cs.AI < prev   |   next > new | recent | 2026-03 Change to browse by: cs cs.CL References & Citations NASA ADS Google Scholar Semantic Scholar Export BibTeX Citation Bookmark Bibliographic Tools Bibliographic and Citation Tools Bibliographic Explorer Toggle Bibliographic Explorer (What is the Explorer?) Connected Papers Toggle Connected Papers (What is Connected Papers?) Litmaps Toggle Litmaps (What is Litmaps?) scite.ai Toggle scite Smart Citations (What are Smart Citations?) Code, Data, Media Demos Related Papers About arXivLabs Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?)