Introducing Background Temperature to Characterise Hidden Randomness in Large Language Models

Computer Science > Artificial Intelligence [Submitted on 24 Apr 2026] Introducing Background Temperature to Characterise Hidden Randomness in Large Language Models Alberto Messina, Stefano Scotta Even when decoding with temperature T=0, large language models (LLMs) can produce divergent outputs for identical inputs. Recent work by Thinking Machines Lab highlights implementation-level sources of nondeterminism, including batch-size variation, kernel non-invariance, and floating-point non-associativity. In this short note we formalize this behavior by introducing the notion of \emph{background temperature} T_{\mathrm{bg}}, the effective temperature induced by an implementation-dependent perturbation process observed even when nominal T=0. We provide clean definitions, show how T_{\mathrm{bg}} relates to a stochastic perturbation governed by the inference environment I, and propose an empirical protocol to estimate T_{bg} via the equivalent temperature T_n(I) of an ideal reference system. We conclude with a set of pilot experiments run on a representative pool from the major LLM providers that demonstrate the idea and outline implications for reproducibility, evaluation, and deployment. Subjects: Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG) Cite as: arXiv:2604.22411 [cs.AI]   (or arXiv:2604.22411v1 [cs.AI] for this version)   https://doi.org/10.48550/arXiv.2604.22411 Focus to learn more Journal reference: Transactions on Machine Learning Research (TMLR), February 2026, https://openreview.net/pdf?id=bz0he4bARF Submission history From: Alberto Messina Dr. [view email] [v1] Fri, 24 Apr 2026 10:11:06 UTC (484 KB) Access Paper: HTML (experimental) view license Current browse context: cs.AI < prev   |   next > new | recent | 2026-04 Change to browse by: cs cs.CL cs.LG References & Citations NASA ADS Google Scholar Semantic Scholar Export BibTeX Citation Bookmark Bibliographic Tools Bibliographic and Citation Tools Bibliographic Explorer Toggle Bibliographic Explorer (What is the Explorer?) Connected Papers Toggle Connected Papers (What is Connected Papers?) Litmaps Toggle Litmaps (What is Litmaps?) scite.ai Toggle scite Smart Citations (What are Smart Citations?) Code, Data, Media Demos Related Papers About arXivLabs Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?)