How Clued up are LLMs? Evaluating Multi-Step Deductive Reasoning in a Text-Based Game Environment

Computer Science > Artificial Intelligence [Submitted on 17 Mar 2026] How Clued up are LLMs? Evaluating Multi-Step Deductive Reasoning in a Text-Based Game Environment Rebecca Ansell, Autumn Toney-Wails Deducing whodunit proves challenging for LLM agents. In this paper, we implement a text-based multi-agent version of the classic board game Clue as a rule-based testbed for evaluating multi-step deductive reasoning, with six agents drawn from GPT-4o-mini and Gemini-2.5-Flash. We further investigate whether fine-tuning on structured logic puzzles transfers to improved in-game reasoning and gameplay. Across 18 simulated games, agents achieve only four correct wins, indicating difficulty in maintaining consistent deductive reasoning over the course of a full game. Additionally, we find that fine-tuning does not reliably improve performance and, in some cases, appears to increase reasoning volume without improving reasoning precision. Subjects: Artificial Intelligence (cs.AI); Computation and Language (cs.CL) ACM classes: I.2.8; I.2.7 Cite as: arXiv:2603.17169 [cs.AI]   (or arXiv:2603.17169v1 [cs.AI] for this version)   https://doi.org/10.48550/arXiv.2603.17169 Focus to learn more Submission history From: Autumn Toney [view email] [v1] Tue, 17 Mar 2026 22:01:11 UTC (538 KB) Access Paper: HTML (experimental) view license Current browse context: cs.AI < prev   |   next > new | recent | 2026-03 Change to browse by: cs cs.CL References & Citations NASA ADS Google Scholar Semantic Scholar Export BibTeX Citation Bookmark Bibliographic Tools Bibliographic and Citation Tools Bibliographic Explorer Toggle Bibliographic Explorer (What is the Explorer?) Connected Papers Toggle Connected Papers (What is Connected Papers?) Litmaps Toggle Litmaps (What is Litmaps?) scite.ai Toggle scite Smart Citations (What are Smart Citations?) Code, Data, Media Demos Related Papers About arXivLabs Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?)