CircuitProbe: Predicting Reasoning Circuits in Transformers via Stability Zone Detection

Computer Science > Artificial Intelligence [Submitted on 1 Apr 2026] CircuitProbe: Predicting Reasoning Circuits in Transformers via Stability Zone Detection Rajkiran Panuganti Transformer language models contain localized reasoning circuits, contiguous layer blocks that improve reasoning when duplicated at inference time. Finding these circuits currently requires brute-force sweeps costing 25 GPU hours per model. We propose CircuitProbe, which predicts circuit locations from activation statistics in under 5 minutes on CPU, providing a speedup of three to four orders of magnitude. We find that reasoning circuits come in two types: stability circuits in early layers, detected through the derivative of representation change, and magnitude circuits in late layers, detected through anomaly scoring. We validate across 9 models spanning 6 architectures, including 2025 models, confirming that CircuitProbe top predictions match or are within 2 layers of the optimal circuit in all validated cases. A scaling experiment across the Qwen 2.5 family reveals that layer duplication consistently benefits models under 3B parameters but degrades performance in 7B+ models, making this a practical scaling technique for small language models. CircuitProbe requires as few as 10 calibration examples and its predictions are stable across English, Hindi, Chinese, and French. Comments: 11 pages, 1 figure, 3 tables. Code available at this https URL Subjects: Artificial Intelligence (cs.AI); Machine Learning (cs.LG) ACM classes: I.2.6; I.2.7 Cite as: arXiv:2604.00716 [cs.AI]   (or arXiv:2604.00716v1 [cs.AI] for this version)   https://doi.org/10.48550/arXiv.2604.00716 Focus to learn more Submission history From: Rajkiran Panuganti [view email] [v1] Wed, 1 Apr 2026 10:26:12 UTC (104 KB) Access Paper: HTML (experimental) view license Current browse context: cs.AI < prev   |   next > new | recent | 2026-04 Change to browse by: cs cs.LG References & Citations NASA ADS Google Scholar Semantic Scholar Export BibTeX Citation Bookmark Bibliographic Tools Bibliographic and Citation Tools Bibliographic Explorer Toggle Bibliographic Explorer (What is the Explorer?) Connected Papers Toggle Connected Papers (What is Connected Papers?) Litmaps Toggle Litmaps (What is Litmaps?) scite.ai Toggle scite Smart Citations (What are Smart Citations?) Code, Data, Media Demos Related Papers About arXivLabs Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?)