The Sim-to-Real Gap of Foundation Model Agents: A Unified MDP Perspective

Computer Science > Artificial Intelligence [Submitted on 5 Jun 2026] The Sim-to-Real Gap of Foundation Model Agents: A Unified MDP Perspective Xiaoou Liu, Tiejin Chen, Weibo Li, Xiyang Hu, Hua Wei Foundation model agents are increasingly deployed for real-world decision-making, but suffer from the sim-to-real gap. While robotics and classical control have mature frameworks to address this gap, the foundation model community is treating agent robustness as an entirely novel phenomenon. Our paper proposes formalizing the foundation model agent evaluation and training gap as a classical sim-to-real problem structured entirely around the four elements of a Markov Decision Process, including Observation, Action, Transition, and Reward. In this paper, we set a comprehensive research agenda that translates classical discrepancies into the foundation model domain and advocates for adopting established solutions like domain randomization. We provide concrete examples, such as a multilingual tool calling to demonstrate how severe observation space gaps lead to operationally invalid actions despite correct semantic intent. Ultimately, this agenda aims to drive a paradigm shift, yielding a unified vocabulary and standardized stress test benchmarks to foster a new generation of highly trustworthy agents for reliable real-world applications. Comments: 7 pages, 2 figures, 2 tables. Accepted by KDD 2026 Blue Sky Ideas Track Subjects: Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Emerging Technologies (cs.ET) MSC classes: 68T50, 68T37, 68Q32 ACM classes: I.2.7; I.2.6; I.2.4 Cite as: arXiv:2606.07017 [cs.AI]   (or arXiv:2606.07017v1 [cs.AI] for this version)   https://doi.org/10.48550/arXiv.2606.07017 Focus to learn more Related DOI: https://doi.org/10.1145/3770855.3818660 Focus to learn more Submission history From: Hua Wei [view email] [v1] Fri, 5 Jun 2026 08:00:25 UTC (3,109 KB) Access Paper: HTML (experimental) view license Current browse context: cs.AI < prev   |   next > new | recent | 2026-06 Change to browse by: cs cs.CL cs.ET References & Citations NASA ADS Google Scholar Semantic Scholar Export BibTeX Citation Bookmark Bibliographic Tools Bibliographic and Citation Tools Bibliographic Explorer Toggle Bibliographic Explorer (What is the Explorer?) Connected Papers Toggle Connected Papers (What is Connected Papers?) Litmaps Toggle Litmaps (What is Litmaps?) scite.ai Toggle scite Smart Citations (What are Smart Citations?) Code, Data, Media Demos Related Papers About arXivLabs Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?)