R2D-RL: A RoboCup 2D Soccer Environment for Multi-Agent Reinforcement Learning

Computer Science > Artificial Intelligence [Submitted on 17 Jun 2026] R2D-RL: A RoboCup 2D Soccer Environment for Multi-Agent Reinforcement Learning Haobin Qin, Baofeng Zhang, Hidehisa Akiyama, Keisuke Fujii Robot soccer is a challenging testbed for multi-agent reinforcement learning because it combines partial observability, cooperative and adversarial interaction, sparse rewards, and long-horizon tactical behavior. RoboCup 2D Soccer Simulation (RCSS2D) provides a mature robot-soccer platform, but its competition-oriented server-client architecture is difficult to use directly with modern Python-based MARL workflows. We introduce R2D-RL, a reinforcement learning environment that connects RCSS2D and HELIOS-based player clients to a Python MARL interface through shared-memory communication and cycle-level synchronization. R2D-RL supports full-field and scenario-based training with configurable opponents, Base discrete and Hybrid parameterized action spaces, action masks, expected possession value (EPV)-based reward shaping, and parallel execution. We provide front-goal scenarios and an 11-vs-11 full-field benchmark, together with baseline results. Comments: Code is available at: this https URL Subjects: Artificial Intelligence (cs.AI) Cite as: arXiv:2606.18786 [cs.AI]   (or arXiv:2606.18786v1 [cs.AI] for this version)   https://doi.org/10.48550/arXiv.2606.18786 Focus to learn more Submission history From: Haobin Qin [view email] [v1] Wed, 17 Jun 2026 07:57:06 UTC (6,181 KB) Access Paper: HTML (experimental) view license Current browse context: cs.AI < prev   |   next > new | recent | 2026-06 Change to browse by: cs References & Citations NASA ADS Google Scholar Semantic Scholar Export BibTeX Citation Bookmark Bibliographic Tools Bibliographic and Citation Tools Bibliographic Explorer Toggle Bibliographic Explorer (What is the Explorer?) Connected Papers Toggle Connected Papers (What is Connected Papers?) Litmaps Toggle Litmaps (What is Litmaps?) scite.ai Toggle scite Smart Citations (What are Smart Citations?) Code, Data, Media Demos Related Papers About arXivLabs Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?)