RAMP: Hybrid DRL for Online Learning of Numeric Action Models

Computer Science > Artificial Intelligence [Submitted on 9 Apr 2026] RAMP: Hybrid DRL for Online Learning of Numeric Action Models Yarin Benyamin, Argaman Mordoch, Shahaf S. Shperberg, Roni Stern Automated planning algorithms require an action model specifying the preconditions and effects of each action, but obtaining such a model is often hard. Learning action models from observations is feasible, but existing algorithms for numeric domains are offline, requiring expert traces as input. We propose the Reinforcement learning, Action Model learning, and Planning (RAMP) strategy for learning numeric planning action models online via interactions with the environment. RAMP simultaneously trains a Deep Reinforcement Learning (DRL) policy, learns a numeric action model from past interactions, and uses that model to plan future actions when possible. These components form a positive feedback loop: the RL policy gathers data to refine the action model, while the planner generates plans to continue training the RL policy. To facilitate this integration of RL and numeric planning, we developed Numeric PDDLGym, an automated framework for converting numeric planning problems to Gym environments. Experimental results on standard IPC numeric domains show that RAMP significantly outperforms PPO, a well-known DRL algorithm, in terms of solvability and plan quality. Comments: Accepted as a workshop paper at the Adaptive and Learning Agents (ALA) Workshop at AAMAS 2026 Subjects: Artificial Intelligence (cs.AI) Cite as: arXiv:2604.08685 [cs.AI]   (or arXiv:2604.08685v1 [cs.AI] for this version)   https://doi.org/10.48550/arXiv.2604.08685 Focus to learn more Submission history From: Yarin Benyamin [view email] [v1] Thu, 9 Apr 2026 18:16:19 UTC (1,052 KB) Access Paper: HTML (experimental) view license Current browse context: cs.AI < prev   |   next > new | recent | 2026-04 Change to browse by: cs References & Citations NASA ADS Google Scholar Semantic Scholar Export BibTeX Citation Bookmark Bibliographic Tools Bibliographic and Citation Tools Bibliographic Explorer Toggle Bibliographic Explorer (What is the Explorer?) Connected Papers Toggle Connected Papers (What is Connected Papers?) Litmaps Toggle Litmaps (What is Litmaps?) scite.ai Toggle scite Smart Citations (What are Smart Citations?) Code, Data, Media Demos Related Papers About arXivLabs Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?)