Learning Lifted Action Models from Unsupervised Visual Traces

Computer Science > Artificial Intelligence [Submitted on 21 Apr 2026] Learning Lifted Action Models from Unsupervised Visual Traces Kai Xi, Stephen Gould, Sylvie Thiébaux Efficient construction of models capturing the preconditions and effects of actions is essential for applying AI planning in real-world domains. Extensive prior work has explored learning such models from high-level descriptions of state and/or action sequences. In this paper, we tackle a more challenging setting: learning lifted action models from sequences of state images, without action observation. We propose a deep learning framework that jointly learns state prediction, action prediction, and a lifted action model. We also introduce a mixed-integer linear program (MILP) to prevent prediction collapse and self-reinforcing errors among predictions. The MILP takes the predicted states, actions, and action model over a subset of traces and solves for logically consistent states, actions, and action model that are as close as possible to the original predictions. Pseudo-labels extracted from the MILP solution are then used to guide further training. Experiments across multiple domains show that integrating MILP-based correction helps the model escape local optima and converge toward globally consistent solutions. Comments: Accepted to the 36th International Conference on Automated Planning and Scheduling (ICAPS-26) Subjects: Artificial Intelligence (cs.AI) Cite as: arXiv:2604.19043 [cs.AI]   (or arXiv:2604.19043v1 [cs.AI] for this version)   https://doi.org/10.48550/arXiv.2604.19043 Focus to learn more Submission history From: Kai Xi [view email] [v1] Tue, 21 Apr 2026 03:49:04 UTC (731 KB) Access Paper: HTML (experimental) view license Current browse context: cs.AI < prev   |   next > new | recent | 2026-04 Change to browse by: cs References & Citations NASA ADS Google Scholar Semantic Scholar Export BibTeX Citation Bookmark Bibliographic Tools Bibliographic and Citation Tools Bibliographic Explorer Toggle Bibliographic Explorer (What is the Explorer?) Connected Papers Toggle Connected Papers (What is Connected Papers?) Litmaps Toggle Litmaps (What is Litmaps?) scite.ai Toggle scite Smart Citations (What are Smart Citations?) Code, Data, Media Demos Related Papers About arXivLabs Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?)