Cross-Domain Demo-to-Code via Neurosymbolic Counterfactual Reasoning

Computer Science > Artificial Intelligence [Submitted on 19 Mar 2026] Cross-Domain Demo-to-Code via Neurosymbolic Counterfactual Reasoning Jooyoung Kim, Wonje Choi, Younguk Song, Honguk Woo Recent advances in Vision-Language Models (VLMs) have enabled video-instructed robotic programming, allowing agents to interpret video demonstrations and generate executable control code. We formulate video-instructed robotic programming as a cross-domain adaptation problem, where perceptual and physical differences between demonstration and deployment induce procedural mismatches. However, current VLMs lack the procedural understanding needed to reformulate causal dependencies and achieve task-compatible behavior under such domain shifts. We introduce NeSyCR, a neurosymbolic counterfactual reasoning framework that enables verifiable adaptation of task procedures, providing a reliable synthesis of code policies. NeSyCR abstracts video demonstrations into symbolic trajectories that capture the underlying task procedure. Given deployment observations, it derives counterfactual states that reveal cross-domain incompatibilities. By exploring the symbolic state space with verifiable checks, NeSyCR proposes procedural revisions that restore compatibility with the demonstrated procedure. NeSyCR achieves a 31.14% improvement in task success over the strongest baseline Statler, showing robust cross-domain adaptation across both simulated and real-world manipulation tasks. Comments: Accepted at CVPR 2026 Subjects: Artificial Intelligence (cs.AI) Cite as: arXiv:2603.18495 [cs.AI]   (or arXiv:2603.18495v1 [cs.AI] for this version)   https://doi.org/10.48550/arXiv.2603.18495 Focus to learn more Submission history From: Jooyoung Kim [view email] [v1] Thu, 19 Mar 2026 05:04:20 UTC (17,248 KB) Access Paper: HTML (experimental) view license Current browse context: cs.AI < prev   |   next > new | recent | 2026-03 Change to browse by: cs References & Citations NASA ADS Google Scholar Semantic Scholar Export BibTeX Citation Bookmark Bibliographic Tools Bibliographic and Citation Tools Bibliographic Explorer Toggle Bibliographic Explorer (What is the Explorer?) Connected Papers Toggle Connected Papers (What is Connected Papers?) Litmaps Toggle Litmaps (What is Litmaps?) scite.ai Toggle scite Smart Citations (What are Smart Citations?) Code, Data, Media Demos Related Papers About arXivLabs Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?)