Learning What Matters Now: Dynamic Preference Inference under Contextual Shifts

Computer Science > Artificial Intelligence [Submitted on 24 Mar 2026] Learning What Matters Now: Dynamic Preference Inference under Contextual Shifts Xianwei Cao, Dou Quan, Zhenliang Zhang, Shuang Wang Humans often juggle multiple, sometimes conflicting objectives and shift their priorities as circumstances change, rather than following a fixed objective function. In contrast, most computational decision-making and multi-objective RL methods assume static preference weights or a known scalar reward. In this work, we study sequential decision-making problem when these preference weights are unobserved latent variables that drift with context. Specifically, we propose Dynamic Preference Inference (DPI), a cognitively inspired framework in which an agent maintains a probabilistic belief over preference weights, updates this belief from recent interaction, and conditions its policy on inferred preferences. We instantiate DPI as a variational preference inference module trained jointly with a preference-conditioned actor-critic, using vector-valued returns as evidence about latent trade-offs. In queueing, maze, and multi-objective continuous-control environments with event-driven changes in objectives, DPI adapts its inferred preferences to new regimes and achieves higher post-shift performance than fixed-weight and heuristic envelope baselines. Comments: 10 pages, ICLR 2026 poster paper Subjects: Artificial Intelligence (cs.AI) Cite as: arXiv:2603.22813 [cs.AI]   (or arXiv:2603.22813v1 [cs.AI] for this version)   https://doi.org/10.48550/arXiv.2603.22813 Focus to learn more Submission history From: Xianwei Cao [view email] [v1] Tue, 24 Mar 2026 05:22:04 UTC (3,108 KB) Access Paper: HTML (experimental) view license Current browse context: cs.AI < prev   |   next > new | recent | 2026-03 Change to browse by: cs References & Citations NASA ADS Google Scholar Semantic Scholar Export BibTeX Citation Bookmark Bibliographic Tools Bibliographic and Citation Tools Bibliographic Explorer Toggle Bibliographic Explorer (What is the Explorer?) Connected Papers Toggle Connected Papers (What is Connected Papers?) Litmaps Toggle Litmaps (What is Litmaps?) scite.ai Toggle scite Smart Citations (What are Smart Citations?) Code, Data, Media Demos Related Papers About arXivLabs Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?)