Human-Inspired Context-Selective Multimodal Memory for Social Robots

Computer Science > Artificial Intelligence [Submitted on 13 Apr 2026] Human-Inspired Context-Selective Multimodal Memory for Social Robots Hangyeol Kang, Slava Voloshynovskiy, Nadia Magnenat Thalmann Memory is fundamental to social interaction, enabling humans to recall meaningful past experiences and adapt their behavior accordingly based on the context. However, most current social robots and embodied agents rely on non-selective, text-based memory, limiting their ability to support personalized, context-aware interactions. Drawing inspiration from cognitive neuroscience, we propose a context-selective, multimodal memory architecture for social robots that captures and retrieves both textual and visual episodic traces, prioritizing moments characterized by high emotional salience or scene novelty. By associating these memories with individual users, our system enables socially personalized recall and more natural, grounded dialogue. We evaluate the selective storage mechanism using a curated dataset of social scenarios, achieving a Spearman correlation of 0.506, surpassing human consistency (\rho=0.415) and outperforming existing image memorability models. In multimodal retrieval experiments, our fusion approach improves Recall@1 by up to 13\% over unimodal text or image retrieval. Runtime evaluations confirm that the system maintains real-time performance. Qualitative analyses further demonstrate that the proposed framework produces richer and more socially relevant responses than baseline models. This work advances memory design for social robots by bridging human-inspired selectivity and multimodal retrieval to enhance long-term, personalized human-robot interaction. Comments: Proc. of the 25th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2026) Subjects: Artificial Intelligence (cs.AI) Cite as: arXiv:2604.12081 [cs.AI]   (or arXiv:2604.12081v1 [cs.AI] for this version)   https://doi.org/10.48550/arXiv.2604.12081 Focus to learn more Submission history From: Hangyeol Kang [view email] [v1] Mon, 13 Apr 2026 21:42:40 UTC (1,132 KB) Access Paper: HTML (experimental) view license Current browse context: cs.AI < prev   |   next > new | recent | 2026-04 Change to browse by: cs References & Citations NASA ADS Google Scholar Semantic Scholar Export BibTeX Citation Bookmark Bibliographic Tools Bibliographic and Citation Tools Bibliographic Explorer Toggle Bibliographic Explorer (What is the Explorer?) Connected Papers Toggle Connected Papers (What is Connected Papers?) Litmaps Toggle Litmaps (What is Litmaps?) scite.ai Toggle scite Smart Citations (What are Smart Citations?) Code, Data, Media Demos Related Papers About arXivLabs Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?)