Learning When to Trust in Contextual Bandits

Computer Science > Artificial Intelligence [Submitted on 9 Mar 2026] Learning When to Trust in Contextual Bandits Majid Ghasemi, Mark Crowley Standard approaches to Robust Reinforcement Learning assume that feedback sources are either globally trustworthy or globally adversarial. In this paper, we challenge this assumption and we identify a more subtle failure mode. We term this mode as Contextual Sycophancy, where evaluators are truthful in benign contexts but strategically biased in critical ones. We prove that standard robust methods fail in this setting, suffering from Contextual Objective Decoupling. To address this, we propose CESA-LinUCB, which learns a high-dimensional Trust Boundary for each evaluator. We prove that CESA-LinUCB achieves sublinear regret \tilde{O}(\sqrt{T}) against contextual adversaries, recovering the ground truth even when no evaluator is globally reliable. Subjects: Artificial Intelligence (cs.AI); Machine Learning (cs.LG) Cite as: arXiv:2603.13356 [cs.AI]   (or arXiv:2603.13356v1 [cs.AI] for this version)   https://doi.org/10.48550/arXiv.2603.13356 Focus to learn more Submission history From: Majid Ghasemi [view email] [v1] Mon, 9 Mar 2026 01:35:37 UTC (8,343 KB) Access Paper: HTML (experimental) view license Current browse context: cs.AI < prev   |   next > new | recent | 2026-03 Change to browse by: cs cs.LG References & Citations NASA ADS Google Scholar Semantic Scholar Export BibTeX Citation Bookmark Bibliographic Tools Bibliographic and Citation Tools Bibliographic Explorer Toggle Bibliographic Explorer (What is the Explorer?) Connected Papers Toggle Connected Papers (What is Connected Papers?) Litmaps Toggle Litmaps (What is Litmaps?) scite.ai Toggle scite Smart Citations (What are Smart Citations?) Code, Data, Media Demos Related Papers About arXivLabs Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?)