Strikingness-Aware Evaluation for Temporal Knowledge Graph Reasoning

Computer Science > Artificial Intelligence [Submitted on 13 May 2026] Strikingness-Aware Evaluation for Temporal Knowledge Graph Reasoning Rikui Huang, Shengzhe Zhang, Wei Wei Temporal Knowledge Graph Reasoning (TKGR) aims at inferring missing (especially future) events from historical data. Current evaluation in TKGR uniformly weights all events, ignoring that most are trivial repetitions, which overestimate the true reasoning ability. Therefore, the rare outstanding events, whose prediction demands deeper reasoning, should be distinguished and emphasized. To this end, we propose a strikingness-aware evaluation framework, which introduces a rule-based strikingness measuring framework (RSMF) to quantify event strikingness by comparing its expected occurrence with peer events derived from temporal rules. Strikingness is then integrated as a weighting factor into metrics like weighted MRR and Hits@k. Experiments on four TKG benchmarks reveal: 1) All representative models perform worse as event strikingness increases, 2) Path-based methods excel on low-strikingness events and representation-based ones on high-strikingness events, 3) We design an ensemble method whose gains stem from fitting trivial events rather than reasoning improvement. Our framework provides a more rigorous evaluation, refocusing the field on predicting outstanding events. Comments: Accepted to IJCAI-ECAI 2026 Subjects: Artificial Intelligence (cs.AI) Cite as: arXiv:2605.13153 [cs.AI]   (or arXiv:2605.13153v1 [cs.AI] for this version)   https://doi.org/10.48550/arXiv.2605.13153 Focus to learn more Submission history From: Rikui Huang [view email] [v1] Wed, 13 May 2026 08:17:54 UTC (755 KB) Access Paper: HTML (experimental) view license Current browse context: cs.AI < prev   |   next > new | recent | 2026-05 Change to browse by: cs References & Citations NASA ADS Google Scholar Semantic Scholar Export BibTeX Citation Bookmark Bibliographic Tools Bibliographic and Citation Tools Bibliographic Explorer Toggle Bibliographic Explorer (What is the Explorer?) Connected Papers Toggle Connected Papers (What is Connected Papers?) Litmaps Toggle Litmaps (What is Litmaps?) scite.ai Toggle scite Smart Citations (What are Smart Citations?) Code, Data, Media Demos Related Papers About arXivLabs Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?)