GraphARC: A Comprehensive Benchmark for Graph-Based Abstract Reasoning

Computer Science > Artificial Intelligence [Submitted on 29 May 2026] GraphARC: A Comprehensive Benchmark for Graph-Based Abstract Reasoning Saku Peltonen, August Bøgh Rønberg, Andreas Plesner, Roger Wattenhofer Relational reasoning lies at the heart of intelligence, but existing benchmarks are typically confined to formats such as grids or text. We introduce GraphARC, a benchmark for abstract reasoning on graph-structured data. GraphARC generalizes the few-shot transformation learning paradigm of the Abstraction and Reasoning Corpus (ARC). Each task requires inferring a transformation rule from a few input-output pairs and applying it to a new test graph, covering local, global, and hierarchical graph transformations. Unlike grid-based ARC, GraphARC instances can be generated at scale across diverse graph families and sizes, enabling systematic evaluation of generalization abilities. We evaluate state-of-the-art language models on GraphARC and observe clear limitations. Models can answer questions about graph properties but often fail to solve the full graph transformation task, revealing a comprehension-execution gap. Performance further degrades on larger instances, exposing scaling barriers. More broadly, by combining aspects of node classification, link prediction, and graph generation within a single framework, GraphARC provides a promising testbed for future graph foundation models. Comments: Accepted at KDD 2026 Datasets and Benchmarks Track Subjects: Artificial Intelligence (cs.AI) Cite as: arXiv:2605.31031 [cs.AI]   (or arXiv:2605.31031v1 [cs.AI] for this version)   https://doi.org/10.48550/arXiv.2605.31031 Focus to learn more Related DOI: https://doi.org/10.1145/3770855.3817591 Focus to learn more Submission history From: Saku Peltonen [view email] [v1] Fri, 29 May 2026 09:03:30 UTC (500 KB) Access Paper: HTML (experimental) view license Current browse context: cs.AI < prev   |   next > new | recent | 2026-05 Change to browse by: cs References & Citations NASA ADS Google Scholar Semantic Scholar Export BibTeX Citation Bookmark Bibliographic Tools Bibliographic and Citation Tools Bibliographic Explorer Toggle Bibliographic Explorer (What is the Explorer?) Connected Papers Toggle Connected Papers (What is Connected Papers?) Litmaps Toggle Litmaps (What is Litmaps?) scite.ai Toggle scite Smart Citations (What are Smart Citations?) Code, Data, Media Demos Related Papers About arXivLabs Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?)