SpeechDx: A Multi-Task Benchmark for Clinical Speech AI

Computer Science > Artificial Intelligence [Submitted on 15 Jun 2026] SpeechDx: A Multi-Task Benchmark for Clinical Speech AI Sejal Bhalla, Larry Kieu, Aina Merchant, Eyal de Lara, Alex Mariakakis Speech offers a uniquely informative window into health by simultaneously engaging neurological, motor, respiratory, and vocal systems. Current clinical speech AI methods have largely progressed through isolated condition-specific studies, making results difficult to compare and generalization difficult to assess. We introduce SpeechDx, a large-scale benchmark for clinical speech AI spanning 12 datasets and 27 tasks across diverse health conditions. To enable evaluation across shared clinical mechanisms, SpeechDx structures tasks by the stage of speech production they disrupt: conceptualization, formulation, and articulation. The benchmark tests generalization by including tasks with limited labeled data and evaluating the same health condition across multiple datasets, distinguishing clinically meaningful patterns from dataset artefacts. We systematically evaluate 12 state-of-the-art audio encoders across all tasks and under zero-shot cross-condition transfer. Results show that large-scale speech models represent the strongest overall baselines, domain-specific models improve performance only on closely matched tasks, and no current representation generalizes reliably across the clinical speech landscape. SpeechDx establishes a shared evaluation framework for tracking progress toward general-purpose clinical speech representations Subjects: Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Sound (cs.SD) ACM classes: I.2.6; I.2.1; J.3 Cite as: arXiv:2606.17339 [cs.AI]   (or arXiv:2606.17339v1 [cs.AI] for this version)   https://doi.org/10.48550/arXiv.2606.17339 Focus to learn more Submission history From: Sejal Bhalla [view email] [v1] Mon, 15 Jun 2026 22:38:36 UTC (321 KB) Access Paper: HTML (experimental) view license Current browse context: cs.AI < prev   |   next > new | recent | 2026-06 Change to browse by: cs cs.CL cs.SD References & Citations NASA ADS Google Scholar Semantic Scholar Export BibTeX Citation Bookmark Bibliographic Tools Bibliographic and Citation Tools Bibliographic Explorer Toggle Bibliographic Explorer (What is the Explorer?) Connected Papers Toggle Connected Papers (What is Connected Papers?) Litmaps Toggle Litmaps (What is Litmaps?) scite.ai Toggle scite Smart Citations (What are Smart Citations?) Code, Data, Media Demos Related Papers About arXivLabs Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?)