Understanding the Theoretical Foundations of Deep Neural Networks through Differential Equations

Computer Science > Artificial Intelligence [Submitted on 18 Mar 2026] Understanding the Theoretical Foundations of Deep Neural Networks through Differential Equations Hongjue Zhao, Yizhuo Chen, Yuchen Wang, Hairong Qi, Lui Sha, Tarek Abdelzaher, Huajie Shao Deep neural networks (DNNs) have achieved remarkable empirical success, yet the absence of a principled theoretical foundation continues to hinder their systematic development. In this survey, we present differential equations as a theoretical foundation for understanding, analyzing, and improving DNNs. We organize the discussion around three guiding questions: i) how differential equations offer a principled understanding of DNN architectures, ii) how tools from differential equations can be used to improve DNN performance in a principled way, and iii) what real-world applications benefit from grounding DNNs in differential equations. We adopt a two-fold perspective spanning the model level, which interprets the whole DNN as a differential equation, and the layer level, which models individual DNN components as differential equations. From these two perspectives, we review how this framework connects model design, theoretical analysis, and performance improvement. We further discuss real-world applications, as well as key challenges and opportunities for future research. Subjects: Artificial Intelligence (cs.AI) Cite as: arXiv:2603.18331 [cs.AI]   (or arXiv:2603.18331v1 [cs.AI] for this version)   https://doi.org/10.48550/arXiv.2603.18331 Focus to learn more Submission history From: Hongjue Zhao [view email] [v1] Wed, 18 Mar 2026 22:41:02 UTC (2,090 KB) Access Paper: HTML (experimental) view license Current browse context: cs.AI < prev   |   next > new | recent | 2026-03 Change to browse by: cs References & Citations NASA ADS Google Scholar Semantic Scholar Export BibTeX Citation Bookmark Bibliographic Tools Bibliographic and Citation Tools Bibliographic Explorer Toggle Bibliographic Explorer (What is the Explorer?) Connected Papers Toggle Connected Papers (What is Connected Papers?) Litmaps Toggle Litmaps (What is Litmaps?) scite.ai Toggle scite Smart Citations (What are Smart Citations?) Code, Data, Media Demos Related Papers About arXivLabs Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?)