PA2D-MORL: Pareto Ascent Directional Decomposition based Multi-Objective Reinforcement Learning

Computer Science > Artificial Intelligence [Submitted on 20 Mar 2026] PA2D-MORL: Pareto Ascent Directional Decomposition based Multi-Objective Reinforcement Learning Tianmeng Hu, Biao Luo Multi-objective reinforcement learning (MORL) provides an effective solution for decision-making problems involving conflicting objectives. However, achieving high-quality approximations to the Pareto policy set remains challenging, especially in complex tasks with continuous or high-dimensional state-action space. In this paper, we propose the Pareto Ascent Directional Decomposition based Multi-Objective Reinforcement Learning (PA2D-MORL) method, which constructs an efficient scheme for multi-objective problem decomposition and policy improvement, leading to a superior approximation of Pareto policy set. The proposed method leverages Pareto ascent direction to select the scalarization weights and computes the multi-objective policy gradient, which determines the policy optimization direction and ensures joint improvement on all objectives. Meanwhile, multiple policies are selectively optimized under an evolutionary framework to approximate the Pareto frontier from different directions. Additionally, a Pareto adaptive fine-tuning approach is applied to enhance the density and spread of the Pareto frontier approximation. Experiments on various multi-objective robot control tasks show that the proposed method clearly outperforms the current state-of-the-art algorithm in terms of both quality and stability of the outcomes. Comments: AAAI 2024 Subjects: Artificial Intelligence (cs.AI); Machine Learning (cs.LG) Cite as: arXiv:2603.19579 [cs.AI]   (or arXiv:2603.19579v1 [cs.AI] for this version)   https://doi.org/10.48550/arXiv.2603.19579 Focus to learn more Journal reference: Proceedings of the AAAI Conference on Artificial Intelligence, 38(11), 12547-12555, 2024 Related DOI: https://doi.org/10.1609/aaai.v38i11.29148 Focus to learn more Submission history From: Tianmeng Hu [view email] [v1] Fri, 20 Mar 2026 02:46:46 UTC (2,093 KB) Access Paper: HTML (experimental) view license Current browse context: cs.AI < prev   |   next > new | recent | 2026-03 Change to browse by: cs cs.LG References & Citations NASA ADS Google Scholar Semantic Scholar Export BibTeX Citation Bookmark Bibliographic Tools Bibliographic and Citation Tools Bibliographic Explorer Toggle Bibliographic Explorer (What is the Explorer?) Connected Papers Toggle Connected Papers (What is Connected Papers?) Litmaps Toggle Litmaps (What is Litmaps?) scite.ai Toggle scite Smart Citations (What are Smart Citations?) Code, Data, Media Demos Related Papers About arXivLabs Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?)