Cross-Entropy Games and Frost Training

Computer Science > Artificial Intelligence [Submitted on 26 May 2026] Cross-Entropy Games and Frost Training Arthur Renard, Franck Gabriel, Valentin Hartmann, Clément Hongler We present Frost Training, a method for improving Monte Carlo-based policy optimization for a large family of LLM-as-a-judge tasks called Cross-Entropy Games. The key idea is to exploit the gradient of the reward function in embedding space. This signal is used in the Greedy Coordinate Gradient (GCG) jailbreaking technique; we demonstrate for the first time that it can also be used to boost model training. We validate our method using GRPO training for maximum-likelihood infilling. Frost Training improves the model's ability to generate high-scoring outputs, reaching higher maximum scores in a best-of-k setting, and does so at an increased speed. Comments: 14 pages, 6 figures Subjects: Artificial Intelligence (cs.AI) Cite as: arXiv:2605.27701 [cs.AI]   (or arXiv:2605.27701v1 [cs.AI] for this version)   https://doi.org/10.48550/arXiv.2605.27701 Focus to learn more Submission history From: Clément Hongler [view email] [v1] Tue, 26 May 2026 21:20:45 UTC (183 KB) Access Paper: HTML (experimental) view license Current browse context: cs.AI < prev   |   next > new | recent | 2026-05 Change to browse by: cs References & Citations NASA ADS Google Scholar Semantic Scholar Export BibTeX Citation Bookmark Bibliographic Tools Bibliographic and Citation Tools Bibliographic Explorer Toggle Bibliographic Explorer (What is the Explorer?) Connected Papers Toggle Connected Papers (What is Connected Papers?) Litmaps Toggle Litmaps (What is Litmaps?) scite.ai Toggle scite Smart Citations (What are Smart Citations?) Code, Data, Media Demos Related Papers About arXivLabs Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?)