[2K4-GS-7b] Vision, speech media processing

2:45 PM - 3:00 PM JST(5:45 AM - 6:00 AM UTC)

[2K4-GS-7b-06] Diversity-Aware Max@k Optimization for Improving Best-of-N Performance in Image Generation with Diffusion Models

〇Ku Onoda¹, Yuta Oshima¹, Shohei Taniguchi¹, Soichiro Nishimori¹, Paavo Parmas¹, Hiroki Furuta, Yutaka Matsuo¹ (1. The University of Tokyo)

Keywords:

image generation,diffusion models,reinforcement learning

Recent advancements in reinforcement learning for image generation with diffusion models have focused on maximizing the expected reward of single generated samples. However, in practical deployment, generation is often performed using a Best-of-N (BoN) strategy, where multiple candidates are generated and the best one is selected. Standard optimization methods tend to collapse the generation distribution to a single high-reward mode, severely limiting the potential gains from BoN sampling due to a lack of diversity. In this paper, we propose a diversity-aware optimization method that directly maximizes the max@k objective while maintaining diversity. By guiding generated samples into distinct semantic classes inherent in the base model, our method prevents mode collapse and ensures that the set of candidates covers diverse high-reward regions. We demonstrate that our approach achieves superior BoN performance compared to standard baselines while preserving generative diversity.