Presentation Information
[4Yin-A-08]Safety-Aligned Vision-Language-Action Models via Plackett-Luce Preference Learning with NLL Regularization
〇Yun Li1, Simon Thompson2, Ehsan Javanmardi1, Alex Orsholits1, Manabu Tsukada1 (1. The University of Tokyo, 2. TIER IV, Inc.)
Keywords:
Autonomous Driving,Vision-Language-Action Model,Large Language Model,Preference Learning,Safety
Vision-Language-Action (VLA) models demonstrate strong reasoning capabilities for autonomous driving, yet aligning them with strict safety constraints remains challenging due to training data imbalance and probability collapse in preference learning. We propose PL-DPO-NLL, a novel alignment framework that integrates risk-ranked multi-preference learning with Negative Log-Likelihood (NLL) regularization to align VLA models with expert safety constraints. Specifically, we (1) employ a Plackett-Luce model to learn from multiple action candidates ranked by risk level, (2) dynamically weight gradients according to scene criticality, and (3) introduce a regularization term that preserves expert action probability. Evaluation on the CARLA benchmark shows that the proposed method achieves a Driving Score of 58.26 (+8.4% over baseline), Route Completion of 65.9%, and Infraction Penalty of 0.891.