[3E1-GS-2d] Machine learning

9:45 AM - 10:00 AM JST(12:45 AM - 1:00 AM UTC)

[3E1-GS-2d-04] Viewpoint-Robust Latent Action Learning Using Zero-Action Regularization

〇Yusei Koen¹, Masahiro Suzuki¹, Yutaka Matsuo¹ (1. The University of Tokyo)

Keywords:

Latent Action,Imitation Learning,Representation Learning

In robot policy learning, latent action learning is a promising approach for reducing data collection costs. However, because latent actions are trained to explain changes between consecutive observations, they can be confounded by action-irrelevant visual distractions, leading to representations that are less useful for control. Viewpoint changes are a typical distraction, yet their impact on existing latent-action methods has not been analyzed. In this work, we evaluate existing Latent Action Models (LAMs) under viewpoint shifts and show that rollout performance degrades substantially as the intensity of viewpoint changes increases. To improve robustness, we propose Zero-Action Regularization (ZAR), which encourages the decoded action from pairs of identical observations to be close to zero. This anchors the latent action space around a zero-action reference and promotes viewpoint-invariant action outputs. Experiments demonstrate that incorporating ZAR into an existing LAM consistently improves rollout performance under viewpoint-shifted conditions.