[2K6-GS-7d] Vision, speech media processing

6:15 PM - 6:30 PM JST(9:15 AM - 9:30 AM UTC)

[2K6-GS-7d-04] FasTARFlow: Distilling Transformer-based Autoregressive Flows into Fast Inverse Autoregressive Models

〇Kai Yamashita¹, Shohei Taniguchi¹, Masahiro Suzuki¹, Yutaka Matsuo¹ (1. The University of Tokyo)

Keywords:

Generative Model,Normalizing Flows,Distillation

TARFlow introduces Transformer-based autoregressive flows that overcome challenges of expressiveness and training in Normalizing Flows (NFs), achieving state-of-the-art likelihoods and high-quality samples. However, its autoregressive nature leads to slow sampling, leaving limitations in practical applications. In this work, we propose FasTARFlow, a distillation framework that transfers knowledge from a trained TARFlow model (teacher) to a fast, parallelizable inverse autoregressive flow (IAF) model (student). The distilled model retains competitive sample quality while substantially accelerating inference. Experiments on standard image benchmarks demonstrate that FasTARFlow matches TARFlow in sample quality, while achieving significantly faster sampling, thereby advancing the practicality of NF-based generative modeling.