Presentation Information
[U18-01]Analysis of Deep Learning Dynamics for Understanding and Controlling AI: An approach from Physics★Invited Papers
*Imaizumi Masaaki1,2,3 (1.The University of Tokyo, 2.RIKEN AIP, 3.Kyoto University)
Alongside the advancement of AI, theories for understanding the underlying mechanisms of deep learning have also evolved. Particular attention has been focused on the role of learning dynamics in explaining the predictive capabilities of deep learning, yet elucidating the learning dynamics emerging within the multi-layered structure of neural networks remains an evolving field. This talk introduces several research findings analyzing the dynamics of neural networks. First, we report a theoretical result showing that time structure discrepancies across different layers of a neural network affect learning efficiency. Second, we report techniques for describing and estimating the learning dynamics and prediction performance even in neural networks with many layers. If time permits, we will also introduce patterns of variable distributions that emerge autonomously within neural networks, particularly Transformers. These analyses are achieved by incorporating insights from statistical mechanics and complex systems physics into machine learning.