[4M4-GS-2e] Machine learning

2:00 PM - 2:15 PM JST(5:00 AM - 5:15 AM UTC)

[4M4-GS-2e-03] Kernel Occupation Readout for Oscillatory Recurrent Neural Networks

〇Yuto Inui¹, Masahiro Ikeda^1,2, Takuya Konishi^1,2, Yoshinobu Kawahara^1,2 (1. The University of Osaka, 2. RIKEN)

Keywords:

oscillatory recurrent neural networks,reservoir computing,kernel mean embedding,occupation measure,sequence classification

Oscillatory recurrent neural networks achieve strong performance on sequence tasks. However, readouts based on the last hidden state not only fail to fully exploit the trajectory of hidden states but can also be vulnerable to phase shifts. Mean pooling mitigates this vulnerability, yet it still does not sufficiently leverage information contained in the trajectory. To address these issues, we propose the kernel occupation readout (KOR), which applies the kernel mean embedding to the occupation measure induced by the trajectory of hidden states and performs linear classification based on distributional information beyond the mean, including higher-order moments. Experiments on benchmarks of sequence classification show that KOR improves accuracy over last-state and mean-pooling readouts in many settings. Moreover, a robustness evaluation demonstrates that KOR is more robust to phase shifts than last-state and mean-pooling readouts.