Presentation Information
[A-14-24]A Survey on FPGA Implementation of Sequential Models
△Koyo Uekubo1, Shintaro Shiba1,2, Yoshihiro Kawahara1 (1. The University of Tokyo, 2. Keio University)
Keywords:
FPGA,Sequential Model,AI,Neural Network,RNN,Transformer
In recent years, with the rapid proliferation of large language models (LLMs), the computational efficiency of the sequential models that underpin them has once again drawn attention. While the currently mainstream Transformer offers high parallelism and expressive power, it faces challenges in long-sequence processing and inference-time latency. By contrast, classical recurrent neural networks (RNNs) are independent of sequence length but are difficult to parallelize. Linear RNNs, such as the state space models (SSMs) that have attracted attention in recent years, are models that mitigate both of these challenges. In this paper, we conduct a systematic, algorithm-level comparison of three types of sequential models: classical RNNs, Transformers, and linear RNNs. Furthermore, we organize the design challenges and existing efforts involved in implementing each model on an FPGA, and discuss the hardware affinity of each architecture. Through this, we aim to provide guidelines for selecting sequential models in edge inference and low-power environments.