講演情報
[16a-K306-5]Analysis of Non-Volatile Memory-based CiM Non-Idealities on Small Language Models’ Performance and Robustness During Inference
〇(D)Adil Padiyal1, Tao Wang1, Naoko Misawa1, Chihiro Matsui1, Ken Takeuchi1 (1.The University Of Tokyo)
キーワード：
Computation in Memory、Small Language Model、Convolutional Neural Networks
The novel paradigm of computation-in-memory (CiM) aims to eliminate the energy efficiency due to extraordinarily high memory accesses by removing the need to access memory in the traditional von Neumann architecture by performing computation where the data resides i.e. the memory itself. This paper explores the impact of CiM systems based on non-volatile memory which incurs non-idealities in noise due to non-ideal memory systems. This research studies the impact of the said noise on the performance and robustness of small language models of varying model sizes while establishing a trend between robustness and model sizes. This study compares the noise tolerance of machine learning models based on vastly different architectures such as CNNs and SLMs. It is determined that SLMs’ sensitivity to memory non-idealities increases proportionally with model size i.e. larger models exhibiting higher susceptibility to noise. This research contributes valuable insights into the trade-offs between CiM non-idealities and model performance, emphasizing the importance of noise tolerance for CiM-based computing systems.