[N-1] Nonlinear Problems

11:45 AM - 12:00 PM JST(2:45 AM - 3:00 AM UTC)Candidate for Academic Incentive AwardCareer Explorer

[N-1-32] Hysteresis-Inspired Cross-Layer Attention for ViTs

◎△Jirayus Lapamnuaypol¹, Kenya Jin'no¹ (1. Tokyo City University)

Keywords:

Attention,ViT

Vision Transformers (ViTs) compute attention independently at each layer, potentially limiting attention consistency across depth. Inspired by hysteresis, the proposed methods propagate historical attention information across layers. Experiments on CIFAR-100
demonstrate intermediate attention consistency while
maintaining competitive classification accuracy.