[WP-F] Poster Session 2

[WP-F-9] From In-Network AllReduce to In-NIC AllReduce in Support of Optically Switched GPU Networks

○Libin Wang¹, Hongxiang Guo¹, Cen Wang², Yuepeng Wu¹, Xiyang Lan¹, Jian Wu¹ (1. Beijing University of Posts and Telecommunications (China), 2. KDDI Research (Japan))

Keywords:

Distributed Learning,AllReduce,In-Network Computing,Optical Switching,Wavelength-selective Switches for Routing,FPGA-based NIC

We propose in-NIC AllReduce to lower communication times, adapting to optically switched GPU network for distributed training. The experimental results show a 1.89× acceleration aligning with a theoretical analysis of 1.90× acceleration on average.