Presentation Information
[BT-1-04]High-Speed Access Technologies for Remote Large-Scale Data in Geo-Distributed Computing Infrastructure
〇Hiroki Kano1 (1. TOYOTA Motor Corporation)
Keywords:
Geo-Distributed Computing Infrastructure,Data I/O,Cache,RDMA,AI Training
Renewable energy is sometimes curtailed when power supply exceeds demand. To address this issue, geographically distributed computing systems have attracted attention. These systems assign AI training jobs to regions with available power capacity. However, computing resources and training data may be located far apart. In this case, remote data access over a wide area network (WAN) becomes a bottleneck and significantly increases training time.
This paper presents two approaches to accelerate remote access to large-scale data. The first approach extends Remote Direct Memory Access (RDMA) to long-distance networks to improve communication performance. The second approach combines a distributed cache with a job scheduler to reduce the number of remote data accesses.
This paper presents two approaches to accelerate remote access to large-scale data. The first approach extends Remote Direct Memory Access (RDMA) to long-distance networks to improve communication performance. The second approach combines a distributed cache with a job scheduler to reduce the number of remote data accesses.