Presentation Information
[B-15-13]Curriculum Learning for Noisy-Label-Robust Model-Circulation-Based FL
◎△Haruki Kainuma1, Takayuki Nishio1 (1. Science Tokyo)
Keywords:
Federated Learning,Curriculum Learning,Model-circulation-based Learning,Distributed Machine Learning
Tram-FL can realize communication-efficient Decentralized FL by training a single model while circulating it among data holders. However, the data held by FL participants may contain label errors, which degrade the performance of the trained model.
In this paper, we propose a new Tram-FL method that is robust to label errors by applying curriculum learning.
Since mislabeled samples tend to have large losses, the proposed method gradually expands the training set from low-loss samples, thereby suppressing the influence of label errors in the early stage of training. Because Tram-FL circulates a single model, nodes that do not currently hold the model cannot directly compute the loss of their local samples at the current time. Therefore, we introduce a mechanism for predicting the current loss from the temporal evolution of past losses.
Machine learning experiments using CIFAR-10 and ResNet-18 demonstrate that the proposed method achieves higher accuracy than Tram-FL.
In this paper, we propose a new Tram-FL method that is robust to label errors by applying curriculum learning.
Since mislabeled samples tend to have large losses, the proposed method gradually expands the training set from low-loss samples, thereby suppressing the influence of label errors in the early stage of training. Because Tram-FL circulates a single model, nodes that do not currently hold the model cannot directly compute the loss of their local samples at the current time. Therefore, we introduce a mechanism for predicting the current loss from the temporal evolution of past losses.
Machine learning experiments using CIFAR-10 and ResNet-18 demonstrate that the proposed method achieves higher accuracy than Tram-FL.