Abstract
Federated learning (FL) with noisy labels poses a significant challenge.
Existing methods designed for handling noisy labels in centralized learning
tend to lose their effectiveness in the FL setting, mainly due to the small
dataset size and the heterogeneity of client data. While some attempts have
been made to tackle FL with noisy labels, they primarily focused on scenarios
involving class-conditional noise. In this paper, we study the more challenging
and practical issue of instance-dependent noise (IDN) in FL. We introduce a
novel algorithm called FedBeat (Federated Learning with Bayesian
Ensemble-Assisted Transition Matrix Estimation). FedBeat aims to build a global
statistically consistent classifier using the IDN transition matrix (IDNTM),
which encompasses three synergistic steps: (1) A federated data extraction step
that constructs a weak global model and extracts high-confidence data using a
Bayesian model ensemble method. (2) A federated transition matrix estimation
step in which clients collaboratively train an IDNTM estimation network based
on the extracted data. (3) A federated classifier correction step that enhances
the global model's performance by training it using a loss function tailored
for noisy labels, leveraging the IDNTM. Experiments conducted on CIFAR-10 and
SVHN verify that the proposed method significantly outperforms state-of-the-art
methods.