Federated Learning

In Brief

Federated Learning is a paradigm of distributed processing, where models instead of data are shared among peers.

More in Detail

Data are often massively distributed over different parties, such as mobile devices and organizations. Due to privacy restrictions and regulations, like the GDPR, these distributed datasets cannot be transmitted to a central server to train a unique ML model. To address this issue, in 2016 McMahan et al. [1] proposed the idea of Federated Learning (FL), which allows a set of distributed parties to train a global model while keeping their data private.

Depending on the clients nature, we can identify two main scenarios: the cross-device and cross-silo FL. In the first case the clients are mobile devices (smartphones, laptops, sensors), while in the second are organizations (hospitals, banks, data centers. One important aspect that differentiates the two scenarios is the clients’ statefulness. While in the cross-device setting, the parties are supposed to be stateless, since they are supposed to be able to leave the federation at any time due to connection problems, the clients in the cross-silo scenario are assumed to be stateful, that is they can save intermediate states of the training process and reuse them later on if needed.

Another key factor in FL is the architecture employed. The most widely used architecture is the centralized one, where a central server orchestrates the communication between the clients and the server itself. However, to reduce the workload on the central server, Liu et al. [2] suggested adopting a hierarchical architecture, putting one or more intermediate layers made up of proxies between the server and the participants.

Bibliography

1

J. Konečný, H.B. McMahan, D. Ramage, and P. Richtárik. Federated optimization: distributed machine learning for on-device intelligence. CoRR, 2016.

2

L. Liu, J. Zhang, S. Song, and K.B. Letaief. Client-edge-cloud hierarchical federated learning. In IEEE International Conference on Communications. 2020.

This entry was readapted from Comandé et al. Elgar Encyclopedia of Law and Data Science. Edward Elgar Publishing (2022) ISBN: 978 1 83910 458 9 by Francesca Pratesi, Roberto Pellungrini, and Anna Monreale.