An Introduction to Gossip Protocol Based Learning in Peer-to-Peer Federated Learning

Machine learning (ML) has been progressively implemented in a distributed manner to harness the data abundance produced on billions of end user devices. Federated learning (FL) is a type of distributed machine learning that provides robust data privacy by training ML models locally on each participating node without directly exchanging raw data with others. The local ML model updates from all nodes are aggregated in order to obtain a global model. There are different ways to aggregate local model updates for obtaining a global model, such as centralized and decentralized/peer-to-peer. Centralized FL (CFL) requires a server to collect and aggregate all model updates for producing a global model, whereas decentralized FL (DFL)/peer-to-peer FL (P2PFL) requires coordination among all the nodes to communicate and aggregate all models updates and produce a global model. In CFL, the server can pose a bottleneck problem as it is a single point of failure, though, this issue can be resolved using DFL/P2PFL. However, the design of the decentralized/peer-to-peer architecture is complex and challenging, and may incur significant communication overhead due to a large number of nodes involved in the learning process. Gossip protocols are one of the most effective ways to communicate in DFL/P2PFL and optimises its performance, therefore, this decentralized learning is also known as gossip protocol-based learning. Considering the importance of gossip protocol in DFL/P2PFL, this paper will analyse gossip protocol, working of gossip protocol, types of gossip protocol, benefits and limitations of gossip protocol, usages of gossip protocol in DFL/P2PFL and its two main types structured P2PFL and unstructured P2PFL.