Beyond the Central Server: Decentralised and Asynchronous Federated Learning

The project

Federated Learning (FL) trains machine learning models across many data owners – devices, organisations or edge nodes – without moving the raw data. Each participant trains locally and shares only model updates, which are then aggregated into a global model. This makes FL attractive wherever data is sensitive or too large to centralise, e.g. in healthcare, industry, energy and the Internet of Things (IoT).

Yet in its most common form, FL is distributed only with respect to data. Two key assumptions remain centralised: (1) a single aggregation server coordinates the whole process, and (2) training proceeds in synchronised rounds, where the server waits for selected clients before updating the global model. The first assumption makes the aggregator a single point of failure and a performance bottleneck. The second makes training only as fast as the slowest participant, and fragile when devices drop out, go offline or produce data at different rates – which is the norm rather than the exception in cyber-physical systems working with live sensor data.

Research focus

This MSc project will investigate how to relax one or both of these assumptions, and what the trade-offs are. Depending on the student's interests, the thesis may focus on one or a combination of the following directions:

• Decentralised aggregation: peer-to-peer and gossip learning, where nodes exchange and merge models directly with their neighbours instead of a central server, and how the network topology affects convergence and communication cost.
• Resilient coordination: replicating the aggregator's state (e.g. using consensus protocols such as Raft), or using hierarchical/clustered aggregation, so that another node can take over from the latest checkpoint if the aggregator fails.
• Asynchronous and semi-asynchronous training: letting participants contribute at their own pace, while dealing with stale updates, model inconsistency, communication overhead and concurrency control (e.g. staleness-aware or buffered aggregation).
• Combining the two: asynchronous, decentralised learning in dynamic networks where nodes join, leave and fail, and where data is non-IID across nodes.

The student will build a prototype on top of an existing FL framework (e.g. Flower) and evaluate it on an emulated or physical edge/IoT testbed, comparing it with a conventional centralised, synchronous baseline in terms of model accuracy and convergence, communication overhead, and tolerance to failures and slow participants.

Expected results and learning outcome

• A working prototype of a decentralised and/or asynchronous FL system.
• An experimental evaluation showing when – and at what cost – decentralisation and asynchrony pay off.
• Hands-on experience with FL, distributed systems and edge computing, and an in-depth understanding of the trade-offs between accuracy, communication efficiency and resilience. Results of sufficient quality can be published at a scientific workshop or conference together with the supervisors.

Desired qualifications

• Experience with machine learning and common frameworks (PyTorch, TensorFlow, scikit-learn, Hugging Face, etc.)
• Good programming skills in Python
• Good understanding of distributed systems, networking and the IoT
• Experience with a Federated Learning framework such as Flower, TensorFlow Federated or FATE (desirable)

References

• T. Li, A. K. Sahu, A. Talwalkar, V. Smith. Federated learning: Challenges, methods, and future directions. IEEE Signal Processing Magazine, 2020.
• E. T. Martínez Beltrán et al. Decentralized federated learning: Fundamentals, state of the art, frameworks, trends, and challenges. IEEE Communications Surveys & Tutorials, 2023.
• I. Hegedűs, G. Danner, M. Jelasity. Decentralized learning works: An empirical comparison of gossip learning and federated learning. Journal of Parallel and Distributed Computing, 2021.
• C. Xu, Y. Qu, Y. Xiang, L. Gao. Asynchronous federated learning on heterogeneous devices: A survey. Computer Science Review, 2023.