Jesus Martinez del Rincon, & Ihsen Alouani
Centre For Secure Information Technology (CSIT) – Queens University Belfast
Federated Learning (FL) is rapidly emerging as a privacy-first machine learning paradigm, enabling organisations to train models collaboratively without sharing raw data. Instead of centralising sensitive information, FL keeps data on local devices—such as smartphones, medical systems, or edge hardware—and shares only model updates. This architecture aligns strongly with modern data protection demands in sectors like healthcare, finance, and cross-border cloud services, where compliance and confidentiality are critical.
From a strategic standpoint, FL can be considered as a “privacy by design” AI infrastructure. It reduces regulatory risk, enhances user trust, and enables collaboration across institutions that would otherwise be unable to share data. However, this privacy advantage introduces a crucial trade-off: system integrity.
Because FL relies on decentralised contributions, it becomes vulnerable to poisoning and backdoor attacks. Malicious participants can manipulate model updates—either by degrading overall model performance (untargeted attacks) or by inserting hidden behaviours that activate under specific conditions (targeted backdoors). Importantly, privacy-preserving techniques such as secure aggregation—designed to hide individual contributions—can inadvertently make it harder to detect these malicious updates.
To address these risks, researchers and practitioners are developing robust aggregation methods. These include statistical techniques such as Median, Trimmed Mean, Krum, Multi-Krum, and Bulyan, which aim to filter out anomalous or adversarial updates before they impact the global model. While promising, these methods often rely on assumptions—such as attacks being significantly different from normal updates—that may not hold in real-world scenarios.
The CSIT LASR project positions itself at the forefront of AI security innovation, tackling the critical balance between privacy and integrity in FL systems. Its core objective is to develop a comprehensive benchmarking framework that evaluates both attack strategies and defensive mechanisms under realistic conditions. Unlike existing research that may rely on simplified threat models, this approach emphasises practical deployment environments, particularly edge-device ecosystems with large numbers of lightweight nodes.
The benchmark will simulate a range of attack types—including poisoning and backdoor insertion—alongside multiple aggregation strategies. It will allow users to systematically assess performance using key metrics such as overall accuracy, targeted attack success rates, and the proportion of malicious contributions influencing the model. This provides a decision-support tool for organisations looking to deploy secure FL solutions.
A key innovation lies in challenging conventional assumptions. For example, instead of assuming attackers produce easily detectable anomalies, the project explores stealthier, in-distribution attacks that mimic legitimate updates. This reflects real-world adversarial behaviour and ensures that proposed defences are robust under realistic threat conditions.
The impact of this work is substantial. By identifying vulnerabilities early and proposing effective mitigations, the project supports the secure scaling of federated learning technologies. This is particularly valuable in high-stakes industries where both privacy and reliability are non-negotiable.
In summary, this initiative reframes FL not just as a privacy solution, but as a secure, enterprise-ready AI framework. It highlights that achieving trust in distributed AI systems requires balancing two pillars: protecting user data and safeguarding model integrity. Organisations that invest in both will be best positioned to unlock the full potential of collaborative, privacy-preserving machine learning.
