A Scalable and Privacy-Enhanced Federated Learning Framework with Adaptive Trade-offs Between Communication Efficiency, Privacy Guarantees, and Model Performance in Non-IID Environments
DOI:
https://doi.org/10.15408/frxdzc84Keywords:
adaptive algorithms, data heterogeneity, differential privacy, distributed systems, scalabilityAbstract
Federated learning (FL) has become a promising paradigm for collaborative machine learning that preserves the privacy of distributed data sources. However, implementing privacy-preserving federated learning (PPFL) in real-world settings poses several critical challenges, particularly in balancing communication efficiency, strong privacy guarantees, and reliable model performance. These issues are further exacerbated in non-IID (non-independent and identically distributed) environments, which are common in decentralized data scenarios. This study introduces a scalable framework for PPFL that incorporates an adaptive mechanism to optimize trade-offs among communication, privacy, and performance, tailored to dynamic, resource-constrained settings. The proposed framework integrates advanced differential privacy techniques with efficient communication strategies and employs robust aggregation algorithms to address data heterogeneity. Analytical evaluations highlight the scalability and effectiveness of the approach, while experimental validations demonstrate its advantages in terms of privacy-accuracy trade-offs across diverse datasets, including applications in healthcare and IoT. This work contributes to enhancing the practicality of FL systems by demonstrating a 6.5% accuracy improvement on CIFAR-10 in non-IID settings, maintaining 87.2% accuracy at a strict privacy budget of ε=1.0, and reducing communication overhead by 40% compared to baselines, addressing key barriers to deployment and setting a foundation for future research in dynamic, privacy-preserving machine learning systems.
Keywords: Adaptive algorithms; Data heterogeneity; Differential privacy; Distributed systems; Scalability.
Abstrak
Pembelajaran terfederasi (Federated learning) telah menjadi paradigma yang menjanjikan untuk pembelajaran mesin kolaboratif yang menjaga privasi sumber data terdistribusi. Namun, penerapan pembelajaran terfederasi yang menjaga privasi (PPFL) dalam dunia nyata menghadapi beberapa tantangan kritis, terutama dalam mencapai keseimbangan antara efisiensi komunikasi, jaminan privasi yang kuat, dan kinerja model yang andal. Masalah-masalah ini semakin diperparah dalam lingkungan non-IID (non-independen dan terdistribusi identik), yang umum terjadi dalam skenario data terdesentralisasi. Artikel ini memperkenalkan kerangka kerja yang skalabel untuk PPFL yang menggabungkan mekanisme adaptif untuk mengoptimalkan trade-off antara komunikasi, privasi, dan kinerja, yang disesuaikan dengan pengaturan dinamis dan terbatas sumber daya. Kerangka kerja yang diusulkan mengintegrasikan teknik privasi diferensial tingkat lanjut dengan strategi komunikasi yang efisien dan menggunakan algoritma agregasi yang tangguh untuk mengatasi heterogenitas data. Evaluasi analitis menyoroti skalabilitas dan efektivitas pendekatan ini, sementara validasi eksperimental menunjukkan keunggulannya dalam hal trade-off privasi-akurasi di berbagai set data, termasuk aplikasi di bidang kesehatan dan IoT. Hal ini berkontribusi dalam meningkatkan kepraktisan sistem FL dengan menunjukkan peningkatan akurasi sebesar 6,5% pada CIFAR-10 dalam pengaturan non-IID, mempertahankan akurasi sebesar 87,2% pada anggaran privasi yang ketat sebesar ε=1,0, dan mengurangi overhead komunikasi sebesar 40% dibandingkan dengan model dasarnya, mengatasi hambatan utama untuk penerapan dan menetapkan landasan untuk penelitian selanjutnya dalam sistem pembelajaran mesin yang dinamis dan menjaga privasi.
Kata Kunci: Algoritma adaptif; Heterogenitas data; Privasi diferensial; Sistem terdistribusi; Skalabilitas.
2020MSC: 68T07, 68W15.
References
Abadi, M., Chu, A., Goodfellow, I., McMahan, H. B., Mironov, I., Talwar, K., & Zhang, L. (2016). Deep Learning with Differential Privacy. Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security (CCS '16), 308-318. https://doi.org/10.1145/2976749.2978318
Dwork, C., & Roth, A. (2014). The Algorithmic Foundations of Differential Privacy. Foundations and Trends in Theoretical Computer Science, 9(3-4), 211-407. https://doi.org/10.1561/0400000042
McMahan, H. B., Moore, E., Ramage, D., Hampson, S., & Arcas, B. A. y. (2017). Communication-Efficient Learning of Deep Networks from Decentralized Data. Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS), 1273-1282. https://arxiv.org/abs/1602.05629
Bonawitz, K., Ivanov, V., Kreuter, B., et al. (2017). Practical Secure Aggregation for Privacy-Preserving Machine Learning. Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security (CCS '17), 1175-1191. https://doi.org/10.1145/3133956.3133982
McMahan, H. B., Ramage, D., & Talwar, K. (2018). Federated Learning: Collaborative Machine Learning Without Centralized Training Data. Google AI Blog. https://research.google/pubs/pub45648/
Konečny, J., McMahan, H. B., Yu, F. X., Richtárik, P., Suresh, A. T., & Bacon, D. (2016). Federated Learning: Strategies for Improving Communication Efficiency. arXiv preprint arXiv:1610.05492. https://arxiv.org/abs/1610.05492
Nishio, T., & Yonetani, R. (2019). Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edge. ICC 2019 - 2019 IEEE International Conference on Communications (ICC), 1-7. https://doi.org/10.1109/ICC.2019.8761315
Zhao, Y., Li, M., Lai, L., Suda, N., Civin, D., & Chandra, V. (2018). Federated Learning with Non-IID Data. arXiv preprint arXiv:1806.00582. https://arxiv.org/abs/1806.00582
Agarwal, N., Charles, Z., Duchi, J., et al. (2020). Personalized Federated Learning: A Meta-Learning Approach. Advances in Neural Information Processing Systems (NeurIPS). https://proceedings.neurips.cc/paper/2020/hash/c5d7368098e32e6dcd065b53161e0b16-Abstract.html
Li, T., Sahu, A. K., Talwalkar, A., & Smith, V. (2020). Federated Optimization in Heterogeneous Networks. Proceedings of Machine Learning and Systems 2020 (MLSys). https://proceedings.mlsys.org/paper/2020/file/38af86134b75ce6d78e44b51fbe4bfcc-Paper.pdf
Yang, Q., Liu, Y., Chen, T., & Tong, Y. (2019). Federated Machine Learning: Concept and Applications. ACM Transactions on Intelligent Systems and Technology (TIST), 10(2), 12. https://doi.org/10.1145/3298981
Wang, J., Liu, Q., Liang, H., et al. (2020). Optimizing Federated Learning on Resource-Constrained Edge Nodes. Proceedings of the 39th IEEE International Conference on Distributed Computing Systems (ICDCS), 1232-1241. https://doi.org/10.1109/ICDCS47251.2020.00123
Mironov, I. (2017). Rényi Differential Privacy. 2017 IEEE 30th Computer Security Foundations Symposium (CSF), 263-275. https://doi.org/10.1109/CSF.2017.11
Dwork, C., Rothblum, G. N., & Vadhan, S. P. (2010). Boosting and Differential Privacy. 2010 IEEE 51st Annual Symposium on Foundations of Computer Science, 51-60. https://doi.org/10.1109/FOCS.2010.12
Hard, A., Rao, K., Mathews, R., et al. (2018). Federated Learning for Mobile Keyboard Prediction. arXiv preprint arXiv:1811.03604. https://arxiv.org/abs/1811.03604
Sun, Y., Zhang, A., Li, H., et al. (2020). Federated Learning with Privacy Preservation in Industrial IoT. IEEE Transactions on Industrial Informatics, 16(5), 3174-3184. https://doi.org/10.1109/TII.2019.2944397
Kairouz, P., McMahan, H. B., Al-Shedivat, M., et al. (2021). Advances and Open Problems in Federated Learning. Foundations and Trends in Machine Learning, 14(1-2), 1-210. https://doi.org/10.1561/2200000083
Truex, S., Liu, L., Gursoy, M. E., et al. (2019). A Hybrid Approach to Privacy-Preserving Federated Learning. Proceedings of the 2019 IEEE International Conference on Big Data (Big Data), 1901-1910. https://doi.org/10.1109/BigData47090.2019.9006110
Chai, Z., Sun, C., Lyu, X., et al. (2020). Secure Federated Learning for Internet of Medical Things. IEEE Internet of Things Journal, 7(4), 3447-3459. https://doi.org/10.1109/JIOT.2020.2966437
Mohassel, P., & Zhang, Y. (2017). SecureML: A System for Scalable Privacy-Preserving Machine Learning. 2017 IEEE Symposium on Security and Privacy (SP), 19-38. https://doi.org/10.1109/SP.2017.12
Jayaraman, B., & Evans, D. (2019). Evaluating Differentially Private Machine Learning in Practice. Proceedings of the 28th USENIX Security Symposium, 1895-1912. https://www.usenix.org/conference/usenixsecurity19/presentation/jayaraman
Wei, K., Li, Q., Ma, T., et al. (2020). Federated Learning with Differential Privacy: Algorithms and Experiments. IEEE Transactions on Information Forensics and Security, 15(2), 254-267. https://doi.org/10.1109/TIFS.2019.2940587
Jiang, Y., Zhang, Z., Chen, X., et al. (2019). Federated Learning in Cloud-Edge Computing: A Survey. IEEE Transactions on Knowledge and Data Engineering, 1-14. https://doi.org/10.1109/TKDE.2019.2942708
Li, T., Sanjabi, M., Beirami, A., et al. (2020). Fair Resource Allocation in Federated Learning. Proceedings of the 8th International Conference on Learning Representations (ICLR). https://openreview.net/forum?id=ByexElSYDr
Yang, Z., Zhang, Y., Liu, L., et al. (2019). Privacy-Preserving Deep Learning via Additively Homomorphic Encryption. IEEE Transactions on Information Forensics and Security, 15(1), 3078-3091. https://doi.org/10.1109/TIFS.2019.2941121
Zhang, X., Guo, J., & Wang, Y. (2021). Adaptive Federated Learning for Smart IoT Systems. ACM Transactions on Internet Technology (TOIT), 21(3), 1-19. https://doi.org/10.1145/3439824
Downloads
Published
Issue
Section
License
Copyright (c) 2025 Milad Rahmati
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
