This repository maintains a collection of papers, articles, videos, frameworks, etc of federated learing, for the purpose of learning and research.
Note:
- Italic contents are the ones worth reading
- Last Update: 16 Aug, 2020
- Awesome Federated Learning
- Federated Learning Comic [Google Blog]
- Federated Learning: Collaborative Machine Learning without Centralized Training Data [Google Blog]
- GDPR, Data Shotrage and AI (AAAI-19) [Video]
- Federated Learning: Machine Learning on Decentralized Data (Google I/O'19) [Youtube]
- Federated Learning White Paper V1.0 [Paper]
- Federated learning: distributed machine learning with data locality and privacy [Blog]
- Federated Machine Learning: Concept and Applications [Paper]
- Federated Learning: Challenges, Methods, and Future Directions [Paper]
- Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection [Paper]
- Federated Learning in Mobile Edge Networks: A Comprehensive Survey [Paper]
- Federated Learning for Wireless Communications: Motivation, Opportunities and Challenges [Paper]
- Convergence of Edge Computing and Deep Learning: A Comprehensive Survey [Paper]
- Advances and Open Problems in Federated Learning [Paper]
- LEAF: A Benchmark for Federated Settings [Paper] [Github] [Recommend]
- A Performance Evaluation of Federated Learning Algorithms [Paper]
- Edge AIBench: Towards Comprehensive End-to-end Edge Computing Benchmarking [Paper]
- One-Shot Federated Learning [Paper]
- Federated Learning with Unbiased Gradient Aggregation and Controllable Meta Updating [Paper] (NIPS 2019 Workshop)
- Bayesian Nonparametric Federated Learning of Neural Networks [Paper] (ICML 2019)
- Agnostic Federated Learning [Paper] (ICML 2019)
- Federated Learning with Matched Averaging [Paper] (ICLR 2020)
- Astraea: Self-balancing federated learning for improving classification accuracy of mobile deep learning applications [Paper]
- Federated Learning with Non-IID Data [Paper]
- The Non-IID Data Quagmire of Decentralized Machine Learning [Paper]
- Robust and Communication-Efficient Federated Learning from Non-IID Data [Paper] [IEEE transactions on neural networks and learning systems]
- FedMD: Heterogenous Federated Learning via Model Distillation [Paper] (NIPS 2019 Workshop)
- First Analysis of Local GD on Heterogeneous Data [Paper]
- SCAFFOLD: Stochastic Controlled Averaging for On-Device Federated Learning [Paper]
- Federated Optimization for Heterogeneous Networks [Paper]
- On the Convergence of FedAvg on Non-IID Data [Paper] [OpenReview]
- Agnostic Federated Learning [Paper] (ICML 2019)
- Local SGD Converges Fast and Communicates Little [Paper]
- Improving Federated Learning Personalization via Model Agnostic Meta Learning [Paper] (NIPS 2019 Workshop)
- Adaptive Gradient-Based Meta-Learning Methods [Paper] (NIPS 2019 Workshop)
- Federated Adversarial Domain Adaptation [Paper] (ICLR 2020)
- LoAdaBoost: Loss-Based AdaBoost Federated Machine Learning on Medical Data [Paper]
- On Federated Learning of Deep Networks from Non-IID Data: Parameter Divergence and the Effects of Hyperparametric Methods [Paper] [Rejected in ICML 2020]
- Overcoming Forgetting in Federated Learning on Non-IID Data [Paper] [NIPS 2019 Workshop]
- FedMAX: Activation Entropy Maximization Targeting Effective Non-IID Federated Learning [Video] [NIPS 2019 Workshop]
- Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classification [Paper] [NIPS 2019 Workshop]
- Fair Resource Allocation in Federated Learning [Paper]
- Communication-efficient on-device machine learning: Federated distillation and augmentation under non-iid private data [Paper]
- Think Locally, Act Globally: Federated Learning with Local and Global Representations [Paper] [NIPS 2019 Workshop]
- Federated Meta-Learning with Fast Convergence and Efficient Communication [Paper]
- Federated Meta-Learning for Recommendation [Paper]
- Adaptive Gradient-Based Meta-Learning Methods [Paper]
- MOCHA: Federated Multi-Task Learning [Paper] [NIPS 2017] [Slides]
- Variational Federated Multi-Task Learning [Paper]
- Federated Kernelized Multi-Task Learning [Paper]
- Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraints [Paper] [NIPS 2019 Workshop]
- A Linear Speedup Analysis of Distributed Deep Learning with Sparse and Quantized Communication [Paper] [NIPS 2018]
- SCAFFOLD: Stochastic Controlled Averaging for On-Device Federated Learning [Paper]
- Federated Optimization for Heterogeneous Networks [Paper]
- On the Convergence of FedAvg on Non-IID Data [Paper] [OpenReview]
- Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descent [Paper] [NIPS 2017]
- Communication Efficient Decentralized Training with Multiple Local Updates [Paper]
- First Analysis of Local GD on Heterogeneous Data [Paper]
- MATCHA: Speeding Up Decentralized SGD via Matching Decomposition Sampling [Paper]
- Local SGD Converges Fast and Communicates Little [Paper]
- SlowMo: Improving Communication-Efficient Distributed SGD with Slow Momentum [Paper]
- Adaptive Federated Learning in Resource Constrained Edge Computing Systems [Paper] [IEEE Journal on Selected Areas in Communications, 2019]
- Parallel Restarted SGD with Faster Convergence and Less Communication: Demystifying Why Model Averaging Works for Deep Learning [Paper] [AAAI 2018]
- On the Linear Speedup Analysis of Communication Efficient Momentum SGD for Distributed Non-Convex Optimization [Paper] [ICML 2019]
- Communication-efficient on-device machine learning: Federated distillation and augmentation under non-iid private data [Paper]
- Convergence of Distributed Stochastic Variance Reduced Methods without Sampling Extra Data [Paper] [NIPS 2019 Workshop]
- Towards Federated Learning at Scale: System Design [Paper] [Must Read]
- Demonstration of Federated Learning in a Resource-Constrained Networked Environment [Paper]
- Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection [Paper]
- Applied Federated Learning: Improving Google Keyboard Query Suggestions [Paper]
- Federated Learning and Differential Privacy: Software tools analysis, the Sherpa.ai FL framework and methodological guidelines for preserving data privacy [Paper] (Startup)
- Communication-Efficient Learning of Deep Networks from Decentralized Data [Paper] [Github] [Google] [Must Read]
- Two-Stream Federated Learning: Reduce the Communication Costs [Paper] [2018 IEEE VCIP]
- Client-Edge-Cloud Hierarchical Federated Learning [Paper]
- PowerSGD: Practical Low-Rank Gradient Compression for Distributed Optimization [Paper] [NIPS 2019], Thijs Vogels, Sai Praneeth Karimireddy, and Martin Jaggi.
- Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training [Paper] [ICLR 2018] Yujun Lin, Song Han, Huizi Mao, Yu Wang, and William J Dally
- The Error-Feedback Framework: Better Rates for SGD with Delayed Gradients and Compressed Communication [Paper] Sebastian U Stich and Sai Praneeth Karimireddy, 2019.
- A Communication Efficient Collaborative Learning Framework for Distributed Features [Paper] [NIPS 2019 Workshop]
- Active Federated Learning [Paper] [NIPS 2019 Workshop]
- Communication-Efficient Distributed Optimization in Networks with Gradient Tracking and Variance Reduction [Paper] [NIPS 2019 Workshop]
- Gradient Descent with Compressed Iterates [Paper] [NIPS 2019 Workshop]
- Robust and Communication-Efficient Federated Learning from Non-IID Data [Paper], 2019
- Expanding the Reach of Federated Learning by Reducing Client Resource Requirements [Paper] Sebastian Caldas, Jakub Konecny, H Brendan McMahan, and Ameet Talwalkar, 2018
- Federated Learning: Strategies for Improving Communication Efficiency [Paper] [NIPS2016 Workshop] [Google]
- Natural Compression for Distributed Deep Learning [Paper] Samuel Horvath, Chen-Yu Ho, Ludovit Horvath, Atal Narayan Sahu, Marco Canini, and Peter Richtarik, 2019.
- FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantization [Paper], 2019
- ATOMO: Communication-efficient Learning via Atomic Sparsification [Paper] [NIPS 2018], H. Wang, S. Sievert, S. Liu, Z. Charles, D. Papailiopoulos, and S. Wright.
- vqSGD: Vector Quantized Stochastic Gradient Descent [Paper] Venkata Gandikota, Raj Kumar Maity, and Arya Mazumdar, 2019.
- QSGD: Communication-efficient SGD via gradient quantization and encoding [Paper] [NIPS 2017], Dan Alistarh, Demjan Grubic, Jerry Li, Ryota Tomioka, and Milan Vojnovic.
- cpSGD: Communication-efficient and differentially-private distributed SGD [Paper]
- Federated Optimization: Distributed Machine Learning for On-Device Intelligence [Paper] [Google]
- Distributed Mean Estimation with Limited Communication [Paper] [ICML 2017], Ananda Theertha Suresh, Felix X. Yu, Sanjiv Kumar, and H Brendan McMahan.
- Randomized Distributed Mean Estimation: Accuracy vs Communication [Paper] Frontiers in Applied Mathematics and Statistics, Jakub Konecny and Peter Richtarik, 2016
- Error Feedback Fixes SignSGD and other Gradient Compression Schemes [Paper] [ICML 2019], Sai Praneeth Karimireddy, Quentin Rebjock, Sebastian Stich, and Martin Jaggi.
- ZipML: Training Linear Models with End-to-End Low Precision, and a Little Bit of Deep Learning [Paper] [ICML 2017], H. Zhang, J. Li, K. Kara, D. Alistarh, J. Liu, and C. Zhang.
- eSGD: Communication Efficient Distributed Deep Learning on the Edge [Paper] [USENIX 2018 Workshop (HotEdge 18)]
- CMFL: Mitigating Communication Overhead for Federated Learning [Paper]
- Communication Compression for Decentralized Training [Paper] [NIPS 2018], H. Tang, S. Gan, C. Zhang, T. Zhang, and J. Liu.
- ๐ณ๐๐๐๐๐๐๐๐๐ฃ๐: Decentralization Meets Error-Compensated Compression [Paper] Hanlin Tang, Xiangru Lian, Shuang Qiu, Lei Yuan, Ce Zhang, Tong Zhang, and Ji Liu, 2019
-
Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edge [Paper] (FedCS)
-
Hybrid-FL for Wireless Networks: Cooperative Learning Mechanism Using Non-IID Data [Paper]
- Ask to upload some data from client to server
-
Efficient Training Management for Mobile Crowd-Machine Learning: A Deep Reinforcement Learning Approach [Paper]
- Reward function: accumulated data, energy consumption, training accuracy
-
Fair Resource Allocation in Federated Learning [Paper]
-
Low-latency Broadband Analog Aggregation For Federated Edge Learning [Paper]
-
Federated Learning over Wireless Fading Channels [Paper]
-
Federated Learning via Over-the-Air Computation [Paper]
- Asynchronous Federated Learning for Geospatial Applications [Paper] [ECML PKDD Workshop 2018]
- Asynchronous Federated Optimization [Paper]
- Adaptive Federated Learning in Resource Constrained Edge Computing Systems [Paper] [IEEE Journal on Selected Areas in Communications, 2019]
- Incentive Mechanism for Reliable Federated Learning: A Joint Optimization Approach to Combining Reputation and Contract Theory [Paper]
- Motivating Workers in Federated Learning: A Stackelberg Game Perspective [Paper]
- Incentive Design for Efficient Federated Learning in Mobile Networks: A Contract Theory Approach [2019] [Paper]
- A Quasi-Newton Method Based Vertical Federated Learning Framework for Logistic Regression [Paper] [NIPS 2019 Workshop]
- Can You Really Backdoor Federated Learning? [Paper]
- Model Poisoning Attacks in Federated Learning [Paper] [NIPS workshop 2018]
- Gradient-Leaks: Understanding and Controlling Deanonymization in Federated Learning [Paper] [NIPS 2019 Workshop]
- Quantification of the Leakage in Federated Learning [Paper]
- Applied Cryptography [Udacity]
- Cryptography basics
- A Brief Introduction to Differential Privacy [Blog]
- Deep Learning with Differential Privacy [Paper]
- Martin Abadi, Andy Chu, Ian Goodfellow, H. Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang.
- Learning Differentially Private Recurrent Language Models [Paper]
- Federated Learning with Bayesian Differential Privacy [Paper] [NIPS 2019 Workshop]
- Private Federated Learning with Domain Adaptation [Paper] [NIPS 2019 Workshop]
- cpSGD: Communication-efficient and differentially-private distributed SGD [Paper]
- Federated Learning with Bayesian Differential Privacy [Paper] [NIPS 2019 Workshop]
- Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data [Paper]
- Nicolas Papernot, Martiฬn Abadi, Uฬlfar Erlingsson, Ian J. Goodfellow, and Kunal Talwar.
- Private Aggregation of Teacher Ensembles (PATE)
- Scalable Private Learning with PATE [Paper]
- Extension of PATE
- The original PATE paper at ICLR 2017 and recording of the ICLR oral
- The ICLR 2018 paper on scaling PATE to large number of classes and imbalanced data.
- GitHub code repo for PATE
- GitHub code repo for the refined privacy analysis of PATE
- Simple Introduction to Sharmir's Secret Sharing and Lagrange Interpolation [Youtube]
- Secret Sharing, Part 1 [Blog]: Shamir's Secret Sharing & Packed Variant
- Secret Sharing, Part 2 [Blog]: Improve efficiency
- Secret Sharing, Part 3 [Blog]
-
Basics of Secure Multiparty Computation [Youtube]: based on Shamir's Secret Sharing
-
What is SPDZ?
-
The SPDZ Protocol [Blog]: implementation codes included
- Multiparty Computation from Somewhat Homomorphic Encryption [Paper]
- SPDZ introduction
- Practical Covertly Secure MPC for Dishonest Majority โ or: Breaking the SPDZ Limits [Paper]
- MASCOT: Faster Malicious Arithmetic Secure Computation with Oblivious Transfer [Paper]
- Removing the crypto provider and instead letting the parties generate these triples on their own
- Overdrive: Making SPDZ Great Again [Paper]
- Safetynets: Verifiable execution of deep neural networks on an untrusted cloud
-
Building Safe A.I. [Blog]
- A Tutorial for Encrypted Deep Learning
- Use Homomorphic Encryption (HE)
-
Private Deep Learning with MPC [Blog]
- A Simple Tutorial from Scratch
- Use Multiparty Compuation (MPC)
-
Private Image Analysis with MPC [Blog]
- Training CNNs on Sensitive Data
- Use SPDZ as MPC protocol
Helen: Maliciously Secure Coopetitive Learning for Linear Models [Paper] (NIPS 2019 Workshop)
- Privacy-Preserving Deep Learning [Paper]
- Privacy Partition: A Privacy-Preserving Framework for Deep Neural Networks in Edge Networks [Paper]
- Practical Secure Aggregation for Privacy-Preserving Machine Learning [Paper] (Google)
- Secure Aggregation: The problem of computing a multiparty sum where no party reveals its update in the clearโeven to the aggregator
- Goal: securely computing sums of vectors, which has a constant number of rounds, low communication overhead, robustness to failures, and which requires only one server with limited trust
- Need to have basic knowledge of cryptographic algorithms such as secret sharing, key agreement, etc.
- Practical Secure Aggregation for Federated Learning on User-Held Data [Paper] (Google)
- Highly related to Practical Secure Aggregation for Privacy-Preserving Machine Learning
- Proposed 4 protocol one by one with gradual improvement to meet the requirement of secure aggregation propocol.
- SecureML: A System for Scalable Privacy-Preserving Machine Learning [Paper]
- DeepSecure: Scalable Provably-Secure Deep Learning [Paper]
- Chameleon: A Hybrid Secure Computation Framework for Machine Learning Applications [Paper]
- Contains several MPC frameworks
- PySyft [Github]
- A Generic Framework for Privacy Preserving Peep Pearning [Paper]
- Tensorflow Federated [Web]
- FATE [Github]
- Baidu PaddleFL [Github]
- Nvidia Clara SDK [Web]
- NIPS 2019 Workshop on Federated Learning for Data Privacy and Confidentiality 1 [Video]
- NIPS 2019 Workshop on Federated Learning for Data Privacy and Confidentiality 2 [Video]
- NIPS 2019 Workshop on Federated Learning for Data Privacy and Confidentiality 3 [Video]
- Federated Learning Approach for Mobile Packet Classification [Paper]
- Federated Learning for Ranking Browser History Suggestions [Paper] [NIPS 2019 Workshop]
- HHHFL: Hierarchical Heterogeneous Horizontal Federated Learning for Electroencephalography [Paper] [NIPS 2019 Workshop]
- Learn Electronic Health Records by Fully Decentralized Federated Learning [Paper] [NIPS 2019 Workshop]
- Patient Clustering Improves Efficiency of Federated Machine Learning to predict mortality and hospital stay time using distributed Electronic Medical Records [Paper] [News]
- MIT CSAI, Harvard Medical School, Tsinghua University
- Federated learning of predictive models from federated Electronic Health Records. [Paper]
- Boston University, Massachusetts General Hospital
- FedHealth: A Federated Transfer Learning Framework for Wearable Healthcare [Paper]
- Microsoft Research Asia
- Multi-Institutional Deep Learning Modeling Without Sharing Patient Data: A Feasibility Study on Brain Tumor Segmentation [Paper]
- Intel
- NVIDIA Clara Federated Learning to Deliver AI to Hospitals While Protecting Patient Data [Blog]
- Nvidia
- What is Federated Learning [Blog]
- Nvidia
- Split learning for health: Distributed deep learning without sharing raw patient data [Paper]
- Two-stage Federated Phenotyping and Patient Representation Learning [Paper] [ACL 2019]
- Federated Tensor Factorization for Computational Phenotyping https://dl.acm.org/doi/10.1145/3097983.3098118 SIGKDD 2017
- FedHealth- A Federated Transfer Learning Framework for Wearable Healthcare [Paper] [ICJAI19 workshop]
- Multi-Institutional Deep Learning Modeling Without Sharing Patient Data- A Feasibility Study on Brain Tumor Segmentation [Paper] [MICCAI'18 Workshop]
- Federated Patient Hashing [Paper] [AAAI'20]
- Federated Learning for Mobile Keyboard Prediction [Paper]
- Applied Federated Learning: Improving Google Keyboard Query Suggestions [Paper]
- Federated Learning Of Out-Of-Vocabulary Words [Paper]
- Federated Learning for Emoji Prediction in a Mobile Keyboard [Paper]
Snips
- Performance Optimization for Federated Person Re-identification via Benchmark Analysis [Paper] [ACMMM20]
- Real-World Image Datasets for Federated Learning [Paper]
- Webank & Extreme Vision
- FedVision- An Online Visual Object Detection Platform Powered by Federated Learning [Paper] [IAAI20]
- Federated Learning for Vision-and-Language Grounding Problems [Paper] [AAAI20]
- Federated Collaborative Filtering for Privacy-Preserving Personalized Recommendation System [Paper]
- Huawei
- Federated Meta-Learning with Fast Convergence and Efficient Communication [Paper]
- Huawei