Generative Models for Effective ML on Private, Decentralized Datasets

Citations Over Time

Abstract

To improve real-world applications of machine learning, experienced modelers develop intuition about their datasets, their models, and how the two interact. Manual inspection of raw data - of representative samples, of outliers, of misclassifications - is an essential tool in a) identifying and fixing problems in the data, b) generating new modeling hypotheses, and c) assigning or refining human-provided labels. However, manual data inspection is problematic for privacy sensitive datasets, such as those representing the behavior of real-world individuals. Furthermore, manual data inspection is impossible in the increasingly important setting of federated learning, where raw examples are stored at the edge and the modeler may only access aggregated outputs such as metrics or model parameters. This paper demonstrates that generative models - trained using federated methods and with formal differential privacy guarantees - can be used effectively to debug many commonly occurring data issues even when the data cannot be directly inspected. We explore these methods in applications to text with differentially private federated RNNs and to images using a novel algorithm for differentially private federated GANs.

Related Papers

• → Advances and Open Problems in Federated Learning(2020)4,363 cited
• → Practical Secure Aggregation for Privacy-Preserving Machine Learning(2017)3,273 cited
• → Communication-Efficient Learning of Deep Networks from Decentralized Data(2016)5,172 cited
• → Federated Learning: Strategies for Improving Communication Efficiency(2016)3,046 cited
• → Federated Learning for Mobile Keyboard Prediction(2018)1,105 cited