Differential Privacy Machine Learning Based on Attention Residual Networks

Citations Over Time

Abstract

To address the problems that the differential privacy stochastic gradient descent algorithm DP-SGD does not apply to deep network models with more parameters and noise addition affects model performance, a differential privacy machine learning model based on attention residual networks is proposed, which solves the problem of differential privacy training on larger models by embedding the attention module, replacing the batch normalization with group normalization, and replacing the activation function with more efficient Experimental results on MNIST and CIFAR-10 datasets show that the classification accuracy of the model is improved by 3.99% and 2.43%, respectively, compared with the original model before the improvement, with the same privacy budget $\varepsilon$.

Related Papers

• → Direct Estimation of Weights and Efficient Training of Deep Neural Networks without SGD(2019)2 cited
• → Training on test data: Removing near duplicates in Fashion-MNIST(2019)3 cited
• → Weighted Aggregating Stochastic Gradient Descent for Parallel Deep Learning(2020)
• → Dynamic Momentum for Deep Learning with Differential Privacy(2023)