Practical Differentially Private Hyperparameter Tuning with Subsampling

Citations Over Time

Abstract

Tuning the hyperparameters of differentially private (DP) machine learning (ML) algorithms often requires use of sensitive data and this may leak private information via hyperparameter values. Recently, Papernot and Steinke (2022) proposed a certain class of DP hyperparameter tuning algorithms, where the number of random search samples is randomized itself. Commonly, these algorithms still considerably increase the DP privacy parameter $\varepsilon$ over non-tuned DP ML model training and can be computationally heavy as evaluating each hyperparameter candidate requires a new training run. We focus on lowering both the DP bounds and the computational cost of these methods by using only a random subset of the sensitive data for the hyperparameter tuning and by extrapolating the optimal values to a larger dataset. We provide a Rényi differential privacy analysis for the proposed method and experimentally show that it consistently leads to better privacy-utility trade-off than the baseline method by Papernot and Steinke.

Related Papers

• → Grid Search Based Hyperparameter Optimization for Machine Learning Based Non-Intrusive Load Monitoring(2023)6 cited
• → Deep Neural Network with Hyperparameter Tuning for Detection of Heart Disease(2021)23 cited
• → Hyperparameter Tuning on Classification Algorithm with Grid Search(2022)16 cited
• → Novel Suboptimal approaches for Hyperparameter Tuning of Deep Neural Network [under the shelf of Optical Communication](2019)3 cited
• → Hyperparameter Tuning of Dense Neural Network for ECG Signal Classification(2022)