Active Surrogate Estimators: An Active Learning Approach to Label-Efficient Model Evaluation

Citations Over Time

Abstract

We propose Active Surrogate Estimators (ASEs), a new method for label-efficient model evaluation. Evaluating model performance is a challenging and important problem when labels are expensive. ASEs address this active testing problem using a surrogate-based estimation approach that interpolates the errors of points with unknown labels, rather than forming a Monte Carlo estimator. ASEs actively learn the underlying surrogate, and we propose a novel acquisition strategy, XWED, that tailors this learning to the final estimation task. We find that ASEs offer greater label-efficiency than the current state-of-the-art when applied to challenging model evaluation problems for deep neural networks.

Related Papers

• → Conservative strategy-based ensemble surrogate model for optimal groundwater remediation design at DNAPLs-contaminated sites(2017)32 cited
• → Estimation of the general population parameter in single- and two-phase sampling(2023)1 cited
• → On Modification of Some Ratio Estimators using Parameters of Auxiliary Variable for the Estimation of the Population Mean(2023)1 cited
• → A new class of ratio type estimators in single- and two-phase sampling(2022)1 cited
• → Estimators for the Product of Two Population Means Using Auxiliary Attribute in the Presence of Non-response(2022)