Overcome Support Vector Machine Diagnosis Overfitting

Citations Over TimeTop 14% of 2014 papers

Abstract

Support vector machines (SVMs) are widely employed in molecular diagnosis of disease for their efficiency and robustness. However, there is no previous research to analyze their overfitting in high-dimensional omics data based disease diagnosis, which is essential to avoid deceptive diagnostic results and enhance clinical decision making. In this work, we comprehensively investigate this problem from both theoretical and practical standpoints to unveil the special characteristics of SVM overfitting. We found that disease diagnosis under an SVM classifier would inevitably encounter overfitting under a Gaussian kernel because of the large data variations generated from high-throughput profiling technologies. Furthermore, we propose a novel sparse-coding kernel approach to overcome SVM overfitting in disease diagnosis. Unlike traditional ad-hoc parametric tuning approaches, it not only robustly conquers the overfitting problem, but also achieves good diagnostic accuracy. To our knowledge, it is the first rigorous method proposed to overcome SVM overfitting. Finally, we propose a novel biomarker discovery algorithm: Gene-Switch-Marker (GSM) to capture meaningful biomarkers by taking advantage of SVM overfitting on single genes.

Related Papers

• → An Overview of Overfitting and its Solutions(2019)2,168 cited
• → Lazy Overfitting Control(2013)10 cited
• → Machine Learning Students Overfit to Overfitting(2022)5 cited
• → Overfitting: Causes and Solutions (Seminar Slides)(2020)3 cited
• → Benign Overfitting in Classification: Provably Counter Label Noise with Larger Models(2022)3 cited