Modify Leave-One-Out Cross Validation by Moving Validation Samples around Random Normal Distributions: Move-One-Away Cross Validation

Citations Over Time

Abstract

The leave-one-out cross validation (LOO-CV), which is a model-independent evaluate method, cannot always select the best of several models when the sample size is small. We modify the LOO-CV method by moving a validation point around random normal distributions—rather than leaving it out—naming it the move-one-away cross validation (MOA-CV), which is a model-dependent method. The key point of this method is to improve the accuracy rate of model selection that is unreliable in LOO-CV without enough samples. Errors from LOO-CV and MOA-CV, i.e., LOO-CVerror and MOA-CVerror, respectively, are employed to select the best one of four typical surrogate models through four standard mathematical functions and one engineering problem. The coefficient of determination (R-square, R2) is used to be a calibration of MOA-CVerror and LOO-CVerror. Results show that: (i) in terms of selecting the best models, MOA-CV and LOO-CV become better as sample size increases; (ii) MOA-CV has a better performance in selecting best models than LOO-CV; (iii) in the engineering problem, both the MOA-CV and LOO-CV can choose the worst models, and in most cases, MOA-CV has a higher probability to select the best model than LOO-CV.

Related Papers

• → kNNDM: k-fold Nearest Neighbour Distance Matching Cross-Validation for map accuracy estimation(2023)11 cited
• → Modify Leave-One-Out Cross Validation by Moving Validation Samples around Random Normal Distributions: Move-One-Away Cross Validation(2020)8 cited
• → A comparative QSPR study on aqueous solubility of polycyclic aromatic hydrocarbons by GA-SVM, GA-RBFNN and GA-PLS(2011)5 cited
• Structural Characterization for Flavonoids Through MEDV and Xanthine-oxidase Inhibition Activity Prediction by QSAR(2006)
• Application of Cross Validation in Choice of Data Modeling(2011)