Alternating Decision Forests

Citations Over TimeTop 10% of 2013 papers

Abstract

This paper introduces a novel classification method termed Alternating Decision Forests (ADFs), which formulates the training of Random Forests explicitly as a global loss minimization problem. During training, the losses are minimized via keeping an adaptive weight distribution over the training samples, similar to Boosting methods. In order to keep the method as flexible and general as possible, we adopt the principle of employing gradient descent in function space, which allows to minimize arbitrary losses. Contrary to Boosted Trees, in our method the loss minimization is an inherent part of the tree growing process, thus allowing to keep the benefits of common Random Forests, such as, parallel processing. We derive the new classifier and give a discussion and evaluation on standard machine learning data sets. Furthermore, we show how ADFs can be easily integrated into an object detection application. Compared to both, standard Random Forests and Boosted Trees, ADFs give better performance in our experiments, while yielding more compact models in terms of tree depth.

Related Papers

• → GBMVis: Visual Analytics for Interpreting Gradient Boosting Machine(2021)6 cited
• → Sales Prediction of Big Mart based on Linear Regression, Random Forest, and Gradient Boosting(2023)1 cited
• → On Application of Machine Learning Models for Prediction of Failures in Production Lines(2021)1 cited
• → Gradient Boosting Machine: A Survey(2019)25 cited
• → Proximal boosting: aggregating weak learners to minimize non-differentiable losses(2018)