Speeding up prediction performance of BDT-based models

Citations Over Time

Abstract

The outcome of a machine learning algorithm is a prediction model. Typically, these models are computationally expensive, where improving of the quality the prediction leads to a decrease in the inference speed. However it is not always tradeoff between quality and speed. In this paper we show it is possible to speed up the model by using additional memory without losing significat prediction quality for a novel boosted trees algorithm called CatBoost. The idea is to combine two approaches: training fewer trees and merging trees into a kind of hashmaps called DecisionTensors. The proposed method allows for pareto-optimal reduction of the computational complexity of the decision tree model with regard to the quality of the model. In the considered example the number of lookups was decreased from 5000 to only 6 (speedup factor of 1000) while AUC score of the model was reduced by less than 10−3.

Related Papers

• → Pareto-optimal reinsurance arrangements under general model settings(2017)64 cited
• → Some Sufficiency Conditions for Pareto-Optimal Control(1973)59 cited
• → Some properties of pareto-optimal choices in decision problems(1976)33 cited
• → Multi-objective reinforcement learning method for acquiring all pareto optimal policies simultaneously(2012)6 cited
• → Weak Pareto optimality and the approximate support property(1993)3 cited