Computational Complexity of Gradient Descent Algorithm

Citations Over Time

Abstract

Information is mounting exponentially, and the world is moving to hunt knowledge with the help of Big Data. The labelled data is used for automated learning and data analysis which is termed as Machine Learning. Linear Regression is a statistical method for predictive analysis. Gradient Descent is the process which uses cost function on gradients for minimizing the complexity in computing mean square error. This work presents an insight into the different types of Gradient descent algorithms namely, Batch Gradient Descent, Stochastic Gradient Descent and Mini-Batch Gradient Descent, which are implemented on a Linear regression dataset, and hence determine the computational complexity and other factors like learning rate, batch size and number of iterations which affect the efficiency of the algorithm.

Related Papers

• → Stochastic Gradient Descent(2015)47 cited
• → Training a Two-Layer ReLU Network Analytically(2023)7 cited
• → Accelerating Extreme Search Based on Natural Gradient Descent with Beta Distribution(2021)4 cited
• → Computational Complexity of Gradient Descent Algorithm(2021)3 cited
• → Training Neural Networks Using Predictor-Corrector Gradient Descent(2018)