Asymmetric Sigmoidal Activation Function for Feed-Forward Artificial Neural Networks

Abstract

Artificial neural networks of the feed – forward kind, are an established technique under the supervised learning paradigm for the solution of learning tasks. The mathematical result that allows one to assert the usefulness of this technique is that these networks can approximate any continuous function to the desired degree. The requirement imposed on these networks is to have non-linear functions of a specific kind at the hidden nodes of the network. In general, sigmoidal non-linearities, called activation functions, are generally used. In this paper we propose an asymmetric activation function. The networks using the proposed activation function are compared against those using the generally used logistic and the hyperbolic tangent activation function for the solution of 12 function approximation problems. The results obtained allow us to infer that the proposed activation function, in general, reaches deeper minima of the error measures and has better generalization error values.

Related Papers

• → Towards activation function search for long short-term model network: A differential evolution based approach(2020)68 cited
• → Combine Relu with Tanh(2020)12 cited
• → FPGA Implementation and Comparison of Sigmoid and Hyperbolic Tangent Activation Functions in an Artificial Neural Network(2022)12 cited
• → A novel Scaled-Gamma-Tanh (SGT) activation function in 3D CNN applied for MRI classification(2022)1 cited
• → TaLU: A Hybrid Activation Function Combining Tanh and Rectified Linear Unit to Enhance Neural Networks(2023)1 cited