Activation Functions

Mathematical equations that determine the output of a neural network.

Description

Activation functions are mathematical equations that determine the output of a neural network. They are attached to each neuron in the network and determine whether it should be activated or not, based on the relevance of the input. Activation functions also help normalize the output of each neuron to a range between 1 and 0 or between -1 and 1.

Examples

  • πŸ“ˆ ReLU (Rectified Linear Unit)
  • πŸ”„ Sigmoid
  • 〰️ Tanh (Hyperbolic Tangent)

Applications

βž— Introducing non-linearity in neural networks
πŸ—ΊοΈ Feature mapping
🏷️ Classification tasks

Related Terms