Description

Activation functions are mathematical equations that determine the output of a neural network. They are attached to each neuron in the network and determine whether it should be activated or not, based on the relevance of the input. Activation functions also help normalize the output of each neuron to a range between 1 and 0 or between -1 and 1.

Examples

• 📈 ReLU (Rectified Linear Unit)
• 🔄 Sigmoid
• 〰️ Tanh (Hyperbolic Tangent)

Applications