Activation Functions

What is Activation Functions?

Mathematical functions (like ReLU, Sigmoid, Softmax) that introduce non-linearity into the network, allowing it to learn complex patterns.

Where did the term "Activation Functions" come from?

Essential for deep learning to work.

How is "Activation Functions" used today?

ReLU is the default for hidden layers; Softmax for output.

Related Terms

• Neural Network
• ReLU (Rectified Linear Unit)
• sigmoid