Residual Block (Skip Connection)

What is Residual Block (Skip Connection)?

A component that adds the input of a layer to its output (y = f(x) + x), creating a 'highway' for gradients to flow. This solved the vanishing gradient problem for deep networks.

Where did the term "Residual Block (Skip Connection)" come from?

ResNet (He et al., 2015).

How is "Residual Block (Skip Connection)" used today?

Enabled networks to go from 20 layers to 100+.

Related Terms

• Deep Learning
• Backpropagation