Backpropagation, short for 'backward propagation of errors', is the central algorithm for training artificial neural networks. It efficiently computes the gradient of the loss function with respect to the weights of the network. By applying the chain rule of calculus, it calculates how much each weight contributed to the error, allowing the optimization algorithm (like Gradient Descent) to adjust the weights to minimize the error.
The mathematical foundations of backpropagation were derived by several researchers independently, including Kelley (1960) and Bryson (1961). However, its application to neural networks was popularized by Rumelhart, Hinton, and Williams in their influential 1986 paper 'Learning representations by back-propagating errors'.
Backpropagation is the workhorse of modern AI. It enables the training of deep multi-layer networks that power everything from image recognition to large language models. Without backpropagation, the deep learning revolution would not have been possible.