Backprogapation is a subtopic of [neural networks](../neural-networks/index.md).
**Purpose:** It is an algorithm/process with the aim of minimizing the cost function (in other words, the error) of parameters in a neural network.
**Method:** This is done by calculating the gradients of each node in the network. These gradients measure the "error" each node contributes to the output layer, so in training a neural network, these gradients are minimized.
Note: Backpropagation, and machine learning in general, requires significant familiarity with linear algebra and matrix manipulation. Coursework or reading on this topic is highly recommended before trying to understand the contents of this article.
In this case, for matrix M, M' will denote the transpose of matrix M
1. Assign all entries of the Delta(i), for i from 1 to L, zero.
2. For each training example t from 1 to m, perform the following:
- perform forward propagation on each example to compute a(l) and z(l) for each layer
- compute d(L) = a(L) - y(t)
- compute d(l) = (Theta(l)' • d(l+1)) • g(z(l)) for l from L-1 to 1
- increment Delta(l) by delta(l+1) • a(l)'
3. Plug the Delta matricies into our partial derivative matricies
D(l) = 1\m(Delta(l) + lambda • Theta(l)); if l≠0
D(l) = 1\m • Delta(l); if l=0
Of course, just seeing this article looks hugely complicated and should really only be understood in the greater contexts of neural networks and machine learning. Please look at the arrached references for a better understanding of the topic as a whole.
* [Andrew Ng's ML Course](https://www.coursera.org/learn/machine-learning/)
If you'd like to learn how to implement a full-blown single (hidden) layer neural network in python, whilst learning more about the math behind the algorithms used, you can register for [Andrew Ng's Deep Learning Specialization] (https://www.coursera.org/specializations/deep-learning)
