gaspersic/freeCodeCamp
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Create initial RNN description, added more information link (#32297)

Browse Source 
* Create initial RNN description, added more information link

Included a generic RNN diagram of a stacked and unfolded RNN. Motivated material with example sentence.

* Add additional link

* Add helpful link on LSTMs
This commit is contained in:
Jason Yum
2019-06-27 18:32:24 -04:00
committed by Randell Dawson
parent 625bb4416d
commit 039b53dfd8
1 changed files with 10 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files

14
guide/english/machine-learning/neural-networks/recurrent-neural-networks/index.md
View File

@ -5,11 +5,17 @@ title: Recurrent Neural Networks
Recurrent Neural Networks (RNN) are a category of Neural Networks which are mainly used for data which are sequential in nature. In Recurrent Neural Networks, the output of the previous time is fed as an input to the current time. RNN's have been extensively used for Natural Language Processing, Time Series Data prediction and other tasks which involve sequential data. Recurrent Neural Networks (RNN) are a category of Neural Networks which are mainly used for data which are sequential in nature. In Recurrent Neural Networks, the output of the previous time is fed as an input to the current time. RNN's have been extensively used for Natural Language Processing, Time Series Data prediction and other tasks which involve sequential data.
RNNs make use of the contextual information embedded in sequences. This is thematically similar to CNNs and how they are advantaged in learning images (by picking up contextual information of nearby pixels) over a normal feedforward neural network that might evaluate each pigment individually. To make this more concrete, if you had a sentence "the fox jumps over the ..." and were asked to predict the next word, the preceding five words strongly suggest that the next word should be a noun instead of a verb. That's the power of context!
![Unfolded RNN](https://upload.wikimedia.org/wikipedia/commons/b/b5/Recurrent_neural_network_unfold.svg)
In this diagram, x_t is the input vector at "time" step t. U_x is the learned weighting vector for input vecotr x_t. Similarly, O_t is the output vector at "time" step t, and W_o is its learned weighting vector. The h_t denotes the hidden state of the neuron, which is a function of its preceding hidden state (h_{t-1}) and the x_t. Generically, this means that h_t = function(h_{t-1}, x_t).
### Types of Recurrent Neural Networks ### Types of Recurrent Neural Networks
* Long Short Term Memory (LSTM) * Long Short Term Memory (LSTM)
* Gated Recurrent Unit (GRU) * Gated Recurrent Unit (GRU)
## Suggested Links #### More Information:
* Stanford CS231 lesson 10 video https://www.youtube.com/watch?v=6niqTuYFZLQ - [Princeton Zhirong Wu, 2015 Slides](http://3dvision.princeton.edu/courses/COS598/2015sp/slides/RNN/RNN.pdf)
* Deep Learning Book https://www.youtube.com/watch?v=6niqTuYFZLQ - [Princeton COS 495, Instructor Yingyu Liang](https://www.cs.princeton.edu/courses/archive/spring16/cos495/slides/DL_lecture9_RNN.pdf)
* Colah's blog http://colah.github.io/posts/2015-08-Understanding-LSTMs/ - [Edward Chen on LSTM](http://blog.echen.me/2017/05/30/exploring-lstms/)