Recurrent Neural Networks

Understanding RNN, Deeper RNN, Bidirectional RNN, and RNN Encoder-Decoder (Sequence-to-sequence, aka seq2seq)

RNNs (recurrent neural networks) handle sequential data where the order of sequence matters. They deal with time-series data, languages (sequence of words), and so forth.

Explanations of RNNs usually include many boxes and arrows, which may be confusing. There are also many diagrams because RNNs come in many different shapes and forms. So, the purpose of this article is to describe the core concepts in RNNs and show various diagrams step-by-step for better understanding.

We discuss the following topics:

• Sequential Data
• Simple RNN
• Deeper RNN
• Bidirectional RNN
• RNN Encoder-Decoder

Sequential Data

Before discussing sequential data, let’s first discuss non-sequential data and then compare it against sequential data to understand the crucial features.

In an image classification task, we typically deal with images randomly sampled from a dataset.