RNN Forward Approach

An approach to model a Recurrent Neural Network based on a State Space Model where the current state is being used to calculate the next state. So it takes at least one more time step to see the effect in the observations.

We can translate this continous model to a Neural Network structure via universal approximation:

Architecture

• Allows to include $y_t$ as input
• Shift along the time labeling