12 can be written as: Since p is a value between zero and one, sum components that are far away in the past are multiplied by a very low value and are effectively removed from the summation.
even though the network is able to learn long-term dependencies, it is not capable of exploiting them during test phase.