For instance, during the computation of the probabilities for the input copy mechanism in Equation 1, the affinity function f between the current context q and a given input u k is generally implemented by projecting u and q into a single vector followed by a non-linearity, which is projected into a single affinity value.
for each possible input u, 3 matrix multiplications must be performed.