The attention is computed as with hard attention, as α i j = softmax(u i j ).
instead of making a hard selection, a weighted average over the encoder vectors is computed as This vector is used instead of h a j for prediction and feeding to the next timestep.