Ideally, the cumulative attention should satisfy The standard choice for attention mechanisms is the softmax transformation, α n = softmax(z n ).
the softmax does not satisfy either of the properties above.