To further investigate what the universal time-decay attention learns, we inspect the learned weights w i and find that the convex attention function almost dominates the whole function.
our model automatically learns that the convex timedecay attention is more suitable for modeling contexts from the dialogue data than the other two types.