We consider that each hidden vector of TDLSTM encodes the semantics of word sequence until the current position.
the model of TDLSTM+ATT actually selects such mixed semantics of word sequence, which is weird and not an intuitive way to selectively focus on parts of contexts.