The reason that existing models lose important information in the context is that they first represent the whole context as a vector and then match the context vector with a response vector.
responses in these models connect with the context until the final step in matching.