Recurrent neural networks(RNNs) such as gated recurrent units (GRUs) have the ability to keep the information from the beginning to the end to some extent.
the cue word added on the first step of the neural networks can still influence the generation of the later steps.