For both of the tasks, we use the weight-tying technique (Inan et al., 2017;Press and Wolf, 2017) by using W p as the word embedding matrix.
e(y t ) is the y t -th row vector in W p , and the technique has shown to be effective in machine translation (Hashimoto and Tsuruoka, 2017) and text summarization (Paulus et al., 2018).