We believe that while textual summarization, question generation, and entailment generation have different training data distributions and low-level representations, they can still benefit from sharing their models' high-level components (e.g., those that capture the skills of saliency and inference).
we keep the lower-level layer (i.e., first layer closer to input words) of the 2layer encoder of all three tasks unshared, while we share the higher layer (second layer in our model as shown in Fig.