These studies showed that performance improves when low-layer RNN representations are trained in easy tasks and high-layer RNN are leveraged for difficult tasks.
we construct a network that hierarchically overlaps a task-specific RNN on a task-independent RNN.