Motivated by the observation that representations from higher layers have better expressibility than lower layers, as claimed in §2.2, we instead employ a coarse-to-fine strategy to select one task for each layer: finer-grained tasks for higher layers while coarser-grained task for lower layers.
suppose 1 ≤ s(l) ≤ K is the selected index regarding task Y s(l) for the l th layer, then it subjects to s(l) < s(l + 1) for each l. In addition, to encourage the diversity among the selected L tasks, we require s(l + 1) − s(l) to be large enough for all l. Formally, the loss of the hierarchical regularization (HR) method is: where is similar to Eq.