To be specific, TARS feeds the shared features s k into individual P-Encoder for each task, to encode task-dependent features as follows: Simultaneously, a gating network decides on output weights for each expert (i.e., individual P-Encoder).
the gating network G, parameterized by θ g , is used to map the shared representation of current task into the correct expert, and each expert is thus learning task-dependent features for that task, estimating task label of s k : where W g and b g is a trainable weight matrix and a bias, respectively.