Specifically, the gating network G, parameterized by θ g , is used to map the shared representation of current task into the correct expert, and each expert is thus learning task-dependent features for that task, estimating task label of s k : where W g and b g is a trainable weight matrix and a bias, respectively.
based on above, the final softprivate representation p(s k ) is a mixture of all expert outputs with respect to s k as the following: Soft-Shared Module In order to learn taskinvariant features, inspired by (Liu et al., 2017), TARS adopts an adversarial network, which contains a feature extractor and a task discriminator D. The basic idea is to learn features that cannot be distinguished by D. D aims to discriminate which task the feature comes from, while the feature extractor (e.g., S-Encoder) tries to fool D so that it cannot identify the task of the feature and is hence task-invariant.