With the two auxiliary tasks, we can learn a neural network model in a standard way to produce sentence embeddings that work well for the auxiliary tasks.
we still use Θ to denote the parameters of the neural network that produces the sentence embeddings (and f Θ the corresponding transformation function), and we use β + and β − to denote the parameters of two softmax classifiers for the two auxiliary tasks, respectively.