Three sentiment knowledge prediction objectives are jointly optimized during pre-training so as to embed sentiment words, polarity, aspect-sentiment pairs into the representation.
the pair prediction is converted into multi-label classification to capture the dependency between aspect and sentiment.