The only difference is that in cases where the next unigram or bigram is a pivot, instead of predicting the actual pivot identity, PBLM should predict PosPiv or NegPiv according to the pivot's class.
the representation learned by the first PBLM model is only sensitive to whether a pivot is positive or negative and not to the actual pivot identity.