Previous work on using context for sentence classification used LSTM and CNN network layers to encode the surrounding context, giving an improvement in classification accuracy (Lee and Dernoncourt, 2016).
the use of CNN and LSTM layers imposes a significant computational cost when training the network, especially if the size of the context is large.