To encode the surface form of s, we can use recurrent neural networks (Kiperwasser and Goldberg, 2016b).
we apply a bidirectional long short-term memory network (biLSTM, (Hochreiter and Schmidhuber, 1997)  where x i is the input of a LSTM cell which includes a randomly initialized word embedding e(w i ), a pre-trained word embedding e ′ (w i ) from Glove (Pennington et al., 2014) and a trainable embedding of w i 's part-of-speech tag e(pos i ), Then, a context-dependent node representation of word i is the concatenation of the two hidden vectors, With the node representations, we can define the score function σ using a multi-layer perceptron σ(i, j) = MLP(c i ⊕ c j ) (Pei et al., 2015), or using a normalized bilinear function (A, b 1 , b 2 are parameters), x 1 x 2 x 3 x 4 Figure 1: The GNN architecture.