The context used for the attention component is simply the hidden state from the previous timestep.
since we use a bi-LSTM, the model essentially has two RNNs, and accordingly we have two context vectors, and associated attentions.