• Apply softmax function to normalize the attention coefficients, • Use the normalized attention coefficients to compute a weighted sum of the comment embedding vectors, as the readout in this computation step, • The LSTM takes q l and r l as input and computes the next hidden state, q l+1 , Note that swapping any two comment embedding vectors, m c i t and m c j t , will not affect the query vector q l as well as the attention readout r l .
we need a procedure to approximate the predicted optimal action from a combinatorial action space.