As we mentioned in Section 3.1, we use an LSTM to store the historical path information.
besides, for every state s t , we also add an additional action (r LOOP , e t ), where LOOP is a manually added selfloop relation.