To solve the above MDP problem, we need a model which has policy to choose actions at each state.
different from normal RL problems, we apply a policy network considering the search history over G. Formally, after embedding all entities and relations in G as e ∈ R d and r ∈ R d respectively, each action a t = (r t+1 ,ê t+1 ) ∈ A t is represented as a t = [r t+1 ;ê t+1 ].