However, when t > N , when the memory becomes full, it selects an existing memory entry to delete.
it outputs an action based on π(i|S (t) ), which denotes the selection of i th memory entry to delete.