We implement the hierarchical tree decoder in a batch mode, so that it can fully utilize GPUs.
as shown in Algorithm 1, every time we pop multiple nonterminals from the queue and decode these nonterminals in one batch.