The gap between GPU and CPU performance is large enough that CPU Viterbi search was a bottleneck, even though it requires asymptotically less work (O(Gs 2 ) worst case, O(Gs) typical) vs O(Gs 3 ) to compute the CKY scores.
we wrote a non-recusive GPU-based Viterbi search.