Caching reduces the baseline's decoding speed from 210 seconds to 128.5; CMLMs do not use cached decoding.
in terms of speed, each mask-predict iteration is virtually equivalent to a refinement iteration.