Using the auxiliary objectives overcame the difference, resulting in a -1.6% average relative WER change.
pretraining with GEO+phn+adv yielded an average relative delta of -7.4% over the monolingual model.