When conditioned on the the full input sequence and the entire prediction history, both locally normalized and globally normalized conditional models should have same expressive power under a highcapacity neural parametrization in theory, as they can both model same set of distributions over all finite length output sequences (Smith and Johnson, 2007).
locally normalized models are constrained in how they respond to search errors during training since the scores at each decoding step must sum to one.