The possible functionality of lower layers might be constructing gradually better contextual representation of the word at each position as suggested in recent contextualized embedding works (Peters et al., 2018;Devlin et al., 2018;Radford et al., 2019 the AERs become better while more unambiguous representations of the corresponding word are formed.
one possible reason is that using zero embedding in inference is exactly the same way as dropout in training, making the trained parameters perform well in inference.