For example, as you go from bottom to top layers, information about the past in left-to-right language models gets vanished and predictions about the future get formed.
for MLM, representations initially acquire information about the context around the token, partially forgetting the token identity and producing a more generalized token representation.