Given instruction x, the language encoder proceeds in two steps, end-to-end, by considering a function decomposition f θ E = f θx→e • f θe→z :  into a single joint feature Previous methods in VLN learn e either from pretrained word embeddings (Pennington et al., 2014) which do not take into account word context, or from scratch.
their representations do not capture contextual information within each instruction.