Fixed-length context windows S running over the corpus are used in word embedding methods as in C-BOW (Mikolov et al., 2013b,a) and GloVe (Pennington et al., 2014).
here we have k = |V| and each cost function f S : R k → R only depends on a single row of its input, describing the observed target word for the given fixed-length context S. for sentence embeddings which are the focus of our paper here, S will be entire sentences or documents (therefore variable length).