The various model choices for the S module include Temporal Convolutional neural networks(Kim, 2014) (CNN) and sequential models like Long Short-Term Memory (Hochreiter and Schmidhuber, 1997) (LSTM) and Gated Recurrent Unit   Let v i ∈ R d be the d-dimensional word embedding of the i th word of a sentence of length n. We show 3 different models to produce a sentence embedding from the word embeddings.
then, the S module can be represented mathematically as a composition function f that takes a sequences of n word embeddings as input to produce a sentence embedding s. where the choice of the composition function are standard generic neural networks like LSt M, GRU, CN N .