At test time, we want to have different combinations of semantic and syntactic inputs, which naturally suggests separate parameterizations for q φ (y|x) and q φ (z|x).
q φ (y|x) is parameterized by a word averaging encoder followed by a three-layer feedforward neural network since it has been observed that word averaging encoders perform surprisingly well for semantic tasks (Wieting et al., 2016).