Weighted Context Vector Sum (WCVS): another approach is to apply a weight vector on the GD context vector.
the information from GD can be controlled before compacting, as in Equation 12: Figure 4: RNN LM with gated adaptation, where indicates the gates mechanisms described in Section 3.3.