Srinivas and Babu (2015) propose to add the outgoing weights of j to the weights of a similar neuron i to compensate for the removal of j.
we have found that this approach does not work well on NMT networks.