For example, Stahlberg and Byrne (2017) proposed a method of unfolding an ensemble of multiple translation models into a single large model once and shrinking it down to a small one.
all methods require extra implementation on a deep-learning framework, and it is not easy to apply them to other models.