While overlarge embedding sizes hurt accuracy because of overfitting issues, smaller sizes are not Test Set tst2011 tst2012 tst2013 tst2014 total RNN (Gulcehre et al., 2015) 18  preferable because of insufficient representation power.
our dense models show that with better model design, the embedding information can be well concentrated on fewer dimensions, e.g., 64.