Task   Table 2 shows that the INI and RND 12 methods perform similarly in the Original test set, arguably be-cause a large part of the learning takes place in the parameters of the layers subsequent to the embedding layer.
in the next section we show that this is no longer the case when unseen words are present.