Adding a hard constraint on the output vocabulary would make this equivalent to a simple form of multitask learning (Caruana, 1997;Collobert and Weston, 2008).
allowing the model to use tokens from either language at any time is simpler and results in better modeling of mixedlanguage data, which is more common in non-English environments.