As a matter of fact, even without any contexts, the lexical translation table in HMM already contains O(|V e | * |V f |) parameters, where |V e | and V f denote source and target vocabulary sizes.
our model does not maintain a separate translation score parameters for every source-target word pair, but computes t lex through a multi-layer network, which naturally handles contexts on both sides without explosive growth of number of parameters.