Our discriminative training procedure includes updating both λ and θ, and we follow (He and Deng, 2012) to optimize them in an alternate manner.
when we optimize θ via EBW, we keep λ fixed and when we optimize λ, we keep λ fixed.