In fact, the short-list approach in (Schwenk, 2007) and the adaptive importance sampling in (Bengio and Senecal, 2008) have exactly this intuition.
in order to train the classifiers, we need to compute all V output nodes and propagate the errors back.