Such a model does not rely on intermediate ASR output and is therefore not subject to error propagation.
the transformation from source speech inputs to target text outputs is much more complex than that of an ASR or MT system taken individually, which may cause the model to require more data to perform well.