They used sequence-to-sequence models to transcribe Spanish speech and translate it in English, by jointly training the two tasks in a multitask scenario where the decoders share the encoder.
to our work, they use a large corpus for training the model on roughly 163 hours of data, using the Spanish Fisher and CALL-HOME conversational speech corpora.