In order to alleviate this shortcoming, we propose to use multitask learning (MTL) and exploit transcribed speech within the end-to-end visuallygrounded setting, and thus combine some features of both the pipeline and end-to-end approaches.
within Natural Language Processing (NLP), Luong et al.