However, current end-to-end approaches only consider user semantic inputs in learning and under-utilize other user information.
we propose to include user sentiment obtained through multimodal information (acoustic, dialogic and textual), in the end-to-end learning framework to make systems more user-adaptive and effective.