We induce lexical information from our multimodal model into the acoustic network during training while still providing a lexical-independent acoustic model for testing or deployment.
our acoustic model learns to capture semantic and contextual information without relying on explicit lexical inputs such as ASR or transcripts.