More importantly, we observe appealing extraction behaviors of SemanticCaps on utterances of emerging intents as well, even if they are not trained to perform semantic extraction on utterances of emerging intents.
for example, in the intent of PlayMusic one self-attention head always focuses on the "play" action while another attention focuses on musician names.