A straightforward approach is to learn attention signals by self-attention and then apply these signals into inputs before feeding them into LSTM.
these attention signals are applied to all gates of a LSTM cell.