We further use several layers of transformer encoders (Vaswani et al., 2017) to learn the correlation between different feature fields.
the matrix product between query qW Q i and key HW K i after softmax normalization is an attention weight that indicates important words among the projected value vectors HW V i .