We further reduce the applied layers to low-level two (Row 5), the above phenomena still holds.
future work includes combining our information aggregation techniques together with other advanced information extraction models for multihead attention .