On the other hand, FT-Lr-AttRNN has difficulty to learn new knowledge since it cannot jump out of local optimum due to a small learning rate.
the performance of FT-Lr-AttRNN is even lower than FT-AttRNN most of the time.