The results show that the embedding layers and linear weights in transformer layers are sensitive on CoLA and MRPC datasets.
we set the compression ratios of corresponding weights zero to ensure the final accuracy.