To obtain this result, the transfer learning capabilities of BERT were not enough and further engineering effort was required.
a conservativeness factor was added during beam decoding to constrain the changes the APE system can make to the mt output.