In spite of their success, the input representation only involves static order dependencies based on discrete numerical information.
any word in the entire vocabulary has the same PE on the same position index.