THE TRANSFORMER SERIES

Transformer’s Positional Encoding

How Does It Know Word Positions Without Recurrence?

In 2017, Vaswani et al. published a paper titled “Attention Is All You Need” for the NeurIPS conference.

They introduced the original transformer architecture for machine translation, performing better and faster than RNN…