Large-Scale Pre-Trained Language Models

BERT

How and Why Does It Use The Transformer Architecture?

BERT stands for Bidirectional Encoder Representations from Transformers. As the name suggests, it generates representations using an encoder from Vaswani et al.’s Transformer architecture. However, there are notable differences between BERT and the original Transformer…