Large-Scale Pre-Trained Language Models

RoBERTa

How did RoBERTa outperform XLNet with no architectural changes to the original BERT?

In 2018, Jacob Devlin et al. (Google AI Language) released BERT that achieved state-of-the-art results on nine NLP tasks, inspiring other researchers to come up with further improvements. Soon, newer models superseded…