GPT (2018)
In 2018, OpenAI released the first version of GPT (Generative Pre-Trained Transformer) for generating texts as if humans wrote. The architecture of GPT is based on the original transformer’s decoder.
They trained GPT in two stages:
- Unsupervised Pre-training pre-trains GPT on unlabeled text…