GPT (2018)

Generative Pre-Trained Transformer

#GPT #Transformer

In 2018, OpenAI released the first version of GPT (Generative Pre-Trained Transformer) for generating texts as if humans wrote. The architecture of GPT is based on the original transformer’s decoder.

They trained GPT in two stages:

1. Unsupervised Pre-training pre-trains GPT on unlabeled text…