GPT-2: Too Dangerous To Release (2019)
GPT-2 is a direct scale-up of GPT-1, with more parameters and trained on more data. However, it was deemed to be too dangerous to release by OpenAI:
Due to our concerns about malicious applications of the technology, we are not releasing the trained model. As an experiment in responsible disclosure, we are instead releasing a much smaller model for researchers to experiment with, as well as a technical paper.
Source: OpenAI Blog — Better Language Models and Their Implications — Feburary 14, 2019
GPT-1 was released to the public without such serious concerns. Therefore, the above claim made the public wonder how powerful GPT-2 must be in generating texts that look like humans wrote.
Moreover, what’s the difference between GPT-1 and GPT-2?
The Difference: GPT-1 vs. GPT-2
In the GPT-1 paper, they experimented with the model on zero-shot task transfer in that they used the pre-trained model with heuristic solutions to perform specific tasks. The experiment’s success suggests that without supervised fine-tuning, the language model already contains information required to perform specific tasks. All that knowledge is stored in…
