These have been around for decades. They're still not very good.
GPT2, though. That really is good. And it's so simple. They just trained a big neural net to predict the next word given some text. It's like autocomplete. 8 million web pages in the training set. You give it a sentence to get it started, and you get some good paragraphs.[1]
Somebody should train that on Stack Overflow and use it for tech support.
GPT2, though. That really is good. And it's so simple. They just trained a big neural net to predict the next word given some text. It's like autocomplete. 8 million web pages in the training set. You give it a sentence to get it started, and you get some good paragraphs.[1]
Somebody should train that on Stack Overflow and use it for tech support.
[1] https://blog.openai.com/better-language-models/