The article starts off by saying they all left Google. There's also an entire section narrating how google execs passed on transformers and didn't pick up until OpenAI released GPT.
>The picture internally is more complicated. “It was pretty evident to us that transformers could do really magical things,” says Uszkoreit. “Now, you may ask the question, why wasn’t there ChatGPT by Google back in 2018? Realistically, we could have had GPT-3 or even 3.5 probably in 2019, maybe 2020. The big question isn’t, did they see it? The question is, why didn’t we do anything with the fact that we had seen it? The answer is tricky.”
>Many tech critics point to Google’s transition from an innovation-centered playground to a bottom-line-focused bureaucracy. As Gomez told the Financial Times, “They weren’t modernizing. They weren’t adopting this tech.”
> There's also an entire section narrating how google execs passed on transformers and didn't pick up until OpenAI released GPT.
Google certainly seem to have fumbled on rapidly exploiting the tech, but I'm not sure that's totally fair.
Google built BERT in 2018, around the same time as OpenAI built GPT-1, but neither were released.
OpenAI's GPT-2 was built in february 2019 but OpenAI were hesitant to release it due to potential danger, only doing so later in november 2019. GPT-3 followed in july 2020, but ChatGPT which the first accessible to public view of the tech didn't appear until november 2022 (how time flies!).
Google's first transformer-based chatbot, LaMDA, was released in may 2021, well over a year before ChatGPT, but was then rapidly withdrawn due to negative feedback and behavior.
The difference between ChatGPT's acceptable behavior, and success, compared to LaMBDA, was OpenAI's invention of RLHF to better control it. One could argue that Google should have invented something similar themselves earlier, but surely this might have been expected to come from these same people who had been working on language models at Google ...
>The picture internally is more complicated. “It was pretty evident to us that transformers could do really magical things,” says Uszkoreit. “Now, you may ask the question, why wasn’t there ChatGPT by Google back in 2018? Realistically, we could have had GPT-3 or even 3.5 probably in 2019, maybe 2020. The big question isn’t, did they see it? The question is, why didn’t we do anything with the fact that we had seen it? The answer is tricky.”
>Many tech critics point to Google’s transition from an innovation-centered playground to a bottom-line-focused bureaucracy. As Gomez told the Financial Times, “They weren’t modernizing. They weren’t adopting this tech.”