On: Ilya Sutskever to leave OpenAI

[nabla9]

There are no moats in deep learning, everything changes so fast.

They have the next iteration of GPT Sutskever helped to finalize. OpenAI lost it's future unless they find new same caliber people.

[sk11001 → nabla9]

> They have the next iteration of GPT Sutskever helped to finalize

How do you know that they have the next GPT?

How do you know what Sutskever contributed? (There was talk that the most valuable contributions came from the less well known researchers not from him)

[nabla9 → sk11001]

sha256:e33135417f7f5b8f4a1c98c28cf26330bea4cc6b120765f59f5d518ea0ce80e5

[jacooper → nabla9]

What should this mean?

[criddell → nabla9]

Isn't access to massive datasets and computation the moat? If you and your very talented friends wanted to build something like GPT-4, could you?

It's going to get orders of magnitude less expensive, but for now, the capital requirements feel like a pretty deep moat.

[Gud → criddell]

How do you know massive datasets are required? Just because that’s how current LLMs operate, doesn’t mean it’s necessarily the only solution.

[datameta → Gud]

Then the resources needed to discover an alternative to brute-forcing a large model are a huge barrier.

I think academia and startups are currently better suited to optimize tinyml and edge ai hardware/compilers/frameworks etc.