On: DevRel at HuggingFace

[osanseviero]

Hi all! I'm Omar from Hugging Face. Happy to answer any questions you might have about Hugging Face in general, llamas, and open ML!

[brianjking → osanseviero]

I'd love to work at Hugging Face! Happen to be hiring any new dev relations/product/AI engineer type roles?

This is such a great piece.

[ZoomerCretin → osanseviero]

Could you talk more about HuggingFace's new benchmark for LLMs? When did it become obvious that the old benchmarks were no longer sufficient:

[swyx → ZoomerCretin]

[author here] we interviewed the maintainer of that leaderboard if you want to hear from her directly! https://www.latent.space/p/benchmarks-201

tldr: old benchmarks saturated, methodology was liable to a lot of subtle biases. as she mentions on the pod, they're already working on leaderboard v3.

[Hooray_Darakian → osanseviero]

How large is the staff at hugging face?

[abidlabs → Hooray_Darakian]

We have ~220 total team members across all roles