tldr: old benchmarks saturated, methodology was liable to a lot of subtle biases. as she mentions on the pod, they're already working on leaderboard v3.
It feels hard to ask this without sounding snarky, but how do you separate growth from making good decision and having a good product from growth (in this case) because AI is hot and HF offers a lot of utility for free?
Interesting. Independent of the specific strategies, I always think of HuggingFace as squandering potential. They could/should/can be the singular place where you explore _and_ productionize these open/social models. But their offerings are nowhere near as good as Replicate and the others. At least last I checked.
DevRel is just a transient phase during ramp-up. Once you get market dominance, you squeeze the independent developers by controlling their platform access and pricing, and keep all the high-revenue products for yourself. See Apple, Microsoft, Google, etc.
I'm curious here how do they make money? i heard that they have ai consulting service. is that sufficient to run the company? i don't think any researchers i know pay for hugging face.
I had a similar idea for something like hugging face in 2016. (I feel so stupid I didn't execute)
My business model was that people with great models trained on proprietary data sets would buy & sell trained weights and fine tuned models for private use.
Here is an example.
No e-commerce business is going to easily publish their sales data for everyone to train so they can build a great model for optimizing recommendations of products.
Hiring an internal AI team may have 2 problems.
- It may cost too much
- May not work as well
- You may not have the appropriate data set or data size (DL is very data intensive).
For a small fraction of what it may cost to hire your own team. You may buy a fine tuned models or readily trained weights that you just "plug & play".
That sounds like reasonable model.
The client (e-commerce) saves money, time, is guaranteed success, and gets the bonus productivity output from AI integration.
What do you guys think? Tarpit idea?
If I owned hugging face, I would follow such a strategy.
I’m not entirely sure but I suspect through their ML hosting/inference/training products and services they have working with enterprises. But I’m still baffled how they cover their AWS storage and egress fees :)
I remember seeing a post last week-ish from Julien Chaumond that they are profitable.
That is a well known lesson. In any goldrush it is always profitable to sell pickaxes and spades to the miners. The miners themselves generally do less well (except the happy few that strike gold)
I mean, it's also the fact that they were the first to create a hub for the most massive secular technology trend since the advent of the internet. It's almost like anything short of utter incompetence would have worked here. So as interesting as it is to hear how they did it, it's not necessarily actionable advice or the most efficient, effective, productive approach
This seems like survivorship bias. Kaggle was pretty well-positioned to do what Hugging Face is doing (and in fact, they are trying now somewhat). There's also a lot of others that tried to build a marketplace for AI models and failed.
Kaggle always was a place to conpete in data science contests to me. I never associated it with anything else. If they did sell ML models they did a horrible job letting people know about it
Lucky to be part of this company and seen this strategy work close up. One thing I'll add is this "decentralized" approach applies to all Hugging Face teams, not just the developer advocacy team. Just to give an example, there's no central comms team at Hugging Face, every team (usually the engineers who work on the product or features) does their own comms across the channels they think work best. That means there's lots of experimentation and most of our hires tend to be generalists who are comfortable wearing many hats.
In every company I've worked for, the biggest fans/lovers of meetings/let's "touch base" have always been: The most useless members of our team.
And I truly mean that, whether PM/TPM/or even SDM/SDE. The common denominator is they were always absolutely useless, adding no value or no true technical contribution, and their only value add was siloing information and claiming to be in "meetings" to lead some effort forward.
Their entire job function could've probably been replaced by a wiki or Google Doc. They intentionally made themselves the only points of contact and did not introduce people cross team because people would probably immediately realize how useless they are.
I've seen this multiple times, multiple roles, L5+, earning 100s of thousands of dollars a year at big tech companies.
Yes, I will waste 45 minutes of my time explaining word for word to you, exactly what is written in this design document, and none of it will be documented, no AIs will come from it, so you can look "busy" for today and tell your manager you did something. Woohoo collaboration!
Their entire job function could've probably been replaced by a wiki or Google Doc.
I'm an Engineering Manager these days and I suspect this is true to an extent for my role. All I seem to do is relay information from one group to another, translate from one jargon to a different jargon, figure out who to talk to in order to get someone to be able to push a button, and, more often than I'd like, sit in meetings and remember things when someone else has forgotten. If my teams and the teams they work with wrote things down in discoverable and organized ways I would be out of a job.
However, due to the fact my teams are made up of fallible, disorganized, and very human people, my job seems pretty safe, and I get a great deal of satisfaction from knowing I help a lot of people get a lot more done than if I wasn't around.
I'll give you a big reason why this behavior exists: people refuse to write high quality documentation regarding their systems/processes/etc. Or they are willing do it, but do an extremely poor job and are oblivious to its low quality. This will necessitate meetings to "Clear things up". And sometimes your documentation ticks all the boxes, but people get pissy about updating it because it's not in their preferred format and it rots away into irrelevance.
When I became an engineering manager I invested a ton of effort into providing solid documentation where people expect it: Swagger docs with real examples for our RESTful services for devs, confluence documentation for wider audiences, various guides and FAQs, and even in-repo ADRs. It probably saves us about 15 labor hours of meetings a week and maximizes my team's hands-on-keyboard time and minimizes interruptions.
Hard to say what situations you have experienced because the details are scant, but in my experience usually those people have helped, in one way or another, to ensure that my pay check has the specific number that it does.
Simply put: they use the information they gain to justify the utility of individual team members to the overall effort and thus the proportionate compensation.
No my friend, the people who ensure the paycheck has the specific number it does are some drones in HR far removed from you. They determine reasonable bands for your role based on industry standards and how the company wants to position itself in the market.
Then your individual contribution and performance determines an additional +-10%.
Don't know which company you work for but this is unequivocally not true at mine. HR are subservient to Engineering (our CEO started as a senior SWE ~15 years ago). I am also one of those "useless" EMs that barter for the comp packages of my most valuable SWEs.
From my experience there's a lot more wiggle room in those pay bands than you'd think. Along with that, at many orgs your EM has to haggle on your behalf with their boss and every other EM that shares their budget ~yearly for your share of a limited pool of incremental dollars. I've specifically requested not to be under certain EMs in the past for that reason.
I don't think meetings or OKRs are inherently bad, I have worked in organisations where these kinds of things are actually done well. Equally no OKRs and no meetings can be fine as well.
Where the problem exists is when the systems and processes business use start to become calcified into the organisation's way of working, and compliance is used to control the work people do.
RACI, RASCI, DACI as an example are all fine mental models for thinking about how people or teams should work on a project. But if I'm planning a project and propose a DACI, and then my project ends up getting derailed because some other manager thinks we should use a RACI instead it's probably game over.
> Yes, I will waste 45 minutes of my time explaining word for word to you, exactly what is written in this design document
And they will misunderstand you, turn around and butcher the explanation to some higher up, indirectly claiming credit for your work as the messenger of its result. If something goes wrong with the project, they'll definitely make sure to emphasize your involvement in the failed initiative, strategically distancing themselves.
The kind of momentum maintaining alignment can provide is unreal. I hope this topic and HuggingFace and other doing it keeps getting more attention and details being shared. Nothings perfect, but it sure seems like it would work well for some people :)
Having a team of thoughtful and professional self-directed learners and creators sounds amazing if you can get a space for a team to come together around it.
Companies that have to, or choose to compromise on their labour force, either from a competency or salary perspective often have to put more supports in place to keep people longer.
It’s kind of crazy you’re describing the person that made me leave tech to the last detail
The weird part to me is I’ve worked with PhD folk, DARPA hackers, FAANG folk and they were so kind to me even when I said dumb things the person on the other hand was just idk how to describe it made coding just painful idk why though to this day
> "growing from 780k to 2.3m repos on the HF Hub in the past year"
Is this what caused the title to be "300%"?
a. that's actually ~200%
b. isn't growth, as defined by number of repos (which might be free) the wrong metric. It's super is to scale adoption when you're giving something away for free
It's kind of confusing because it depends on how you phrase it with percentages.
"The number of cores in the PC went up by 50%" means "there are now 1.5x as many cores"
However, "the number of cores in the PC is 50% what is was last year" means "there are now .5x as many cores"
In this case 200% is fairly unambiguous, but imagine a phrase like "scaling community 50% per year" which could mean different things depending on the direction of the scaling... I think it's more intuitive to use a multiplier: "scaling community 3x bigger per year" makes it easy to visualize an online community tripling in size every year.
(b) growth is still the right metric imo. for example, lots of open-source libraries (which by definition are available to use for free) measure and optimize for growth
Sorry you received unhelpful sass from other commenters. It stands for "Objective and Key Result" - it's a common "business-speak" initialism that basically translates to "a measurable, data-driven goal".
I'd say it's more than that. OKRs are a concrete framework that came out of Intel and was popularized by Google. But yes, the focus is on setting objectives backed by measurable results.
Can be a document, an app, whatever. One flavor is objectives are set at the top, their KRs are translated into objectives for the next management layer with their own KRs, and so on. It's an iterative process with some combination of top-down and bottoms-up activities. And afterwards, the results are reviewed. There's more to it, but that's the gist.
An objective is a qualitative goal. The key results are generally quantitative numbers that can help you measure if you are achieving that goal.
An example: an objective might be "improve application performance". The key results for that objective might be "reduce average page load time from 3s to 1.5s" and "reduce API response times by 60%".
If you were to tell your CEO you're reducing page load time, they will rightfully ask you why and what problem you're solving. The objective is the simple sentence that makes the "why" clear. It might itself be subordinate to a higher level objective. The idea is that all work is clearly aligned to specific goals in a very transparent manner.
It's actually a very solid framework, but as the many comments indicate, poor implementations abound (also see Agile, DevOps, etc).
The point is that you set a goal and made progress towards it. Ideally you choose goals for yourself and it's expected that you won't complete them all.
My experience of them was that upper management set "Objectives" like "1% YoY Online Sales Growth" for your group or team or whatever and then each team in that group (again, or whatever) would come up with their own "Key Results" that they thought, if the result was achieved, would translate to that objective.
So for example, if you worked in online retail like I did, maybe you'd get an objective like that and then hypothesize a few things along the lines of "if we increase product image interactions by X% then sales should go up by Y%." as key results that you then report back out to management to show progress towards your objective.
Why is it that whenever a company happens to be successful, and does meetings/goals differently than most (also successful) companies, the success is attributed to the lack of meetings/goals, and not, y'know, the business?
I think there are a few reasons for this. The primary one is that it's a visible signal, and people make attribution errors almost constantly so it gets fixated on. I think the other reason might be that it's unpopular (and not actionable) to say "this company succeeded because the people there are better than you" or "this company had great product market fit, and the employees efforts really didn't make the difference"
This suggests the iterated optimum strategy is to just keep chasing new points trying to land that golden moment, though. If you think such points and moments are plentiful enough that you'll find them in your lifetime reliably (I do!).
DevRel is collection of strategies and tactics that help companies to work better together with software engineers.
Exactly what developer relations teams do and why they do it depends on what their organisation needs.
