Hacker News new | past | comments | ask | show | jobs | submit | sytelus's comments login

So, NanoGPT took 1.8 days on 8xA100 for 124M model training on 30.7B tokens using flash attention. This would translate to 14.4hr for 10B tokens. With llm.c it is ~1.5 hr which is almost 10X speedup!

Does this look ballpark correct? Is there any summary of where majority of this improvement comes from?


This doesn't make sense. There are tons of other business which will have higher day to day return and property appreciation. I don't think anyone was craving for car wash subscription and I don't know of anyone having car wash in their checklist monthly item. Those people exist for sure but not in that kind of demand that a town needs even two of them.

I think this is just another instance where private equity is creating imaginary business model of subscriptions and selling it to its investors. The money managers will get huge management fees and will disappear before the whole thing collapses in coming years. Meanwhile, they will run a lot of good and necessary businesses out of town.


It's hard to think of many businesses that require lower capital investment and maintenance than a car wash. The ones I see in my city that look like land banking are a big plot with asphalt covering it (don't even need to mow lawns), a few awnings with some plumbing for hoses and soap and other gear, and nothing else. Even a laundromat would cost more to set up.


https://www.carwashadvisory.com/learning/how-much-costs-buil...

These car washes are not simple, nor cheap. They aren't talking about places where you wash your own car, they're talking about the increase in automated ones.


Exactly. These people are centi-millionaires and beyond employment, promotions, bonuses and all that usual drudgery. I guess the article is funded purely not to injure egos. But, hey, what is all that downscaling of YC? The article is clearly official press release and paid for.


> These people are centi-millionaires

They have $10,000?


The dictionary suggests this means they have $100 million. Likewise centibillionaire means $100 billion.


What dictionary is that? Im American and even here we learned hecto means 100 in 1st grade.

Centi means 1/100th. (1) e.g. centifoot = 1/100th of a foot.

(1) https://dictionary.cambridge.org/dictionary/english/centi


centifoot... You had centimeter right there! "Americans will do anything to avoid using the metric system" lol


“American Football is a game of centiyards, not feet.” — Official Hand-egg rulebook


Yeah would've made way more sense. Either that or centigrade.


centipounds?


Hecto- is the Greek root for 100, centi- is the Latin root for 100.

The metric prefixes up to 1000 use Greek roots for the large prefixes (10^+n) and Latin for the small (10^-n), but there are also words that are derived directly from the respective Latin roots and therefore don't necessarily carry the metric system's fractional meaning.

Someone already linked to the definition of centimillionaire, but here are a few more:

* centipede (100 legs, not 1/100 of a leg)

* centenarian (100 years old, not 1/100 of a year old)

* millennium (1000 years, not 1/1000 of a year)



Reminds me of how financial newspapers often use a small "m" when describing millions of something. "$500m", or even worse "500m USD"... The engineer in me says that's 50 cents!


In-depth discussion in this topic:

https://xona.com/2006/12/17.html


Centiliter = 0.01L

Hectoliter = 100 L


Centennial = 100 years

Centurion = Commander of 100 legionaries


You: logic

The English language: "allow me to introduce myself!"


Centipede = 100 legs

Etymology is more complicated than "look up the prefix in the metric system tables", and OP is correct about the definition of centimillionaire.

https://www.merriam-webster.com/dictionary/centimillionaire


Centipede is Entomology


Brilliant, I laughed out loud


I know it is a word people use, and "centi-" is ambiguous without no context, but it's extremely unambiguous in the context of units of measure, and the word "hundred-millionaire" already existed and had the same number of syllables, so it's a really awkward choice.


You can work backwards. For $5T investment, you need revenues of at least $500B. Nvidia revenues are currently ~18B/quarter or roughly $75B/yr. If you think 5X growth in AI in next 5 year is possible but current supply chain cannot deliver then all these totally makes sense.

To me, AI is starting to replace convectional search within early adopters already. The new search is not firing query to get references but rather converse to get answer. In my vicinity, 90% of the population has not even caught up on this progress, let alone change of muscle memory which often takes half decade.

And search is still just one of many segments. So, market has 10X growth potential from here with fair certainty. Although, as it often happens, we will overshoot expectations that causes crash followed by more regularized growth.

Fun thing is that a ton of wild predictions that created Internet bubble in 2000 actually were rather timid. Internet did changed everything and delivered way beyond expected during the bubble. It was only a matter of time.


Try 3 years for 5x ;)


I was going to impulse buy it but then they have put so many barriers that impulse buy is not possible. You must measure your face and have eye prescription ready! Wtf? I am in my bed and not going to do that for the privilege of pressing buy button. Now it’s not surprising that it’s still not sold out.


I thought it was crazy you're supposed to download an app to measure your head to get your band size when I have two soft tape measures laying around.


The shape of your face and how big your nose is also matters.


You don’t need the prescription on hand, you upload it after the fact. Whole process took like 2 minutes in the Apple Store app ¯\_(ツ)_/¯


You can enter your prescription later.


They didn't said that. Classic sales mistake.


Why authors miss to compare with Phi-2?


Agreed, and not only do they not compare their model to Phi-2 directly, the benchmarks they report don't overlap with the ones in the Phi-2 post[1], making it hard for a third party to compare without running benchmarks themselves.

(In turn, in the Phi-2 post they compare Phi-2 to Llama-2 instead of CodeLlama, making it even harder)

[1]: https://www.microsoft.com/en-us/research/blog/phi-2-the-surp...


Just until last months people were asking where is all AI spam. It was slow and then sudden.


Amazon is getting flooded with ChatGPT generated spam.


This is exactly what I learned working on Internet scale data. A new dude will walk in, proclaim that a simple rule will solve all your problems.


There very often are easy solutions for very niche problems, simply because nobody has bothered with it before.

I don't see how a search result with 7 pages is supposed to demonstrate that this idea wouldn't work? I'm not saying whether it would be particularly helpful, but a human can review this entire list in a handful of minutes.


Why Amazon is not able to actually verify sellers real identities and terminate their accounts? I would imagine that they should be able to force them to supply verifiable national identification/bank account etc. How do these sellers get away with these?


Amazon does verify and also actively tries to blocks bad sellers. Details of this are easily found online.


Is there a problem with this seller beyond their tooling malfunctioning?


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: