ilove196884's comments

On: Tensor Product Attention Is All You Need

[ilove196884]

I hate how paper titles are worded like seo techniques.

[spiritplumber → ilove196884]

Turn something into a metric and it will be misused. Ever always was

[jampekka → spiritplumber]

Attention is all you need!

[visarga → jampekka]

138,000 papers are "X is all you need", now 138,001

https://scholar.google.com/scholar?hl=ro&as_sdt=0%2C5&q=%22i...

[hatthew → visarga]

640 if you use intitle, exclude citations, and filter to 2017 onwards

[llm_trw → jampekka]

The attention economy and its consequences.

[littlestymaar → llm_trw]

Unexpected Kaczynski.

[TeMPOraL → spiritplumber]

> Ever always was

Always has been.

silent gunshot

[verdverm → ilove196884]

This is a riff on the original "attention is all you need" paper, there has been a few of these lately

[Matthyze → verdverm]

A few? A multitude.

[verdverm → Matthyze]

This one might be right if they have in fact unified multiple attention approaches into a single framework

see Section 3.4

[LPisGood → ilove196884]

Having a catchy title is great for short hand. If it didn’t have such a catchy name I probably wouldn’t remember Flush+Reload, Spectre, or even Attention is All You Need

[Upvoter33 → ilove196884]

On the one hand, sure, it's dumb.

But, on the other hand, it's hard to get researchers to read your paper, esp. in fast-moving areas. Every little thing might be the difference between reading the abstract or not. Reading the abstract might lead to reading the intro. And so on.

So, for better or worse, the competition for human eyeballs is real.

Ironically, in this case, "attention" is all that the authors want.

On: Show HN: DBOS TypeScript – Lightweight Durable Exe...

[ilove196884]

I know this this might sound scripted or can be considered cliche but what is the use case for DBOS.

[qianli_cs → ilove196884]

The main use case is to build reliable programs. For example, orchestrating long-running workflows, running cron jobs, and orchestrating AI agents with human-in-the-loop.

DBOS makes external asynchronous API calls reliable and crashproof, without needing to rely on an external orchestration service.

On: Microsoft acquires twice as many Nvidia AI chips a...

[ilove196884]

See I absolutely dislike this thought that hyperscalars can easily beat nvidia. It is not their domain of expertise. Tpu are not where near GPU in performance. People really underestimate Nvidia's expertise and strengths.

[Etheryte → ilove196884]

They don't need to beat them on performance though. If you get half the performance at third the price you can just make more chips and be fine. It's not like Google is gonna run out of datacenter space.

[ilove196884 → Etheryte]

TPU support is very minimal. Also XLA just does not compare to CUDA yet.

[Etheryte → ilove196884]

All of this may very well be true — but it doesn't matter. Google is getting results very similar to the rest of the pack, probably at a fraction of the cost. The implementation doesn't matter so long as you get the results.

[mupuff1234 → ilove196884]

And yet Google seems to be doing just fine while spending less on GPUs than other big tech, so reality doesn't seem to align with your comment.

[ilove196884 → mupuff1234]

https://www.hpcwire.com/2024/10/30/role-reversal-google-teas... Well they are still buying GPUs.

[mupuff1234 → ilove196884]

Mostly for external cloud customers so doesn't change my point - TPUs have been more than enough for Google to lead the pack.

On: Microsoft acquires twice as many Nvidia AI chips a...

[ilove196884]

Also TPU v1,v2 and v3 were ASICs, but since v4 they have added some new features so they have a lower performance/watt which is quite near Nvidia's power draw. I think Hopper is at 700W and TPU are around 600W.

[YetAnotherNick → ilove196884]

Power draw doesn't matter in cloud. All that matters is performance/price for the task in hand.

[jeffbee → YetAnotherNick]

In the cloud, opex == energy.

[talldayo → jeffbee]

Which is why you don't send 4 TPUs to do 1 GPU's job.

[jeffbee → talldayo]

If the TPUs cost less, that implies that they draw less power. If they cost more then nobody will use them.

On: Microsoft acquires twice as many Nvidia AI chips a...

[ilove196884]

They are overhyped and not as performant as Nvidia regardless of marketing.

On: Microsoft acquires twice as many Nvidia AI chips a...

[ilove196884]

Meta is Nvidia shop for training and AMD for inferencing. Strict seperation between vendors.

On: Microsoft acquires twice as many Nvidia AI chips a...

[ilove196884]

Meta is still GPUs. Amazon Trainium 1 was a failure and is trying an upgrade with Trainium 2. Google is a TPU shop but still busy GPUs for cloud.

On: SHA-256, ECDH, Ecdsa and RSA Not Approved by ASD i...

[ilove196884]

With what hardware? These PQC standards are still slow and definitely not ready for deployment.

On: Meta's MySQL Fork You Never Knew About

[ilove196884]

I would honestly love to see a full documentation and history of MySQL use a Meta considering it is the largest MySQL shop in the world. Would honestly give soo many small tricks and ideas.

On: How Russian hypersonic missiles work [pdf]

[ilove196884]

Damn. They really implemented an MHD generator. That's like the holy grail in hypersonics. Active flow control is definitely well sought after in hypersonics.