As I understand, the LLM uses the techniques of searchformer - https://arxiv.org/abs/2402.14083. To do "slow thinking" doing a A* search using a transofrmer.
There was no answer to my last question which I think is the most important thing when considering if we are going to have another GFC this year or next year.
Does "GFC" stand for "global financial crisis" here? It seems implausible that the collapse of the LLM bubble will cause one; it might have a pretty dramatic impact on the markets, but it's unclear how it would cause the sort of systemic failure that we saw in the noughties.
TLDR: Don’t buy H100s. The market has flipped from shortage ($8/hr) to oversupplied ($2/hr), because of reserved compute resales, open model finetuning, and decline in new foundation model co’s. Rent instead.
At $2/hr, that's 2.8 years to RoI. And that's just for the GPU and not the other hardware you'll need to plug it into, and doesn't include the power, and also assumes you're using it 100% of the time. Really, you're probably looking at 3.5+ years to RoI.
I know you mean this in jest, but we are much closer to this than we would imagine, the use of LLMs to process communication / translation is becoming ubiquitous. We are 1 bad translation away from a disaster.
> Could you illustrate a likely scenario that's in your mind?
The year is 2034; after a surprisingly cutthroat economic trade-war fought between China and the US left the world in the throes of another great recession, a growing wave of anti-China sentiment captures the attention of domestic political leadership which cultivates the movement despite (or more likely: because of) the growing interest from xenophobic reactionaries and other populist movements looking to scapegoat their way out of a dip in GDP - eventually those same poltiical-actors win the presidency and use their democratic mandate to instigate a new McCarthy-era of anti-China paranoia leading to utterly deranged domestic policy, namely as the executive ordering, by-decree, that the State Department terminate the employment of anyone who even speaks Mandarin[1] - a few weeks later in the South China Sea another Filpino/Sino boat-ramming incident escalates into something serious - the US Navy urges the US civilian government to communicate with China over the D.C.-to-Beijing "red telephone" deescalation e-mail system, but no-one knows how to communicate to the Chinese in their own language, so the overworked federal employee manning the red-Outlook-inbox sees nothing wrong with simply having that Microosft Office 365 CoPilot translate it for him - the same AI bot that's somehow always on his screen with that distracting sidebar (despite the best efforts of the US Federal Gov's Active Directory Group Policy) - it wasn't long before the first warheads exploded over North America that the President learned the AI translated the polite request to China for them to "please stop ramming the fishing boats" was received by them as "I'll ram my fish into your Junk...boats". If there's any upside to this story, the collective mass of AI were wiped out first by the high-altitude EMP bursts, leaving us humans with the last-laugh before we were all incinerated moments later - while those not fortunate enough to die instantly instead suffered months of prolonged fatal radiaiton sickness while what little left of civilization collapsed around them[2].
[1]If you think that's too ridiculous to be realistic, consider the Japanese internment-camp policy or Trump's declared Muslim ban. Elsewhere, in the late-1970s (in the age of CT Scanners and VHS tapes), Pol Pot targeted people for wearing glasses.
[2]Blame James Burke's editorial slant in his documentary series' for turning me into a nhilist.
That scenario has me less worried, just because by 2035 all popular politicians in America and China will be AI deepfakes run off the same cloud servers.
[edit]
I herewith introduce a new shitcoin called NukeCoin. Everyone in China and America gets a NukeCoin that will go up in value every day you hold it that no one nukes anyone.
I also use some email providers ability to have +xyz at the end of the username. So for a registration I would for user.name+sitea@domain.com. has helped me track spam and leaks in the past.
Same here! Both GMail and iCloud Mail support this, among others.
It works 95% of the time, but be aware that it sometimes doesn't:
- Some webpages refuse addresses with a + in it.
- I’ve had at least one instance where I lost an important email because I could register and receive emails from them, but one of their internal databases silently failed on my address. (I lost the proofs for a paper accepted in a journal, and it took some time to figure out what was going on…)
- There was at least one web page that was smart enough to just remove the +… from my address after I registered.
- Some webpages require you to be able to send emails to them from the exact address you sign up with (including the +…)
Overall I’ve been happy with the feature since it makes e-mail sorting easier, and you can just redirect a given subaddress to spam if they leak it, and it’s less effort than creating a “real” email alias per page. Just keep in mind that there are many ways it can fail, so you might not want to use it for anything that is actually important.
Since I run my own email server, I get around some of those issues by configuring Postfix's recipient_delimiter to use . rather than +. I've never had a site treat accounts.company@mydomain.com as anything other than a normal email address. It certainly doesn't justify the effort I've spent dealing with deliverability issues, but it's a nice perk.
I literally setup an alias last week in O365 Outlook using the pattern a.b@c.com? I’ve been able to receive and send using the alias as well. Maybe this is a new feature/behavior?
I may have misunderstood the parent comment - with gmail, you can add dots anywhere in the mailbox and it all goes to the same place (standard gmail, not workspace)
e.g andrew@gmail.com, a.n.d.r.e.w@gmail.com and a.....ndrew@gmail.com all are the same user and will go into their mailbox (which I have used to avoid the + stripping that some sites do)
andrew@outlook.com and a.ndrew@outlook.com are two distinct users.
Obviously if you control the domain or use a provider who supports it you can add an alias with punctuation but then you might as well just use e.g ebay@c.com to track the email source.
I have a mail with catch all address so o can avoid the + so sites can never be strip it as it just randommail@domain, just have to hope nobody else figures it out and spams me at random addresses
The benefit of the + is that I don’t have to set up anything in advance. I can type an email address on any device, including devices I don’t own, and still have it work as an alias.
I do have a “real” email alias as well that I use for true spam (e.g. to get “customer club” discounts while everything they send me is autoredirected to spam by a mail filter). I reserve the + addresses for things I want in my inbox, but still want to be able to filter easily in my mail client.
I think the non English part is mostly hit and miss in this primitive version, probably cos the implementation is not correct. I got to read up a lot and fix it.
I just checked huggingface and the model files download is about 25GB but in a comment below someone mentioned it is 8fp quantized model. Trying to understand how the quantization affects the model (and RAM) size. Can someone please enlighten.
Sure. The talk about 8bit refers to quantization-aware training. Pretty common in image models these days to reduce the impact of quantization on accuracy.
Typically this might mean that you simulate an 8bit forward pass to ensure that the model is robust to quantization ‘noise’. You still use FP16/32 for backward pass & weight updates for numerical stability.
It’s just a way to optimize the model in anticipation of future quantization. The experience of using an 8-bit Nemo quant should more closely mirror that of using the full-fat bf16 model compared to if they hadn’t used QAT.
