> Reasonable to assume that in 1-2 years it will also come down in cost.
Definitely. I'm guessing they used something like quantization to optimize the vram usage to 4bit. The thing is that if you can't fit the weights in memory then you have to chunk it and that's slow = more gpu time = more cost. And even if you can fit it in GPU memory, less memory = less gpus needed.
But we know you _can_ use less parameters, and that the training data + RLHF makes a massive difference in quality. And the model size linearly relates to the VRAM requirements/cost.
So if you can get a 60B model to run at 175B's quality, then you've almost 1/3rd your memory requirements, and can now run (with 4bit quantization) on a single A100 80GB which is 1/8th the previously known 8x A100's that GPT-3.5 ran on (and still half GPT-3.5+4bit).
Also while openai likely doesn't want this - we really want these models to run on our devices, and LLaMa+finetuning has shown promising improvements (not their just yet) at 7B size which can run on consumer devices.
Actually just quit my job at Meta to do just that. So you're not far off, I think you'll be surprised at how many ex-FAANG startups you'll see in the comings months.
At least in our blind chat it seems there are several other like-minded people.
Are you using opencharka[0] then? I investigated that a while ago and was not very impressed or happy with any code that it generated, but curious to see if you've figured something else out.
Hi! I'm an engineer here at Retool. I actually did build just this (a basic diffing logic tool) not long ago. It's currently behind a beta flag, but I opened a PR up this morning to get this to ship to production.
If you're a Retool user you can find it under the Beta section in your org settings. And yes, this is available on our cloud offering
Nice. Yeah you guys have tried to recruit me more than once or twice ;) one thing I would strongly suggest as you roll it out is setting a retention policy and throttling/squashing diffs as you can have users that make lots of edits and it can take up a lot of data and become hard to follow the logs for the end users! Great work coming up with this pragmatic solution, best of luck with the rollout, and congrats to the team!
It’s worth noting that the Supreme Court case that decided this, Schenck vs United States, was primarily about outlawing anti-draft speech. (In a sense, this actually strengthens the parent comment’s argument; speech which might cause harm, like a fake EAS or anti-draft speech, really should be protected by the 1st Amendment.)
How does broadcasting this tone create a clear and present danger?
It's merely a call to attention and then you need to listen to and process the subsequent message. The tone itself does not create a clear and present danger, but the subsequent message may.
There's no notion of panic or stampede with this tone, it only communicates "hey, listen up, there's something that may be important for you to know about". I'm sure that many of those living in America on HN have been subject to EAS messages that were not of serious concern since it's used pretty liberally for even minor concerns like a tropical storm.
It's also worth know about the Supreme Court case (Schenck v United States) that lead to Oliver Wendell Holmes, Jr's opinion that lead to this legal test, since the issue at hand in that case would most likely be protected by the 1st amendment today. Furthermore, Holmes eventually reversed his position in Abrams v United States.
No reasonable person would look at any of the examples where a fine has been issued over use of the EAS signal and conclude that any of those artistic statements were even close to creating a clear and present danger.
"How does broadcasting this tone create a clear and present danger?
It's merely a call to attention and then you need to listen to and process the subsequent message."
False alarms can create a danger because they condition people not to take an alarm seriously.
Even if people are pretty conscientious about treating alarms as if they are real, very few of them will continue doing so against peer pressure.
> because they condition people not to take an alarm seriously.
That'd be a stronger argument if the EAS and its predecessor the EBS were not broadcast with sufficient regularity that people approach such messages with some degree of skepticism already.
I was looking over the Wikipedia article for the EAS and I found one illegal broadcast interesting:
> In February 2011, the morning show of WIZM-FM in La Crosse, Wisconsin played a recording of the aforementioned "dead bodies" EAS hack. It inadvertently triggered the EAS on WKBT-DT, relaying both the message, as well as the hosts' laughter
Not even being able to discuss and listen to related primary material of a recent newsworthy event definitely gets into first amendment violation territory.
Since when is a tropical storm a minor concern? That’s one of the stranger assertions to wake up to on a Saturday morning. The National Weather Service maintains a nice list of EAS event codes here: https://www.nws.noaa.gov/nwr/info/eventcodes.html
47 CFR 11.31 defines the AFSK signal used for transmitting EAS alerts. The encoder settings are unique enough to not be accidentally used in any regular setting. For emergency planning purposes especially in lessons learned post-Katrina, avoiding alert fatigue was deemed by internal & external evaluators as being critical. The cited code section is here: https://www.law.cornell.edu/cfr/text/47/11.31
As for being artistic, sometimes there are boundaries. This is one that shouldn’t be crossed.
I'm no fan of Musk's, but it's so disingenuous and discrediting to say things like that.
Musk very clearly did _not_ get stoned with Joe. Taking a puff off of a joint is akin to taking a drink or two of beer. You're going to have much more of a placebo than anything.
There's no reason to lie to make Musk look stupid. He's doing that plenty well himself.
It’s news. Smoking weed as a guest on a talk show / podcast is something no CEO of a noteworthy company has done before. I don’t care per se but come on, it’s obviously an attention grabbing thing to do.
Nonsense. Plenty of CEOs drink excessively, pay for sex and do cocaine. They don't do it on video with Joe Rogan, though. That shows such a complete lack of discretion.
No it's people who are invested in Tesla being concerned about Musk's mental health and his ability to run his business. In particular because that wasn't his only scandal.
And it's got nothing to do with the US either. Do you see many German, Swiss or Japanese Ceos smoking pot and getting drunk on podcasts?
Because a clearance requires you to have discretion and control of yourself to maintain the secrets you learn due to it.
If you lack the self-control to not use a federally illegal drug on camera, then do you have the self-control to not leak secrets to score points in an argument or if you're drinking with a friend and they apply some social pressure, for instance? Maybe, maybe not, but it raises red flags about how trust worthy you are.
The reason I was told by someone who applied for security clearance is because it's a crime (federally) and they are afraid that someone could use that to blackmail you into giving state secrets.
I think it's a similar thing for illegally downloading music or whatever..
But you don't get to pick the input type. What's your adaptive, non-probabilistic perfect hash function for all data types?
I'm only aware of partial solutions to that problem, of which top down Radix sort is in fact one. But once you do that you don't need the table to solve the problem anyways, you've already got your discrimination function and you'd just pass that over the data.
I cannot see why the tabular part of the hash table proposed solution is anything more than cargo culting.
GPT3 on release was more expensive ($0.06/1000 tokens vs $0.03 input and $0.06 output for GPT4).
Reasonable to assume that in 1-2 years it will also come down in cost.