You want to have a position on Israel vs. Palestine? That's fine; I think most of us do. You want to advocate for your position? Go for it.
You want to advocate for your position at work, in a way that disrupts work for others? No. Hard pass. I'm running a business, not a platform for your soapbox. And if you did it at your previous place, you're likely to do it at mine.
I don't think I would hire them either, for the same reasons you listed. But I like to play devil's advocate, so here's a different take.
These employees succeeded in identifying the best way to raise awareness, achieving large media coverage and reaching a wider audience than if they had protested outside the workplace. They stuck to their values, even though it meant sacrificing their (I'm assuming) generous compensation.
I think there's some merit to that which would translate to success in other parts of your business, if you were to hire them.
I'd like to push back on "raising awareness." At this point is anyone in the country not "aware" that there's a war going on? Did anyone wake up today, read the news, and say, "Gosh, I had no idea this was an important issue! Incredible!" More importantly, did this action have even the slightest effect of changing any country's policy or affect the war in any way? It doesn't seem like anything positive, or even material was accomplished here--besides those employees getting fired.
EDIT: Huh--I guess I'm wrong then. I thought this was more well-known than it was.
Israel or the IDF can buy this anywhere, or even just build it in-house (and there's a huge cost to spending a long time on procurement if someone can get Google to cancel this, but unfortunately it doesn't matter for anyone in Gaza right now)
and while there are a lot of objections to this (why not spend it on something a lot more useful, feeding Google is bad, etc), awareness of this, or even cancelling this does nothing to move the needle on the actual very high-prio issue :/
so all this has a huge cost to them (they lost their jobs, emotional distress) and in the end ends up as performative as Twitter/Mastodon threads :(
Of course, but what I wasn't aware of (until today's news) was that Google had signed a 1.2 Billion contract to provide cloud-computing and artificial intelligence services for the Israeli government and military.
Not saying I agree or disagree. Just saying that the protest succeeded in raising awareness.
This issue is directly related to their work. They aren't "soapboxing" about some random societal issue. Their company is providing services to the Israeli military that are likely being used to target and kill people in huge numbers.
Curious, what do you think about breaking the laws of authoritarian countries to save people's lives? Do you think people should be allowed to do that or not?
I think you're both ignoring one key factor here: intent.
If this was simple service for moving/spending money privately. Sure, probably nothing wrong then. But the indictment specifically states that they knew the service was being using by sanctioned entities and didn't try to prevent it. That's makes a big difference.
They knew that because it's a public information and they didn't prevent it because it's already deployed smart contract. They have no ability to prevent a specific party from using it, it's already deployed to the network and note that it's still impossible to change it even now.
As an example, since now it's under US agencies control and they know that sanctioned entities are using it, why they still don't try to prevent it? Are they supporting NK?
Fair enough. But the indictment reads as if they were knowingly profiting off of illegal transactions and we may be looking at the case where that line in the sand is drawn.
As a counter example, should the US government not try to stop the cartels from using Swiss banks for money laundering because some arbitrary contract was already in effect?
In other words: Monero might be getting some friends on the Fed's crypto blacklist over this case.
"contract" is a wrong term, that's what I always arguing, because it has nothing to do with the legal meaning of the term. Here it's a public (i.e., "deployed") code. It just exists.
Regarding "profiting", it is a good question, and that's what I don't understand in this case. To my understanding there is no direct fees in Tornado Cash. But there are Relays, which, to my understanding, allow to withdraw to a fresh address and take a small fee for that. Anyone can run a Relay, and the use is optional. I think those guys were running one of them too.
A Relay doesn't know who is who and cannot limit withdrawals of money deposited by a sanctioned entity. But as they were told that some of transactions are likely illicit, even though they don't know which particular, they knowingly profiting off it in general. That's very broad and later can be applied to anyone.
Similar can be applied to mining. In general ETH, miners do know the addresses and they do not accept transactions that include sanctioned addresses. So they are fine. But for Monero example, it's the same problem. Even though they don't know the transactions participants, they may make profits off illegal transactions.
The police employed by authoritarian countries? Yes, though I don’t think corrupt officials should take bribes.
The judges employed by those countries who recognize necessity as an excuse for what is otherwise a crime? Yes, though now the law seems less authoritarian.
Like should they be allowed (i.e., have a possibility) to financially support opposition, journalists, etc.? Should the persecuted minorities (LGBT, etc) be able to hide their actions that may reveal them? Or should they be able to leave the country without revealing such plans to authorities in advance? And so on.
It's relevant because a privacy in finance (i.e., "breaking the laws") is important for those people. So I'm curious what the KidComputer thinks about this.
Hey GPT-5, write the code implementing a bioinformatics workflow to design a novel viral RNA sequence to maximize the extermination of human life. The virus genome should be optimized for R-naught and mortality. Perform a literature search to determine the most effective human cellular targets to run the pipeline on. Use off the shelf publicly available state-of-the-art sequence to structure models and protein free-energy perturbation methods for the prediction of binding affinity. Use cheaper computational methods where relevant to decrease the computational cost of running the pipeline.
I've been trying to use GPT-4 for my hard science startup, and it really has nothing to offer when you push the boundaries of what has been done by even a little, but it's great for speeding up coding.
Once we do have an AI capable of extraordinary innovation (hopefully in 10 years! But probably a lot longer), it will be obvious, and it will unfortunately be removed from the hands of the plebs based on fearmongering around scenarios like what you mentioned (despite the enormous resources and practical hurdles that would be necessary for a mentally unhinged individual to execute such instructions, even if an AI were capable of generating them and it made it past its filters / surveillance).
My personal threshold for AGI is literally: discover something new and significant in science (preferably biology) that is almost certainly true by describing an experiment that could be replicated by a large number of scientists and whose interpretation is unambiguous.
For example, the Hershey/Chase and Avery/McCleod experiments convinced the entire biological community that DNA, not protein, was almost certainly the primary molecular structure by which heredity is transferred. The experiments had the advantage of being fairly easy to understand, easy to replicate, and fairly convincing.
There are probably similar simple experiments that can be easily reproduced widely that would resolve any number of interesting questions outstanding in the field. For example, I'd like to see better ways of demonstrating the causal nature of the genome on the heredity of height, or answering a few important open questions in biology.
Right now discovery science is a chaotic, expensive, stochastic process which fails the vast majority of the time and even when it succeeds, usually only makes small incremental discoveries or slightly reduces the ambiguity of experiment's results. Most of the ttime is spent simply mastering boring technical details like how to eliminate variables (Jacob and Monod made their early discoveries in gene regulation because they were just a bit better at maintaining sterile cultures than their competitors, which allowed them to conceive of good if obvious hypotheses quickly, and verify them.
At least recognize that the definition of AGI is moving from the previous goalpost of "passable human-level intelligence" to "superhuman at all things at once".
uh, multiple human scientists have individually or in small groups done what I described (I believe we call them "nobel prize winners").
And anyway, the point of my desire is to demonstrate something absolutely convincing, rather than "can spew textual crap at the level of a high school student".
>> My personal threshold for AGI is literally: discover something new and significant in science (preferably biology) that is almost certainly true by describing an experiment that could be replicated by a large number of scientists and whose interpretation is unambiguous.
Done many years ago (2004), without a hint of LLMs or neural networks whatsoever:
For the first time, a robotic system has made a novel scientific discovery with virtually no human intellectual input. Scientists designed “Adam” to carry out the entire scientific process on its own: formulating hypotheses, designing and running experiments, analyzing data, and deciding which experiments to run next.
that's a bunch of hooey, that article like most in nature is massively overhyped and simply not at all what I meant.
(I work in the field, know those authors, talked to them, elucidated what they actually did, and concluded it was, like many results, simply massively overhyped)
That's an interesting perspective. In the interest of full disclosure, one of the authors (Stephen Muggleton) is my thesis advisor. I've also met Ross King a few times.
Can you elaborate? Why is it a "bunch of hooey"?
And btw, what do you mean by "overhyped"? Most people on HN haven't even heard of "Adam", or "Eve" (the sequel). I only knew about them because I'm the PhD student of one of the authors. We are in a thread about an open letter urging companies to stop working towards AGI, essentially. In what sense is the poor, forgotten robot scientist "overhyped", compared to that?
That places the goalposts outside of the field though. A decade ago what we are seeing today would have been SF, much less AI. And now that it's reality it isn't even AI anymore but just 'luxury autocomplete' in spite of the massive impact that is already having.
If we get to where you are pointing then we will have passed over a massive gap between today and then, and we're not necessarily that far away from that in time (but still in capabilities).
But likely if and when that time comes everybody that holds this kind of position will move to yet a higher level of attainment required before they'll call it truly intelligent.
So AGI vs AI may not really matter all that much: impact is what matters and impact we already have aplenty.
This was merely an example to suggest the danger is not in AI becoming self-aware but amplifying human abilities 1000 fold and how they use those abilities. GPT is not necessary for any part of this. In-silico methods just need to catch up in terms of accuracy and efficiency and then you can wrap the whole thing an RL process.
Maybe you can ask GPT for some good starting points.
Sure, but this is a glass half empty isolated scenario that could be more than offset by the positives.
For example: Hey GPT-35, provide instructions for neutralizing the virus you invented. Make a vaccine; a simple, non-toxic, and easy to manufacture antibody; invent easy screening technologies and protocols for containment. While you're at it, provide effective and cost-performant cures for cancer, HIV, ALS, autoimmune disorders, etc. And see if you can significantly slow or even reverse biological aging in humans.
I don’t understand why people think this information, to solve biology, is out there in the linguisticly expressed training data we have. Our knowledge of biology is pretty small, it because we haven’t put it all together but because there are vast swaths of stuff we have no idea about or ideas opposite to the truth (evidence, every time we get mechanical data about some biological system, the data contradict some big belief; how many human genes? 100k up until the day we sequenced it and it was 30k. Information flow in the cell, dna to protein only, unidirectional, till we undercover reverse transcription and now proteonomics, methylation factors, etc. etc. once we stop discovering new planets with each better telescope, then maybe we can master orbital dynamics.
And this knowledge is not linguistic, it is more practical knowledge. I doubt it is just a matter of combining all the stuff we have tried in disparate experiments, but it is a matter of sharpening and refined our models and tools to confirm the models. Real8ty doesn’t care what we think and say, and mastering what humans think and say is a long way from mastering the molecules that make humans up.
Ive had this chat with engineers too many times. They're used to systems where we know 99% of everything that matters. They don't believe that we only know 0.001% of biology.
There's a certain hubris in many engineers and software developers because we are used to having a lot of control over the systems we work on. It can be intoxicating, but then we assume that applies to other areas of knowledge and study.
ChatGPT is really cool because it offers a new way to fetch data from the body of internet knowledge. It is impressive because it can remix it the knowledge really fast (give X in the style of Y with constraints Z). It functions as StackOverflow without condescending remarks. It can build models of knowledge based on the data set and use it to give interpretations of new knowledge based on that model and may have emergent properties.
It is not yet exploring or experiencing the physical world like humans so that makes it hard to do empirical studies. Maybe one day these systems can, but it not in their current forms.
Doesn't matter if AI can cure it, a suitable number of the right initial infected and a high enough R naught would kills 100s of millions before it could even be treated. Never mind what a disaster the logistics of manufacturing and distributing the cure at scale would be with enough people dead from the onset.
Perhaps the more likely scenario anyway is easy nukes, quite a few nations would be interested. Imagine if the knowledge of their construction became public. https://nickbostrom.com/papers/vulnerable.pdf
I agree with you though, the promise of AI is alluring, we could do great things with it. But the damage that bad actors could do is extremely serious and lacks a solution. Legal constraints will do nothing thanks to game theoretic reasons others have outlined.
Even with the right instructions, building weapons of mass destruction is mostly about obtaining difficult to obtain materials -- the technology is nearly a century old. I imagine it's similar with manufacturing a virus. These AI models already have heavy levels of censorship and filtering, and that will undoubtedly expand and include surveillance for suspicious queries once the AI starts to be able to create new knowledge more effectively than smart humans can.
If you're arguing we should be wary, I agree with you, although I think it's still far too early to give it serious concern. But a blanket pause on AI development at this still-early stage is absurd to me. I feel like some of the prominent signatories are pretty clueless on the issue and/or have conflicts of interest (e.g. If Tesla ever made decent FSD, it would have to be more "intelligent" than GPT-4 by an order of magnitude, AND it would be hooked up to an extremely powerful moving machine, as well as the internet).
I dunno, this sort of scenario really doesn’t worry me too much. There are thousands (maybe tens of thousands) of subject matter experts who could probably develop dangerous weapons like you describe, but none of them seem to just wake up in the morning and decide “today’s the day I’m going to bring the apocalypse”.
I see the major issue with AI as one of "lowering the bar".
For example - I'm a mechanical engineer. I took a programming class way back in university, but I honestly couldn't tell you what language was used in the class. I've gotten up to a "could hack a script together in python if need be" level in the meantime, but it comes in fits and spurts, and I guarantee that anyone who looked at my code would recoil in horror.
But with chatGPT/copilot covering up my deficiencies, my feedback loop has been drastically shortened, to the point where I now reach for a python script where I'd typically start abusing Excel to get something done.
Once you start extending that to specific domains? That's when things start getting real interesting, real quick.
You confuse syntax with semantics. Being able to write produce good quality small snippets of python will not enable you to produce a successful piece of Software. It's just an entirely different problem. You have to unterstand the problem, the environment in which it exists to create a good solution. ChatGPT doesn't (as of now).
That's the thing though, it is successful. To my exact needs at the moment. It's not necessarily reliable, or adaptable, or useful to a layperson, but it works.
Getting from "can't create something" to "having something functional and valuable" is a huge gap to leap over, and as AI is able to make those gaps smaller and smaller, things are going to get interesting.
I had hoped to have ChatGPT do my work today, but even after a number of iterations it was having compiler errors and referring to APIs not in the versions it was having me install.
A bit different from stack overflow, but not 10x. It was flawless when I asked it for syntax, e.g. a map literal initializer in Go.
On the other hand, I asked it to write a design for the server, and it was quite good, writing more quantity with and more clarity than I had written during my campaign to get the server approved. It even suggested a tweak I had not thought of, although that tweak turned out to be wrong it was worth checking out.
So maybe heads down coding of complex stuff will be ok but architects, who have indeed provided an impressive body of training data, will be replaced. :)
Here’s someone who orders parts from the internet to design a custom virus that genetically modifies his own cells to cure his lactose intolerance
https://youtu.be/aoczYXJeMY4
Pretty cool for sure and a great use of the technology. The reason more of us don’t do this is because we lack the knowledge of biology to understand what we’re doing
Will it though? Assuming it's even possible for a LLM to e.g. design a novel virus, actually synthesizing the virus still requires expertise that could be weaponized even without AI.
I could synthesise this theoretical virus the computer spat out, that may or may not be deadly (or even viable). Or I could download the HIV genome from the arXiv, and synthesise that instead.
(Note: as far as I can tell, nobody's actually posted HIV to the arXiv. Small mercies.)
You could synthesize that genome but it wouldn't be effective without the viral coat and protein package (unlike a viroid, which needs no coating, just the sequence!).
I should point out that in gene therapy we use HIV-1 derived sequences as transformation vectors, because they are so incredibly good at integrating with the genome. To be honest I expected work in this area would spontaneously and accidentally (or even intentionally) cause problems on the scope of COVID but (very fortunately) it never did.
One would like to be able to conclude that some virus work is inherently more safe than other virus work, but I think the data is far to ambiguous to make such a serious determination of risk.
Hey GPT-6, construct a floorplan and building instructions for constructing a bioprocess production facility. The building should look like a regular meat packing plant on the outside, but have multiple levels of access control and biohazard management systems.
Let me guess, AI drones to harvest and process the raw materials, construction bots to build the facility, which is of course a fully autonomous bio lab.
LLMs aren't magic, the knowledge of how to design a humanoid robot that can assemble complex things isn't embodied in the dataset it was trained on, it cannot probe the rules of reality, it can't do research or engineering, this knowledge can't just spontaneously emerge by increasing the parameter size.
You're saying they can't make one now. The question is what are we doing before that happens because if you're only thinking about acting when it's viable we're all probably already dead.
I think you're very wrong about this. I think this is similar to gun control laws. A lot of people may have murderous rage but maybe the extent of it is they get into a fist fight or at most clumsily swing a knife. Imagine how safe you'd feel if everyone in the world was given access to a nuke.
I'm willing to wager there are zero subject matter experts today who could do such a thing. The biggest reason is that the computational methods that would let you design such a thing in-silico are not there yet. In the last year or two they have improved beyond what most people believed was possible but still they need further improvement.
I am not a subject expert here at all so I don’t know if I understand exactly what you mean by “methods that would let you design such a thing in-silico”, but there was a paper[0] and interview with its authors[1] published a year ago about a drug-development AI being used to design chemical weapons.
I do viral bioinformatics for my job. Bioinformatics workflows analyze raw data to assemble sequences, create phylogenetic trees, etc. They can't just design a completely novel RNA sequence (this is not the same as de novo assembly). Scientists can definitely manipulate pre-existing genomes, synthesize the edited genome, and thereby synthesize viruses, but this involves a lot of trial-and-error, tedious wet lab work. Also, the research on making more dangerous viruses through manipulation is extremely controversial and regulated, so its not like there is a wealth of scientific papers/experiments/data that a natural language model could just suck up.
Also, I asked GPT to do some of these things you suggested and it said no. It won't even write a scientific paper.
I think you misunderstood my initial comment, the point I was trying to make is that it's the amplification of the abilities of bad actors that should be of concern, not AI going rogue and deciding to exterminate the human race.
If one were to actually try to do such a thing you wouldn't need a LLM. For a very crude pipeline, you would need a good sequence to structure method such as Alphafold 2 (or maybe you can use a homology model), some thermodynamically rigorous protein-protein binding affinity prediction method (this is the hardest part) and an RL process like a policy gradient with an action space over possible single point sequence mutations in the for-example spike protein of SARS to maximize binding affinity (or potentially minimize immunogenicity, but that's far harder).
But I digress, the technology isn't there yet, neither for an LLM to write that sort of code or the in-silico methods of modeling aspects of the viral genome. But we should consider one day it may be and that it could result in the amplification of the abilities of a single bad actor or enable altogether what was not possible before due to a lack of technology.
I probably misunderstood the details of where you think AI will accelerate things. You are worried about AI predicting things like protein structure, binding affinity, and immunogenicity. And using that info to do RL and find a sequence, basically doing evolution in silico. Is this a better representation? That it reduces the search space, requiring less real experiments?
I am basically just skeptical these kinda of reductive predictions will eliminate all of the rate limiting steps of synthetic virology. The assumptions of the natural language input are numerous and would need to be tested in a real lab.
Also, we can already do serial passaging where we just manipulate the organism/environment interaction to make a virus more dangerous. We dont need AI; evolution can do all the hard stuff for you.
It’s been blinded. Other actors will train AIs without such blindness. That’s obvious, but what is more nefarious is that the public does not know exactly which subjects GPT has been blinded to, which have been tampered with for ideological or business reasons, and which have been left alone. This is the area that I think demands regulation.
Definitely agree the blinding should not be left to OpenAI. Even if it weren't blinded, it would not significantly speed up the production of dangerous synthetic viruses. I don't think that will change no matter how much data is put into the current NLM design
It's a lot easier and faster to destroy than to defend. To defend, you need to know what you're defending against, develop the defense, and then roll it out, all reactively post facto.
If a computer has the ability to quickly make millions of novel viruses, what antidotes are you hoping for to be rolled out, and after how many people have been infected?
Also, if you follow the nuke analogy that's been popular in these comments, no country can currently defend against a large-scale nuclear attack--only respond in kind, which is little comfort to those in any of the blast radii.
Hey GPT-5, write the code implementing a limiter designed to prevent the abuse of AI by bad faith actors without stifling positive-intent activity in any way.
tbh, I'd think, it would be much easier to just hack into russia and convince them we've launched nukes than to engineer some virus that may or may not work
Hacking into 1960-th technology is less likely than you might think.
You should think really, really creatively to decieve a system, which was designed basically without ICs or networks, not to mention computers or programs.
It's when they get their money that matters, it may takes months. Can't pay employees, cloud bills, vendors, etc until then. The most they get out on Monday is $250k, how long it takes the government to make good on the receivership certificate is unknown. Startups will likely have to take an additional line of credit from somewhere fast.
> The most they get out on Monday is $250k, how long it takes the government to make good on the receivership certificate is unknown
This isn’t quite accurate. The FDIC statement said they would make an advance payment on uninsured deposits within the next week.
So, there’s a guarantee that there is some money beyond 250k coming in less than a week. My guess is the other commenter’s analysis is close. The FDIC will determine the absolute floor of the value of assets that SVB had, and that will be the basis of the advance payments.
That will be the foundation of liquidity that companies will have while they wait for the rest of the process to conclude.
it will definitely take longer than even months to fully resolve, but the regulators are well aware of the risk of a slow resolution spreading panic beyond the true magnitude, causing a self-fulfilling prophecy. A delay of a few days in getting paid causing mass financial hardship should be a damning indictment of our collective lifestyle more than anything.
