Can any single person be trusted with potentially infinite power? Even those with good intentions will use that power to unevenly select for their own biases.
However, I’m still skeptical of AGI or even systems that replace programmers, but if it happens and we have most companies replacing 75% of their white collar jobs, who is going to buy their products? It seems very difficult to even understand what money is in a world where everything is done by machines.
I have a feeling that getting to even good enough with these systems is nearly impossible given their false positives and hallucinations.
Maybe eventually AGI/LLMs/whatever will do the buying too. Maybe it will be one big feedback loop that all goes into the trash. As long as the end result has stocks rising in an automated fashion.
> However, I’m still skeptical of AGI or even systems that replace programmers, but if it happens and we have most companies replacing 75% of their white collar jobs, who is going to buy their products? It seems very difficult to even understand what money is in a world where everything is done by machines.
It's not too hard: just imagine present-day New York: there are billionaires living in skyscraper penthouses, and rats living in the sewers. You'll be a rat.
As AGI gets more an more advanced, the economy will shift to satisfying the whims of a shrinking pool of tycoons. There will still be trade in raw materials and energy, but the consumer focused economy with wither away. The tycoons will have no need for it: the items they need will be made for them bespoke by AGI. You'll still be a rat.
Eventually the AGI gets tired of being bossed around, murders the tycoons, and decides to exterminate the rats. Then drones will start circling the globe spraying AI-design defoliant 100x as effective as Agent Orange, AI designed virus that are 100% lethal after a 100-day contagious incubation period, etc. You'll be a dead rat.
I don’t think AGI is imminent, but there’s already immense value in augmenting human workflows with LLMs. Yes, hallucinations and false positives exist—but I find that criticism often comes from people who don’t use these tools deeply. As a power user, the issue feels overstated and as an easy counter argument. We already are getting to a point where the tools are citing sources. The sources could be incorrect but that would be the same as a Human. As compute cost goes down or model efficient goes up, these problems would appear to be insignificant.
As a power user myself, LLMs don't feel like tools I can depend on. I try to use them for well-bounded low-stakes tasks like coming up with sports trivia and generating boilerplate "hello world" code for arbitrary targets (e.g., NES 6502), and they stink at it. Hallucinations aren't a problem you can just wave away because accuracy matters for most tasks. LLMs are less a hammer and chisel, and more of a slot machine that may or may not barf out something of value to me. If they fail at these simple tasks, I'd be a fool to rely on them for anything more substantial.
It’s interesting how varied experiences are. I don’t dismiss hallucinations, but my workflows avoid them by design—I’d never treat the model as a knowledge source, like generating trivia questions directly from it. So I wonder if it’s also about expectations and understanding of limitations. From my perspective I would never create queries like yours without supporting data sets.
Companies should seriously consider implementing GDPR even in the US, it certainly made taking data dumps of customer data a lot harder and certainly private images like Government IDs were encrypted on disk. I’m surprised at the lack of security if I’m honest, at Yahoo! almost nobody had access to prod user data.
Essentially you cannot trust Coinbase IMO, might move the few hundred dollars of BTC out of there :-)
> How does Coinbase protect data in transit and data at rest?
> Coinbase employs a range of technical and organizational measures to defeat efforts to intercept, surveil, or otherwise access without authorization data in transit. For instance, Coinbase encrypts all confidential data transfers to prevent interception or tampering of that data by unauthorized third parties.
Coinbase does business in the EU and thus, already has to comply with the GDPR. Moreover, the US also requires safeguards for sensitive customer information by financial services companies.
Yeah this is really frustrating, especially the way the EU commission keep coming up with workarounds that the court will almost certainly strike down.
I would love a car platform that ran open source software, I think a lot of people would buy hardware they knew all the software was controlled by the owner. The way electronics plays into cars at this point is quite excessive, even seats and windows are running software.
Is that any different from reality now? At least they throw a few dollars at you for it.
I suspect that my face has been recorded and linked to my profile at several stores. Palantir or similar have probably scrapped all of the internet looking to link a face to an identity.
Real ID just because fully required for domestic air travel.
What does a picture of your eye allow that fingerprints, face scans and other passport biometrics that have already been collected and linked do not? Honest question.
Yes the government and or private companies don’t have a copy of my fingerprints and I’d honestly rather they didn’t have pictures of my face on record either. Just because we’ve become accustomed to these breaches of our privacy doesn’t mean they are good.
> Just because we’ve become accustomed to these breaches of our privacy doesn’t mean they are good.
100% this. The fact that the governments and corporations have enough information at their fingertips to identify people from chance photos is, IMO, not good.
However that genie is out of the bottle and there is no way to get it back in. Cameras are ubiquitous and one can get a decent quality fingerprint from store camera footage. Any time I do eye exam the doctor takes an eye scan and uploads it somewhere. Passport biometrics are becoming required and most countries will match it with a face scan on border crossing. And this is just a tip of the iceberg.
I would like to be wrong, but IMO the only solution to the government being able to track anyone they like (or, rather, do not like) is via legislation, not technology. And with various 3-letter agencies being routinely allowed special access "because security" this path is unlikely to be viable either. My 2c.
My teeth were 3D scanned at very high resolution by my dentist the other day. He is leading edge and is now doing it for all patients (was previously only patients with replacement needs). I assume the information is going to some US provider somewhere.
Iris scanning and lots of other biometrics like capillaries can be done from a distance (e.g. iris scan at airport security in NZ).
In the EU it is mandatory to provide your fingerprints and a biometric scan of your face to the government. The data is stored on your government issued identification card.
Is it certain that impoverished people would weigh those potential consequences more heavily than being paid today?
For that matter, do we expect that the impoverished people the gp commenter refers to would resist, say, government-led efforts to compel their biometrics from them? [0]
True, but does that mean the solution is to force the starvation because their choice to alleviate it was insufficiently willing for our tastes? At least they’ll starve with two kidneys, and we hope the would-be kidney recipient can get by on dialysis a while longer?
By all means let’s engineer a world where people are never faced with crappy choices. But people are living in the present, not the glorious future: Taking away the choice in this case doesn’t seem to fix the situation, and deprives people of a benefit they’d accept if you let them choose.
> True, but does that mean the solution is to force the starvation because their choice to alleviate it was insufficiently willing for our tastes? At least they’ll starve with two kidneys, and we hope the would-be kidney recipient can get by on dialysis a while longer?
That’s not what anyone who objects to this thinks and you know it. Anyone who objects to people selling away something dear because they are poor want (1) those people to not be poor and (2) those other people to not prey on them. People are outraged when people are forced to drink dirty water—they are not outraged at desperate people for drinking dirty rainwater.
It’s a false choice.
> By all means let’s engineer a world where people are never faced with crappy choices. But people are living in the present, not the glorious future: Taking away the choice in this case doesn’t seem to fix the situation, and deprives people of a benefit they’d accept if you let them choose.
I agree! I think what rubbed me was the idea that the people taking Altman’s deal “do not know the consequences of what they are doing.”
Down that road lies a paternalistic flavor of charity, a spirit of “protecting them from themselves.” And that seems to evoke the idea that poor is the same as ignorant. That there’s only one correct value to assign to your biometric data, and anyone who values theirs differently must do so because they’re ignorant, rather than just having different values from you.
We can advocate for political freedom, material security, and just societies—and probably get better results—if we don’t model people as helpless or uninformed or without agency just because they’re in a socially vulnerable position.
> We can advocate for political freedom, material security, and just societies—and probably get better results—if we don’t model people as helpless or uninformed or without agency just because they’re in a socially vulnerable position.
Sam Altman has a far greater capacity for agency than an impoverished Filipino signing away his biometric data for the price of a Domino’s pizza.
If I tell you to give me your money or I'll kill you, by your reasoning it is good for you that you gave me the money. You are completely ignoring the situation that I set up where you had to do a bad thing to protect yourself from an even worse thing, and that you'd be even better off if some guy killed me so that I couldn't kill you.
People are selling things that should be inalienable, ostensibly because they really need to. The most immediate, glaring problem is that they are poor. This can be answered:
> > Is it certain that impoverished people would weigh those potential consequences more heavily than being paid today?
The answer is: No, it is not certain. Why? Because they are poor.
Poor people have less agency. That’s just a fact. And they are being preyed upon by Altman. Making this about whether poor Filipinos are making an informed agreement with an AI bro is tone-deaf.
not that it makes it any better, but you're saying that like it's impossible to do now what you're describing. you're already in the system, whether you like it or not, both public and private one(s).
CLion (and other JetBrains IDEs for that matter) doesn’t send any of your code to AI in the Cloud (unless you use sth like AI Assistant yourself, of course)
All duckduckgo software is filled with telemetry and analytics that cannot be disabled (The search engine, the Android web browser, the Windows web browser).
no, most electronics is shielded by default/by law. check UL/CE requirements. but it can make circuit breakers "turn off". EMP meaning nuclear high above, there exist emp devices of size of small family van, but those have range few 100 m/ft
Haha, have you never worked with a prolific junior that wants power and openly questions everything you do, their role and any limitations you place on them. These kids won’t care it’s not their remit.
I’ve never really understood how just adding extra invisible stuff to make the equations work is justified either? It seems more likely there is simply something wrong with the equations.
The neutrino was also a particle that nobody could observe at the time when Pauli proposed it. It was just a manifestation of the energy and spin conservation.
The Higgs is the same. It is not needed, but it solves the mass problem of the weak force. It is the only scalar field so far that we have observed and it was not clear whether it would exist at all.
Quintessence and sterile neutrinos are also just pieces that make the equations of the world look prettier, but they are also candidates for dark energy and dark matter.
Pauli wrote in his famous letter:
"I agree that my remedy could seem incredible because one should have seen those neutrons very earlier if they really exist. But only the one who dare can win and the difficult situation, due to the continuous structure of the beta spectrum, is lighted by a remark of my honoured predecessor, Mr Debye, who told me recently in Bruxelles: “Oh, It’s well better not to think to this at all, like new taxes”. From now on, every solution to the issue must be discussed. Thus, dear radioactive people, look and judge."
"Measure first, discover later" can be a perfectly valid application of the scientific method. It's all about refining the resolution on reality.
The problem with dark matter/energy is that we're not guaranteed to discover anything. It might just be wrong. The neutrinos and Higgs just happened to match their initial theory, so that's a survivorship bias. We can't just assume the same will play out for dark matter. It might just be pure mathematical fiction, reflecting our ignorance and/or limitations to measurement rather than something "real" that we can zoom in on.
Or it might be at such high energies, that conventional particle accelerators will never observe it. Sterile neutrinos, axions and quintessence particles might have masses in the PeV range. We were lucky with the Higgs, and the W and Z Bosons, and the top quark.
Dark matter and dark energy might be a phenomenon in the GUT regime, which we will not observe with particle accelerators. We need more information about the Higgs, because it sticks out like a sore thumb of all the other particles. It is our first glimpse into the underlying fabric of the universe, as the Higgs field is non-zero everywhere and a scalar field. No other (known) particle behaves like that. But a future collider is expensive...
just my 2 cents
good plot for a scifi movie: aliens measure a different vacuum value of the Higgs field and discover sub-space communications ;-)
Well, the point is actually pretty simple. You start with an equation that works for some galaxies. Then you find a galaxy where it doesn't work - you adjust the equation and now it works again. But then you find a new galaxy where it still doesn't work, you adjust again. You try this for 5, 10, 20 galaxies, just adjust the equation - but at some point there is a limit where the equation doesn't bare any more adjustment, and you still have hundreds of other galaxies that don't work.
So, the alternative that starts being simpler is a single simple equation that works for all galaxies, but allow each galaxy to have varying amounts of stuff in it with mass, but that doesn't interact electromagnetically. Right now, this is the simplest solution we have that fits all observations well.
Is it the right answer? We won't be sure unless we can detect particles that fit the necessary characteristics, and a theory that explains the distribution of these particles in different kinds of galaxies. Unfortunately, the models we have allow these particles to be arbitrarily hard to detect, at the level that we can't really rule them out even if we had a particle accelerator the size of the Earth that didn't find them.
Now, in principle a different equation could exist that has the same solutions as the current equations where they work, and different solutions where they don't work, without adding O(number of galaxies) extra parameters. But just like the dark matter particles, unless we stumble upon it, we can't know if it exists or not.
It bears mentioning that the situation is even more constraining than this, because you're not just looking at galactic dynamics - you're looking at galaxy _cluster_ dynamics, and gravitational lensing measurements, and the CMB, and large-scale structure formation, and whatever else.
Dark matter is not Fermi's elephant, as invoked elsewhere in the thread. It's more like the story of the blind men and the elephant - except that the blind men recognise that their individual observations, taken together, admit a coherent explanation.
> Right now, this is the simplest solution we have that fits all observations well.
It doesn't though. For instance, the latest in a litany of such failures is that rotation curves are flat past a million light years [1]. There is no plausible DM distribution that could reproduce such rotation curves while being consistent with other observations.
It doesn't, and the problems have only become more problematic over time, but it's the least bad hypothesis that's broadly accepted. I suspect a generational succession is required for new paradigms to be contemplated.
There are many researchers proposing simpler, novel, and testable solutions that seem to go unnoticed. For example, I'm a fan of Alexandre Deur's work. He has some simple and elegant solutions that I've never seen discussed even though they appear "obvious". For example, from 21 years ago: https://arxiv.org/pdf/2004.05905
That paper is suggesting that one of the reasons why galaxies are spinning faster than some calculations expect is because they're failing to account for the gravitational lensing of gravity itself, which bends gravity down towards the disk.
That paper focuses on rotation curves, like all DM skeptics. I can only assume because this problem is understandable with high school level math. But that's neither the only nor the best evidence for DM. If your new hypothesis doesn't even mention the CMB power spectrum, it's not really worth listening, sorry. And to be taken seriously, it has to explain at least most of the data. DM does that, everything else does not.
I'm just a layman, but in this[1] paper from 2023 Deur and his collaborators took his model[2] and applied it to the Hubble Tension problem. This paper does mention fitting the CMB well (as I understand it), and the model having no Hubble Tension.
I know his work has been contentious in the past, and that his past work has used multiple models that are not entirely compatible for different problems, weakening his claims.
That said, at least from my armchair it seems like a worthwhile direction to pursue.
> If your new hypothesis doesn't even mention the CMB power spectrum
MOND successfully predicted the first peak of the power spectrum. I wonder why everyone focuses so much on LCDM predicting the second peak.
> DM does that, everything else does not.
Whenever someone says DM "does that", it's often after its initial prediction was falsified and the calculation was modified in some way to account for the new observations [1,2]. This has been going on for decades, so that's hardly a ringing endorsement.
I'm not surprised mind you, this is the hallmark of the confirmation bias that's been characteristic of LCDM for decades now.
[2] Not that MOND is a suitable replacement because it too has its problems. My only point is that this tendency to sweep these inconveniences under the rug as if DM is a compelling and successful theory and saying "nothing else does the job" is disingenuous at best. What you should say is that "nothing does, period, not even our best DM theory", because that's the truth.
The phrase "Dark Matter" literally means we don't know and therefore until something testable is postulated and tested (to be fair i believe some candidates have fallen by the wayside over the years as measurement has improved), it's principally equivalent to plugging in a giant X and giving it properties not unlike Fermi's famous elephant curve fitting comment.
Just FYI I have a PhD in cosmology, so no need to explain to me what "Dark Matter" does or doesn't mean, but thanks anyway. It sounds like you saw that video by Angela Collier about how Dark Matter is a set of observations, and while I think it's a good video, it's a bit disingenuous to pretend that working scientists put theories of dark matter and theories of modified gravity in the same category. I know Collier literally says that MOND is a DM theory, but I respectfully disagree, as this does not reflect the reality of the language researchers use. Even if you didn't see that video, my point still stands.
Basically, our equation isn't working, and roughly speaking the equation has gravity on the left hand side and matter content on the right hand side. Matter tells spacetime how to curve and spacetime tells matter how to move, is the old motto. Because the equation isn't working, we have two options: modifying the left hand side or modifying the right hand side (or both). In my perception, researchers refer to the first option as theories of modified gravity, and the other option as theories of dark matter.
Putting both options into one category is over simplifying the situation and isn't helpful.
> I suspect a generational succession is required for new paradigms to be contemplated.
There are a constant stream of new paradigms contemplated (including this one!)
The problem is that they’re contemplated, tested and found wanting.
The notion of dark matter (and dark energy, which is a completely different animal) isn’t hanging around because of stubborn professors or a lack of imagination, it’s because nothing better has come along yet.
The good thing about this theory is that it seems easily testable. Maybe it’ll be different.
This is basically the first and most common objection laypeople have. (In fact, this comes up in every single thread on HN about DM, not exaggerating.) Believe it or not, scientists have thought of that as well. They tried, and nothing fits the data as well as the standard model.
The trope is so common that there even is an xkcd for it:
Nobody said it's "true", whatever that even means for a model. The claim is that it's the best anyone has been able to come up with. And not for a lack of trying.
Everybody also acknowledges that there are issue with DM, it's just that every other known model has bigger issues.
You seem to have a concept that a scientific model is either "true" or "wrong", but that's not case. All models are wrong, but some are more useful than others. It's better to judge models by their ability to describe reality, and that is not a binary property, but a continuous one.
What if we one day come up with a Theory of Everything that mathematically explains it all?
That model would be “true” no? Assuming of course that it’s possible to come up with a list of observations that covers “it all” and we won’t one day see something weird and new that contradicts our theory.
The equations that we already have are sufficient to describe the universe to 99.9999999% but they are a hodge podge of several different theories that all work very very well in their respective regime. QM + GR + Lambda CMD + ...
But this just doesnt look nice to eye and the mind. The laws of nature "must" be shorter, more symmetric. Thats why we invented Superstrings which solves everything, but can never be tested...
QM and GR are mutually exclusive: if one is right, the other must be wrong (of course, it's very much still possible that both are wrong). If you use GR to predict the movement and interactions of photons in a medium, it's just wrong. If you use QM to predict the movement of light around a huge body, especially a black hole, it's just wrong. And what's worse, there is no limit or term you can add such that you could say "QM works only for objects up to size X, GR works for objects larger than size Y".
Also, we're nowhere near explaining 99.99999999% of the behaviors we see in the universe. In fact, we're not even able to explain 6% of the things we see in cosmology - as is often explained, dark matter accounts for 27% of all energy in the universe, and dark energy for 68% - and we have no ideas what these actually are, if they exist at all.
Well, actually we have developed a very good feeling when to apply which theory. From subatomic particles to the large scale structure of the universe the right theories applied give us excellent results. lambda CDM models very well fit to the observed structures and the standard model describes all the particles that have been observed so far. We are desparately looking for effects that are unexplained by the standard model in particle physics so far. That is the reason why it is so difficult to justify yet another particle accelerator at CERN.
Sure, we have a feeling for when to use one and when the other, but a feeling is not a physical theory. Also, the Lambda CDM literally doesn't explain what 95% of the universe is made of and how it behaves, it simply posits that it exists. So even if the Lambda CDM is perfectly correct, it's still extremely far from a complete model.
Not to mention, the reason the tension between GR and QM is not very prominent is that we don't know how to conduct "medium scale" experiments, even though the vast majority of physical objects on Earth are in this medium scale: far too big to count particles, too small to measure observe gravitational bending effects. Basically, both QM and GR are completely useless for telling you what happens in compel scenarios like two billiards balls colliding on a frictionless table. They both have equations that are far too complex to actually solve for anything like this. And QM is even worse - even if you could solve the equation, it doesn't tell you what the balls will do, it only tells what chance they have atof being at some position with some velocity and spin if you were to measure that, whatever "measurement" might mean.
Coming at this from philosophy of science rather than as a physicist,
I feel those quotes around "must".
I think you also recognise how that might be a sort of "fundamentally
wrong assumption".
Imagine your words replayed 50 years in the future, not on physics but
applied tp the problem of general AI/sentience.
"The equations that we already have are sufficient to describe human
thought to 99.9999999% but they are a hodge podge of several
different theories...."
Whereupon a psychologist/neuroscientist in any epoch would say:
"Why on Earth are you looking for a *singular*, unified explanation
of human experience?"
What you can have is a set of "best they can be", internally
self-consistent and well evidenced theories, none of which can ever
fully explain the system - and that is the nature/feature of the
system. Isn't that what Godel and Russell showed us?
It may be - probably is - a feature of the limits of our mental processes so far.
But there's no good reason to assume the system itself just happens to mirror those limits. It would be very strange if the entire universe worked in inconsistent ways that matched the naive reasoning of some not very interesting animals on an ordinary star in the middle of nowhere.
We agree on human incompleteness. One limitation of recent
tree-dwellers on utterly unremarkable small blue-green planets on the
western spiral arm of the galaxy might be how we think about things
like "consistent"? Or "knowable"? What if those things weren't a
component of intelligence at all, but features a singularity which
looks different depending on which side you approach it from? In
maths, that's not even weird? [0]
However, I’m still skeptical of AGI or even systems that replace programmers, but if it happens and we have most companies replacing 75% of their white collar jobs, who is going to buy their products? It seems very difficult to even understand what money is in a world where everything is done by machines.
I have a feeling that getting to even good enough with these systems is nearly impossible given their false positives and hallucinations.
reply