Hacker News new | past | comments | ask | show | jobs | submit | logicchains's comments login

Are there any fundamental differences between Mamba, Retnet and RWKV, or are they all variants of this same architecture?

You missed a key thing: before the 70s there was a limit on the amount of cash in the system, but after the US jettisoned all ties to gold there was no limit on how much money the financial system can create. That's why the US dollar has seen more inflation in the past 50 years than in the 150 years before that.

And all we have to show for it is a 1600% increase in GDP.

Number go up!

Money doesn’t buy happiness!

There would be nothing wrong if money wasnt created out of thin air to cover the GDP increase. It's basically the central bank stealing from people and claimimg that low inflation somehow helps. It doesnt. Its's a hidden tax on everyone, that eats GDP growth.

We (mostly) don't create money out of thin air. We borrow it. This obligation acts as a limit on money supply inflation.

Also, slight, but mostly consistent inflation is far better for everyone than what the US went through in it's first 150 years as a country.

The average American household is worth more than $1MM and the median is close to $200k. All time highs, even adjusting for all the inflation.

https://dqydj.com/net-worth-by-year/


Problem with inflation in every country is not that it needs to be above 0. It's actually much higher than they publicly say. With not giving you real numbers, they lie officially to get richer. Intentional

> We (mostly) don't create money out of thin air. We borrow it

This is the same thing - when you go to the bank to get a loan, the money is created out of thin air. It simply appears in a database. It is not the money someone has deposited in the bank. The only reason the bank has to keep reserves is for stability in case of bad loans.


> This is the same thing

It is strictly not. All money is borrowed. This seems like free/unlimited printing when the interest rate is 0%. But when the interest rate rises, this has real consequences and you can't print your way out of a 5.5% interest rate.

This is why US politics got heated about finance as of late. If inflation doesn't come down, the US will have to "print" that money. Essentially rendering the Fed to the likes of Pakistan, Turkey and Argentina.


Why would small inflation be good for anyone bar the centtal bank? Small inflation is ignored by everyone. If there was no artificial inflation (central bank racket) you could ignore it as well.

Inflation encourages investment because if you just sit on your money it gets eaten away by inflation. Once you’re Oprah-rich inflation is the rate at which you’re losing net worth. This is my non-economist understanding.

Inflation affects the value of currency, not assets. Oprah isn’t sitting on billions in cash, presumably. Those billions are invested in assets, which typically inflate at a rate similar to inflation. This is why one of the best hedges against inflation is owning things like real estate and equities. The value is separate from any currency you can denominate it in.

The benefit of mild inflation, to phrase what you wrote a little differently, is that it encourages people to create assets with substantial intrinsic value as a way to dispose of currency. Everyone wants to own as little currency as possible. There are many positive externalities to asset creation.


Assets tend to require continued investment and upkeep.

I do wonder if this conventional wisdom is actually true, though. Somewhat relatedly, there was once an idea that if you lower taxes for rich people, they'll invest more, and those tax savings will trickle down to the middle and lower classes and be a boon for them. But we know it doesn't actually work like that.

People still work with a low inflation (say 2,5%). You claim they would stop whrn inflation was 0%?

I disagree.

This 2,5% is just pocketed by yhe central bank.


Lawyers & economists contribute more to GDP than farmers...yet most lawyers & economists are a net negative in real terms while farmers provide food, essential for life.

If GDP was the end all be all of measurements, why is Russia winning the war despite having a fraction of GDP compared to its western opponents?

Sure seems like the wrong things are being measured...and rewarded.


GDP was a good measure for a while (or reasonably good enough). This ceased to be the case in the last 10-15 years; and I think it can be said now that it has become completely useless.

There are the edge cases (heavily financed/resourced countries) like Ireland, Singapore, Qatar, etc... and then you have Western countries "manipulating" the numbers. For example, your rental payment goes into the GDP; even if you are not renting -__-. So if you are sitting on your ass doing nothing in your own home that because of the financial bubble is now worth $2.5 million and $10k of rent, you are adding $10k/month to the GDP of said country.


GDP is a good measure of how the wealthy are doing, not so much your average citizen.

Well sure, but in Russia’s case, the average income is incredibly low and their wealth disparity is even more extreme than in the USA. I don’t know how any of that relates to the war point

> Why is Russia winning the war despite having a fraction of GDP compared to its western opponents?

This was a huge eye-opener for me, the fact that we are spending 20x more on the military industrial complex and yet it cannot supply enough shells and bullets for the war, to me, indicates that all of our posturing is a bad joke.

Either there is huge waste and abuse in the system, or we've literally set Ukraine up to lose.


Throwing money and war materiel at Ukraine doesn't ensure victory. Clearly the lack of it will ensure failure. But weapons are mostly only as good as the training of the people wielding them.

On top of that, there's fear of nuclear weapons taking the stage. If the US military went head to head with the Russian military, and we didn't have to worry about MAD, I think it's reasonable to assume that the US would win. But is the Ukrainian military, supplied with Western military supplies (and even Western military advisors), a better fighting force than the Russian military, even with a home-field advantage? Unclear.

And on the other hand... when the war started, a lot of people assumed Russia would achieve their objectives within a few weeks, or at most a month or two. Ukraine has turned out to be much more capable than initially expected.

> ...and yet it cannot supply enough shells and bullets for the war

It absolutely can, but politics, as usual, gets in the way. And, as I said, shells and bullets are necessary, but not sufficient.


The object was to break Ukraine's toys. Mission accomplished. Putin attacked Ukraine to prevent being eclipsed by it.

Zelenski became, rich, powerful AND popular much faster than Putin. He was insanely jealous. Consumed. Ukraine was on a path to eliminate corruption and join NATO and then easily become far more wealthy than Russia. It would have consumed Russia by the will of the people on both sides of the border.

Zelenski created his own media empire before he was thirty years old. Then he created a new political party and it beat all the others. He is preferred by the people. He accomplished more in a quarter of his life than Putin could in all of his and was going to help Ukraine achieve prosperity. Putin just wanted to stop that. Period.


That's some hallucinated nonsense.

Putin gave an "explicit warning that Russia perceived NATO's eastward expansion as a threat to its national security" as early as 2007 in his Munich speech [0]. The Russo-Ukrainian war began in 2014 with the ouster of Yanukovych, who was pro-Russia and opposed closer ties with the EU [1]. Zelensky became president in 2019 and initially promised to end the Russo-Ukranian war, but instead continued to pursue NATO protection and in August 2021 "urged NATO members to speed up Ukraine's request for membership" [2]. Russia escalated the Russo-Ukrainian war and invaded Ukraine for a second time in February 2022 [3].

[0] https://en.wikipedia.org/wiki/2007_Munich_speech_of_Vladimir...

[1] https://en.wikipedia.org/wiki/Russo-Ukrainian_War

[2] https://en.wikipedia.org/wiki/Volodymyr_Zelenskyy

[3] https://en.wikipedia.org/wiki/Russian_invasion_of_Ukraine


I see that you bought into it.

And vastly exceeded the carrying capacity of the planet, leading to ecological and likely societal collapse. Thanks, capitalism!

GDP ... denominated in fiat currency

Inflation adjusted the US GDP grew from $5.30 trillion in 1970 to $22.67 trillion in 2023.

Which is about a 4x increase in real terms.

https://www.multpl.com/us-gdp-inflation-adjusted/table/by-ye...


why would you adjust for inflation, what we are talking about would mean adjusting for M2.

Why would you adjust for M2? Inflation exaggerates GDP growth. M2 doesn't.

If you measure a country and M2 and GDP go up together, that's important to know, but you don't "adjust for M2" and pretend that GDP didn't actually go up.


Hmm, let me ask you this. It we double the money supply because say, banks are allowed to loan more to each other, but that money doesn’t make it to the average consumer, does CPI go up and does GDP go up?

We're not changing anything else with businesses? I wouldn't expect those numbers to change very much then. Certainly far less than 2x.

ok, maybe too abstract an example. If a bank makes a loan (creating new money) to loan to some rich guy, who decides to pay 1B to buy a bunch of Microsoft stock, that counts as growing the GDP, even if the majority of the money just stays in the financial sector and never drives up the price of anything measured by CPI.

GDP measures goods and services, not stock purchases. A few trillion dollars of stocks bouncing back and forth don't change it at all.

Why has the US seen so little inflation in the last 30 years?

It saw tremendous inflation, just less than the 70s and 80s, and this due to two different things.

First, companies just began making shitty products. Use glued and compressed pulp instead of wood, use fructose instead of glucose, use bleached and desiccated wheat, use chips instead of anything mechanical, use cheaper less pure and thinner metals, etc. in the realm of food there was also shrinkflation.

Second, the USA exported much of its inflation as dollar imperialism.

Today, making products of less quality is nearly impossible without sacrificing sales, and the USA has run out of countries to add to its empire while simultaneously having sanctioned enough countries that they’ve now formed their own trading bloque. All of this means that dollars are staying at home after printing and that dollars are starting to come home from the rest of the world. This follows after the government shutdown parts of the economy and concomitantly printed trillions. So, there’s now rather high inflation once again.

Deregulation actually played less important a part of all of the financialization of the economy than did the transition to debt-based economics. The inversion of time in money is a dance with the devil, and the devil usually wins that dance.


Wonder if anyone's written anything about the inevitability of the decline of profit.

They have, extensively, but it's not a story the Jedi would tell you.

Can you please share some authors or sources from the Empire that I can read?

https://en.wikipedia.org/wiki/Global_saving_glut

I may have exaggerated slightly, but it's a touchy subject because investment is the legitimate way by which rich people get paid for being rich, which is how class works in capitalist societies. The long term sustainability of investment supply / demand dynamics have deep implications for the legitimacy of this enterprise, its winners and losers, and the policies that prop them up (or don't).


Like ~100% inflation since 1994 sounds bad until you look at the 30 years prior to that which was ~380% inflation.

It also doesn’t sound bad if one calculates that 2.4% annual inflation would exceed 100% over 30 years, and one considers that below about 2% is often harmful to the economy.

Because everyone wants to bank/invest with the US. The money is being pushed in, not pulled in. If it were being pulled in, rates would have to be relatively high to pull it in, but they are not, so it is being pushed in.

Why is money being pushed in? Put yourself in the shoes of a wealth-weighted non-US citizen. Wealth weighting is critical for understanding many aspects of economics yet systematically under-discussed to make the economy seem more altruistic than it is, but that's another story. So you have a bunch of money and you're outside the US. You want to preserve or grow your wealth, but your local government keeps devaluing the currency to pump the real economy, or hiking taxes, or there's a labor party that you think might get in power and do those things, or your government is literally called the Chinese Communist Party and is ideologically predisposed to confiscating your wealth tomorrow. Where do you put your savings? Some of it you still keep locally, for practical reasons, but the rest you invest abroad. Where? A peer country with the same issues? A developing country where investment never comes back? A small banking haven that will fold at the first sign of hard power? Or do you send it to the rabidly capitalist country with a 50 year track record of delivering investor returns and dozens of aircraft carriers and nuclear submarines to put between your bank account and the New Workers Collective of Yourlandia? That's right. You send it to the USA.

What happens when the money lands in the USA? It pumps assets, because that's what the money buys, and it dumps exports, which now must compete with a literal money printer for access to talent and resources. Asset holders do great. People who sell services to the asset holders do OK. People who build shit for export, though, oof. Software, chip designs, and movies might be lucrative enough that they don't have to care (thus far) but in general, oof.

"We used to make shit in this country, build shit. Now all we do is put our hand in the next guy's pocket." - The Wire

That's exactly right, but it's extremely important to understand the macro that drives this so that you can understand how the macro affects you and make informed guesses about where it is headed. For example, I wouldn't bet on this dynamic collapsing tomorrow.


That's right. Because the productivity of the capitalist system has absolutely zero reason to be tethered to one or 2 elements on the periodic table that is mostly for jewelry.

>Are you aware that millions of people were murdered during the actual cultural revolution

Are you aware that the cultural revolution didn't start with this? No successful movement starts with "let's go murder a bunch of our fellow countrymen"; it gradually builds up to it.


Are you aware that we don't live under a Maoist dictatorship or any system of government even slightly reminiscent of what the cultural revolution developed within?

https://heterodoxacademy.org/blog/coddling-of-the-american-m...

Historically, students had consistently opposed administrative calls for campus censorship, yet recently Lukianoff was encountering more demands for campus censorship, from the students.


The same authoritarian spirit is alive and well in the American left. Remember when 45% of Democrats supported putting the unvaxed in camps, and 29% supported taking their kids away?[0]

How many more supported such measures, but had the sense to lie about it?

[0] Here's the poll. Search for 'designated facilities' and 'remove parents’ custody': https://www.rasmussenreports.com/public_content/politics/par...


>How is it someone who is so out of touch with the end user in position to make these decisions?

Maybe it's the same team behind Tensorflow? Google tends to like taking the "we know better than users" approach to the design of their software libraries, maybe that's finally leaked into their AI product design.


Their social agenda leaks into their search and advertising products constantly. I first noticed a major bias like 8 years ago. It was probably biased even before that in ways I was oblivious to.

Proponents of KYC also never backed their policies with empirical evidence of their effectiveness; the onus should be on them to prove that KYC is effective enough to justify the inconveniences and privacy violations it entails.

It’s hard to prove a law that has a deterrent effect, since you can see the number of people it “caught” but the people it deterred are invisible.

The best proxy for the deterrent effect is how much criminals need to pay to launder money. I can’t find a good source on this, but some anectdata that put it between 15-50%.


I mean, sure, but if you're demanding detailed statistics from the anti-side (the start of this thread), letting the pro-side gesture vaguely in the direction of hard to measure deterrence seems like an isolated demand for rigor.

>The article goes on to immediately compare KYC with widespread facial recognition. Why are these equivalent

They're both violations of people's privacy.

>There's no evidence provided for these numbers

And KYC proponents provide no empirical evidence that KYC has actually led to a measurable reduction in organised crime. The onus should be on them to demonstrate why it's effective enough to justify the violation of citizens' privacy that it entails.


I'm not defending KYC here (tho I think it is necessary personally). I'm just requesting the author actually try to make an argument.

So is a license plate. Privacy is not an absolute right.

> Privacy is not an absolute right.

This is meaningless.


You mean to say "nothing is an absolute right". Yes. If you want me to be more precise, I'll say that privacy is a lesser right, one that is important but that is nonetheless frequently and deliberately compromised in the service of other rights.

You get to weird places if you try to wrap an entire worldview around privacy the way people do expression or self-determination, because privacy does not have that standing.


Sure, but the fact that KYC is not motivated by data is not a license just to pull numbers out of your ass.

There's zero evidence that AML laws have actually led to any reduction in crime, while there's countless evidence of them causing unnecessary difficulty to law-abiding citizens. AML proponents should be required to demonstrate that the measures they propose actually lead to an empirically measurable reduction in crime, otherwise it's just a massive violation of citizens' financial privacy rights for no gain.

How about we set up a parallel banking system with none of the rules and regulations, and see what happens?

Oh yeah we did that a few years ago and it was scams all the way down :)


There are also an abundance of people getting scammed the old fashioned way, KYC's done nothing to reduce that as the amount of people losing money to online scams has increased in recent years.

The crypto ecosystem has scams but it also has genuine financial privacy with Monero, while the average person in a western country has zero options for financial privacy in the traditional banking system.


What do you mean by “financial privacy”? Hiding stuff from the taxman?

Yes, that too, but also just fucking privacy. A basic notion that your financial movements shouldn't be an open book to any bureaucrat, functionary, corporate drone or algorithm under the sun. We rightfully take personal privacy seriously in many other things, but suddenly you try to apply the logic to something as deeply sensitive as one's economic life and many would-be nannies lose their shit. Many of you on this very comment thread sound like government agents ranting about "protecting children" with their attempts at encryption backdoors.

The only difference is that you apply it to preventing people from controlling the keys to how and where they keep or send their money. Why? Because "terrorists!" "Money Laundering!", Tax evasion!" Oh my! How many of these justifications wouldn't even exist if government wasn't so heavy handed in slathering on endless regulations to criminalize whole areas of human activity that never needed to be harmful or criminal in the first place?

I'll also note here that in many countries, employees at major banks and at major government offices collude with organized crime to a pervasive degree, and actively look at the accounts of people with signs of having come into large amounts of money (but without corresponding protective power of their own) for the sake of extorting or even kidnapping them. I live in one such country. It's easy to bleat about the need for KYC and making sure everyone "pays their fair share" (a thing that anyhow barely functions as claimed even in the most KYC-obsessed countries, since the wealthy use all kinds of legal loopholes and special connections to keep their funds safe) if you live in a place where the powers that be aren't overtly parasitical and criminal. It's a little more difficult to apply the same in other parts of the world.


Between 2016 and 2021 the number of money laundering cases doubled [1].

So clearly AML laws are making a difference at detection at least.

[1] https://www.eurojust.europa.eu/news/money-laundering-cases-r...


Some random team at Berkeley beat them to it by a couple days: https://largeworldmodel.github.io/ . It's just a matter of throwing compute at it, nothing fancy. OpenAI could probably do 1M token context too but they haven't yet found a way to make it profitable (neither have Google; the most they actually offer customers is 256k).

This brings a very interesting question: if you could have an AI software engineer today but it would cost you 1 trillion dollars, would you want and be able to afford it?

There is a reason why we still have people working at McDonald's even though fully automating it has been possible for a couple of decades now.


It's the same reason that for 80 years after the invention of the commercial icemaker in 1842, the American ice-harvesting industry produced more frozen water than manufacturing plants. And the ice trade did not exist until 1806.

https://en.wikipedia.org/wiki/Ice_trade

It was more economical to send people out to cut ice from a lake in Maine and ship it by rail to Chicago than it was to just freeze water from a local supply. It was also more reliable since the technology was mature, versus ice plants that often broke down when meatpackers needed a consistent supply.

There's no reason why this won't be the case for AI unless semiconductor manufacturing continues its exponential performance/cost growth. The demand for technologically obsolete goods and services do not instantly disappear when a superior product enters the market.

Human software engineers right now are more reliable than AIs for most price-points. This is true for most industries in which machine learning is present.


> cost you 1 trillion dollars

How did you come up with this number? It seems pretty unrealistic.

> There is a reason why we still have people working at McDonald's even though fully automating it has been possible for a couple of decades now.

Maybe the low salary is the reason? If it is a bit more costly to automate certain aspects of manual labor, then the low salaries might remove the incentive to do so. This is not the case for software engineering.


Beyond the snark, this is basically it. It's the same reason the Roman Empire, despite all its technological prowess, never tried hard to automate relatively low-hanging-fruit tasks: because slaves were cheap, plentiful, and more flexible ("reprogrammable") than anything mechanical could ever be.

If it costs $1m p/y to run a machine that cooks burgers and fries, or $30k for an employee who can do that _and_ cover something else when someone else is ill, it's a no-brainer. But businesses had to discover that the hard way; until the 80s, most people were still convinced automation would win everywhere, because it had won (and won big) in manufacturing. A combination of factors, from the '80s onwards, made labor costs effectively fall, which created our reality where certain jobs are so cheap that automating them makes no sense.

The "problem" is that, in certain regions, software development costs reached a point where automation looks very, very appealing. If a machine costs 500k p/y to replace a few 150k p/y SWEs without all those pesky employment complications, businesses will happily choose "AWS AI CloudDeveloper"...


> If a machine costs 500k p/y

Do you mean an AI programmer would cost $500k per year? If so I think you greatly overestimate the cost.

Recently I did some text processing with GPT-4 turbo (128k context) and I reached the daily limit of 5 million tokens. IIRC it cost me around $70 bucks for the day.

I think $70 is the hourly rate of a SE with $150k salary working 40 hours per week. Note that we are at early stages with this tech, it will probably only get cheaper from here.


"IIRC it cost me around $70 bucks for the day."

Sure, for you that was the price. Enterprise cost would be way different.

"Note that we are at early stages with this tech, it will probably only get cheaper from here."

Haha people who pay for these ai tools can only hope...Ask any cloud provider, streaming service, or utility company if their prices are cheaper now than before.

As these ai tools get better, they will require more resources to run (according to altman's 7 trillion dollar request) and most likely drive up the costs.

But hopefully you are right though, as i believe we as humanity would be best served spending as little money and resources as possible on AI.


> If so I think you greatly overestimate the cost.

I suspect you underestimate it. Raw engine cost is one thing; what businesses downstream will actually pay, is another. Look at AWS: a lot of businesses don't even touch it directly, their vendor ISPs do. If "AIDev" really becomes a thing, businesses will buy specialized services (e.g. "ApiBuilder.io", "YAMLCrusher.io", etc etc), which will obviously command a premium on top of top-tier, 5-9s guaranteed, "raw" ml engines.


Aw look, hn is reinventing capitalism from first principles!

Fwiw my question was rethorical.

And it was meant to highlight that even if you have the tech (which we don't - the cheap tricks chatgpt or copilot do are impressive but still cheap tricks - are super expensive when it comes to actually training the models) it may not make economic sense to deploy them.

Even if it makes sense to deploy them the social unrest and volatility that will result in society may not end up well. (What's the point if all the consumers go away or they cannot actually buy the shit you're producing)


GitHub CoPilot charges $10 for a subscription that it loses an average (!) of $20 a user on.

https://www.theregister.com/2023/10/11/github_ai_copilot_mic...

"Make it profitable" appears a secondary concern in the AI space.


I started using it recently. $30, 50, or even $100 a month is litterly nothing for most companies in wester world. They'll hike up the prices eventually.

I believe that is where they are implying they do it without increasing memory utilization dramatically.

If 1M context uses 32x the memory of 32k, its a non-starter. Even a smallish LLM like Mixtral uses 4-8gb of memory just for your prompt. You would have 256+GiB at 1M...


> It's just a matter of throwing compute at it, nothing fancy.

I read somewhere that there was a recent breakthrough that enabled this.

Even if it costs a lot to run inference with 1M token context, it is hard to imagine it would cost anywhere close to a software engineer salary.


It's not just Trump thinks; under NATO those countries are obligated to spend a certain % of their GDP on defense, which they've failed to do.

The full context is he said if those European countries did not meet their NATO defense spending obligations he'd let Putin do what he wants with them.

Not just "let", he explicitly said "encourage".

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: