Cheers for the suggestion, I personally read the book, attended the concert [1], and learnt more about capitalist resource extraction attitudes from meeting the Vogons [2] when working on mapping [3] than I did from Pynchon's filtered Rathenau [4].
It's baffling to me why all the biggest fabs are built in high risk potential conflict zones. South Korea, Taiwan and Israel. It's terrible capital management if we're going to use those words.
It's a bit of a cliche, but as I live in Brazil I'd say it's mostly true that people living happily at the beach aren't desperate for economical and technical development, and mostly to enjoy daily life and nature. When you're under threats or crisis, technical development seems like a necessity for survival. (There are however, many technical poles in scenic locations in Brazil, like Florianopolis-SC and Ceara).
That said, we should be somewhat worried about desperation-fueled endeavours.
People do crank up a few gears when it's war time or generally threaten of their survivals. After all the original engineering was military engineering then after the rest of engineering or engineering for common causes like building a bridge for civilians are called civil engineering.
For example, Fourier Transform was there but no one use it because it is too slow and there was no urgency to use it until the nuclear testings were common place. In order to detect underground nuclear testing there was an urgency to detect the events using Fourier Transform and voila Fast Fourier Transform (FFT) was invented to speed up it up considerably without loosing the accuracy. Then we have Fast Inverse Fourier Transform and the rest is history.
Another great example was the need for robust command and control (C & C) during the Cold War. US military was looking for C & C infrastructure that can survive nuclear catastrophe and voila the Internet was invented. People keep denying this fact but since it's initially and fully funded by DARPA, it's kind of giving away the main motivations. The basic tenet of the Internet design is essentially to have a highly reliable communication in the face of nearly total or 99% infrastructure destruction. If I can recall correctly someone actually simulated the 99% infrastructure down scenario and the results US Internet is still working fine, and if someone know this report or publication please let me know the original source.
The Internet is very funny. It's both incredibly reliable with insane amount of redundancy and extremely easy to screw up. A single bad BGP config can shut down the internet for a whole country ...
well I doubt that if there a nuclear war the internet would safe because its run on electricity, power plant and grid require extensive maintenance and this is all hard even on normal day to day basis
It's a well known fact that after original Internet becoming very popular US military has their very own separated, isolated and operated military network based on the similar design of packet switching of the Internet for their own use cases. Since it's operated by military they can make the networks infrastructure military grade and make the power supply to be fully redundant, but then Cold War was turned into just propaganda machine so perhaps there is no real motivation to make it really reliable.
It's also a very common practices in many countries to have isolated and reliable nationwide network infrastructure for emergency usages for example disaster management purposes including military operations.
Power generation is relatively disributed today, with solar and wind. Big telecom infrastructure usually has backup power. And there's power-over-fiber as an alternative power grid for telecom(altough not sure how often used).
Taiwan is more scenic and safer than probably most of Brazil. The quality of life is probably only matched by some European cities with extra safety everywhere. Most Taïwanaise I have talked to are not that concerned with the prospect of war. Maybe some capitalists out there but most people are unaware of the geopolitics.
I'm glad that war is not really considered a large risk. I know some people from Taiwan, does seem like quite good quality of life! I didn't mention, but I think there are other important factors like the cultural importance given to (formal) education in those places (that might be broadly connected with strong will to develop economically?).
Safer I can totally agree with you but more scenic... I will absolutely completely disagree, just because Brazil is massive (larger than the continental US) and with a very varied geography. Taiwanese cities are more scenic but as a country I highly contest the comparison.
Israel is in a regional cold war with Iran. While Iran is certainly not a global super, it is a serious military force (which could plausibly become a nuclear power).
Further, resource constrained guerrilla/terrorist groups love valuable targets like a $25B factory that provides a critical input for a significant portion of the global economy.
If they were well-armed by the West, they'd be called a military.
> love valuable targets like a $25B factory
Until 7 Oct, Israel had proven more than capable of protecting its assets (at a cost). Since 7 Oct, it looks like they seek to do that once again (at a cost). I don't think megacorps are worried that much given none of the G7 really seems to care about those costs.
You should look into the recent wars in the middle east and earlier conflicts in South East Asia. You're highly underestimating the effectiveness of resource constrained insurgency.
There's actually an argument to be made that we wouldn't even be having this conversation unless the Viet Cong existed. The Internet and modern distributed networks are a direct consequence of western intervention in SEA and the creation of counter-insurgency.
You and others are making the same mistakes as the USSR and others have when countering Islamic Insurgency.The USSR's underestimation of Afghan fighters was their downfall, it's why they called Afghanistan the graveyard of empire, and I would not be surprised at all If we see Isreal meet a similar fate because of this exact same ignorance. In fact I'd bet my entire net worth that is the outcome based on what I see currently. The insane confidence, as displayed by your comment is a great summary of this type of ignorance.
Insurgencies have been squashed in recent past (without resorting to nuclear weapons). Search for the end of Tamil tigers in Sri Lanka, Russia crushing Islamists in Chechnya, China crushing Islamists in Xinxiang (or however it's spelled), China crushing the Tibetan insurgency, and currently, Israel almost crushing Hamas.
You cant beat an insurgency without local support. The Tamil Tigers are the most obvious case, but all the other ones you mentioned involved settling the disputed territory. Israel is nowhere close to squashing Hamas
The comment is likely being downvoted because it doesn’t seem like a response to its parent comment. The parent was saying that Israel is not facing a well-resourced military but a guerrilla group (Hamas). Its opponents don’t have things like nukes. The parent comment was not saying that Israel doesn’t have nukes. On the other hand, Taiwan is threatened by China (a nuclear power) and South Korea is threatened by North Korea (also a nuclear power).
Taiwan and Korea built capable semiconductor fab ecosystems because of their countries’ economic development position, not despite their precarious political position.
People need to understand fabs are both skilled labor and automation intensive. Apart from that there are supplier ecosystems that only exist in those regions. Korea and Taiwan needed to have globalized export economies. Their education standards and culture are a good fit for scale manufacturing, both still had conscription military service. The labor at the time 80-90’s and currently is still highly competitive for their skilled labor force.
EU semiconductor is not a joke but the tools and wafer size are not state of art. I know EUV machine is a product of EU, but that moonshot tool is something that can only come out of a long dev government subsidized program like CERN and the bosson. They aren’t exporting that tool into the EU too it’s going right to Asia.
The EU is full of high living standards and well educated people but the semiconductor production is mainly serving local consumption (or local companies supply chains, automotive parts). EU does research but that doesn’t translate to at scale fabs.
EU gets protective and wants to secure local supply for their own tool vendors and needs to understand that smaller local machine brands cannot compete with the majors for the run of mill machines. EUV is a different story that thing is anomaly and power to them for following up on it.
Also strong labor organizations don’t attract capital. And semiconductor cannot take the month of August off.
US has capital and plenty of cheap land and cheap labor but still more expensive than Asia standards. Depending on state non existent labor protections and competitive environment permits. Not great education standards. Excellent skilled immigration. But still not leading in semi production especially with the old guard just keep losing. Still strong #2 though. And the ecosystem is nowhere near as integrated as Seoul. Like would you leave your hame family and friends in Oregon to startup a next gen Fab in Phoenix? No way. Geography isn’t doing US favors there for dynamism.
South Korea and Taiwan are hell of a lot more stable than they were decades ago.
Taiwan has transitioned from an authoritarian regime under martial law (lifted in 1987) to a vibrant democracy. Regular, free elections have become a norm, and political power transitions smoothly.
Taiwan's economy has grown significantly, becoming one of the "Four Asian Tigers" with a strong technology sector. Companies like TSMC (Taiwan Semiconductor Manufacturing Company) are global leaders.
Taiwan has seen advancements in social issues, such as becoming the first country in Asia to legalize same-sex marriage in 2019.
South Korea has also moved from a period of military dictatorship to a stable, democratic society with regular and peaceful elections.
South Korea has transformed from a war-torn country in the 1950s to one of the world's largest economies, excelling in technology, automotive, and other industries. It is home to major global brands like Samsung, Hyundai, and LG.
South Korea has made significant strides in education, healthcare, and quality of life. Its cultural exports, like K-pop and Korean dramas, have gained international popularity, enhancing its soft power.
How do their goverments being democratic have anything to do with their potential for invasion? Please go back in time and tell half of Europe this argument during WW2. This has nothing to do with the stability of their govt or society. It has a to do with their belligerent neighbors (and their own belligerent policies).
It feels like a cop-out to type this, but I think the unsatisfying-yet-obvious explanation is that capital follows capital, and at some point there was a compelling reason to put capital in all of those now-unstable places. Somebody at some time needed to build up industry in those places.
You say that but honestly given the salary gaps that we're seeing in US vs the rest of the world, somebody is one day going to make a good arbitrage play with "made in EU" prestige + a pretty understood process + low euro salaries + American disposable income.
South America has been promising for decades but it just keeps not fulfilling the expectations. I don't fully understand why, but I think it's not very business friendly (mostly controlled by left wing populists).
Meh, fabs in South Korea and Taiwan are funded and operated by domestic corporations. There's no international body assigning capitals according to a grand master plan, it's just South Korean and Taiwanese corporations' capital (which they attracted by being successful).
Your complaint is basically that some countries in potential conflict zones ended up being economically successful.
The intellectual capital in this countries just happens to be off the charts. It is probably, to a degree, related to the existential jeopardy those countries face.
I have never found a convincing explanation for this either. My hypothesis is that these are easier to control politically and do their "master's bidding" when needed (i.e. Geo-political reasons). Others who might turn out to be strategic Geo-political rivals are simply sidelined.
To me it looks like that the free world and its NIMBYism tends to shift physical fabrication jobs out towards its edges. I've heard that Silicon Valley actually made tons of silicons half a century ago. Japan made rather high-performance CPUs as recent as until 2000s-2010s. Now there are lots more in South Korea, Malaysia/Singapore, East coast Mainland China, and of course Taiwan. Maybe in a decade or two we'll hear about Tibet or Nepalese fabs expanding into Afghanistan or Iran, or maybe something different altogether might happen before that.
(btw, Intel actually has a lot of production capacity in the US[1]. The US don't need Taiwan and SK intact for bare minimum survival in an unlikely event of a global thermonuclear war)
In all those cases you probably had a lot of intellectuals driven there basically by communist or nazi militaries. Then if you're an Einstein type dropped in a refugee zone why not start a tech business? Or encourage the kids to.
The US is not majority rule, assume you're talking about the US. Between Gerrymandering and the First Past the Post system, it's really not that uncommon for minority rule. The easiest example of that being the 2016 Presidential election, where Trump lost the popular vote by just over 2%.
How many smart people run marathons, do all night drink binging sessions, travel to exotic cities and other such follies?
My belief is that a project like this (and probably the marathon and travel too!) make you smarter. You need to do something that ain’t work sometimes!
Ignoring most of your comment and only focusing on this part:
> the world needs smart people like you solving real problems
It has them. And they’re doing pretty much exactly this. What’s described in the OP resembles a pretty standard machine design process – the key is splitting up the complex goal into smaller manageable units. (And using tools for the tedious parts such as gear math; if the tools don’t exist, you of course develop your own!)
The author was presumably able to do this because they’ve already solved a real problem or two.
And the obvious degradation of the body's capabilities and its potential failure points (potentially increased by traveling 300mph at 10,000 ft). Commenter is right, 90 year olds shouldn't be driving cars let alone flying. Incredibly irresponsible. Also there's really nothing that cool or heroic about dedicating your life to testing and operating flying death machines on behalf of a military. Get a grip.
If you follow your argument to a logical conclusion it would require that we ban millions of people with mild impairments from operating machines. That is the worst kind of ageism, discrimination, and ableism.
Imagine that you studied and went through pilot training and got your license. Do you really believe that when you start solo flying that you, an inexperienced pilot, would be a safer pilot than someone like Bill Anders at 90? Experience, guile, and cunning count for far more than you can imagine.
When my mother was getting older I looked at age related accident data. I apologize that I don't have the reference handy, but it was quite striking that teenagers are far more likely to have an accident than an elderly driver. I guess we should ban teenagers from driving also? Experience matters a great deal.
How about if we find that recent immigrants are more likely to have an accident due to lack of familiarity with local driving regs? According to your logic we should make a law banning immigrants from driving. And on and on...
As far as your comments about military service, even if we are anti-war (who isn't?), it is still possible to admire the hero who is willing to put their life on the line to defend their community.
Get outta here with this ageist crap. I've know several 90+ year olds who are fully capable of independence and driving. My grandma was one of them. Hell, I watched a 93 year old deadlift 450lbs. While the likelihood of some 90 year old being too old for something is high, that means less when looking at an individual.
And 10 to 12 year olds can drive too. I wonder why governments of developed countries ban them from driving outright. Most places don’t even let 14 to 15 year olds drive.
There is a difference between having something and then losing it vs not having it yet. If you get a license to do a thing you should be able to keep doing it until you are deemed incapable and that will vary between people.
Also, plenty of children those ages drive on private property, especially farms. There are kids doing backflip jumps on dirt bikes at those ages. Many of them are likely safer to be driving than others who are of "prime" age. I drove with a 30yo who terrified me with his dangerous lack of skill.
People are variable in capacity and skill. The bureaucracy finds it more manageable to put policy in place than to determine individual skill.
>There is a difference between having something and then losing it vs not having it yet.
I disagree. The reason for the policy would be the same.
Probability of person of age x causing collision is too high.
>The bureaucracy finds it more manageable to put policy in place than to determine individual skill.
Yes, of course. Testing every single person all the time can get costly, and it may or may not be deemed worthwhile by a society (or whatever government leader). Obviously, when people are young, their faculties are getting better, so testing once is not unreasonable.
But at advanced ages, faculties can degrade, and degrade at varying rates. For this scenario specifically, maybe it is not onerous to sufficiently test 90+ year olds that want to fly, since there are so few.
However, since an airplane crash in an urban area has a high likelihood of causing damage to others, society does have an interest in controlling who is in the pilot's seat.
Ageism is not inherently wrong. This also deals with a public space, where idealism and liberties are already restricted. If you make individual exceptions based on close examination, then fine. But I think it's fair for observers to assume that this individual flying was a mistake.
Using the same logic you might say that people shouldn't be allowed to drive until they are 24. Statistically far more lives would have been saved yesterday if we'd only simply restrict more liberties based on age alone.
Worth discussing! You might also suggest that driving tests should be harder/longer/repeated, specialized based on different types of driving, and license privileges restricted based on years/miles safely driven. We already do it to truck drivers. Remember, most societies have already crossed over into treating driving on public infrastructure as a privilege, not a right/liberty.
What tasks do you consider a 3.8B model to be useful for? Chat applications on lesser hardware, im still finding it difficult to parse what the real world application would ever be. However, I do understand that the goal is to make the smallest most efficient model to compete with the larger model capabilities one day and you can't get there without making these. But do these types of models have any value for any sort of product or real world project?
I think most of the interesting applications for these small models are in the form of developer-driven automations, not chat interfaces.
A common example that keeps popping up is a voice recorder app that can provide not just a transcription of the recording (which you don't need an LLM for), but also a summary of the transcription, including key topics, key findings, and action items that were discussed in a meeting. With speaker diarization (assigning portions of the transcript to different speakers automatically), it's even possible to use an LLM to assign names to each of the speakers in the transcript, if they ever identified themselves in the meeting, and then the LLM could take that and also know who is supposed to be handling each action item, if that was discussed in the meeting. That's just scratching the surface of what should be possible using small LLMs (or SLMs, as Microsoft likes to call them).
An on-device LLM could summarize notifications if you have a lot of catching up to do, or it could create a title for a note automatically once you finish typing the note, or it could be used to automatically suggest tags/categories for notes. That LLM could be used to provide "completions", like if the user is writing a list of things in a note, the user could click a button to have that LLM generate several more items following the same theme. That LLM can be used to suggest contextually-relevant quick replies for conversations. In a tightly-integrated system, you could imagine receiving a work phone call, and that LLM could automatically summarize your recent interactions with that person (across sms, email, calendar, and slack/teams) for you on the call screen, which could remind you why they're calling you.
LLMs can also be used for data extraction, where they can be given unstructured text, and fill in a data structure with the desired values. As an example, one could imagine browsing a job posting... the browser could use an LLM to detect that the primary purpose of this webpage is a job posting, and then it could pass the text of the page through the LLM and ask the LLM to fill in common values like the job title, company name, salary range, and job requirements, and then the browser could offer a condensed interface with this information, as well as the option to save this information (along with the URL to the job posting) to your "job search" board with one click.
Now, it might be a little much to ask a browser to have special cases for just job postings, when there are so many similar things a user might want to save for later, so you could even let the user define new "boards" where they describe to a (hopefully larger) LLM the purpose of the board and the kinds of information you're looking for, and it would generate the search parameters and data extraction tasks that a smaller LLM would then do in the background as you browse, letting the browser present that information when it is available so that you can choose whether to save it to your board. The larger LLM could still potentially be on-device, but a more powerful LLM that occupies most of the RAM and processing on your device is something you'd only want to use for a foreground task, not eating up resources in the background.
LLMs are interesting because they make it possible to do things that traditional programming could not do in any practical sense. If something can be done without an LLM, then absolutely... do that. LLMs are very computationally intensive, and their accuracy is more like a human than a computer. There are plenty of drawbacks to LLMs, if you have another valid option.
Ive been sorta following together.ai for a while. Cool company. Is this available to be used by anyone atm? Could I potentially use the model to look at my own chest xrays (I've had a lot)?
It's a great description of how the Capatalist Cartel think about the extraction of Earth's resources.
reply