ChatGPT can't access the external internet. The example ChatGPT instruction where they "teach it" just has the user giving it a URL to their docs and ChatGPT is like "oh ok got it" (Figure 3). I'm not sure if the blog post writers are aware it's hallucinating what the JSON implementation is and hasn't actually read the documentation at all.
There's this internet phenomenon (common on Reddit and other such forums) where you find high-quality discussions on a variety of topics and everyone seems so clever and informative, until you come across a thread on a topic that you are actually an expert on and find that every top-voted comment is objectively incorrect. ChatGPT is essentially that but on steroids. Its primary ability is being able to sound extremely confident in all its responses.
I think a lot of overly-eager and under-informed people are jumping on the bandwagon and shouting half-baked proclamations from the rooftops to get clicks and views.
But underneath all the bullshit is something truly useful, so I wouldn't necessarily say that Confidently Incorrect™ is ChatGPT's primary ability.
Coincidentally, over the past few weeks of this ChatGPT craze, I've needed to create a lot of fake data to seed a database. Normally not a big deal, but these records need to have foreign keys pointing to one another. I wondered whether ChatGPT could help me out, so I briefly described the fields I needed, their format and data type, the type of information they contained, etc etc. It did it almost perfectly, and fixing the little errors was trivial. I was dreading having to do this because it's such a pain in the ass.
To me that kind of thing is going to be the most useful application. Everyone's freaking out (in both the "scared" and "excited" sense) over AI's ability to replace creativity, but I'm focused on it's ability to replace toil.
While I agree with your premise, the "toil" that you mention is a very small use case for a tiny minority of people. And automating it away has always been trivial. What would have taken 30 minutes to write a Python script now takes 5 minutes to come up with the perfect prompt. And then you have to test and be sure that it does what you want. Neat, but not world-changing.
Meanwhile ChatGPT is already being hailed as the Google killer. People are using it to write essays and form opinions. Plenty of online and offline debates contains phrases from ChatGPT repeated verbatim. It's being adopted in newsrooms around the world. So what it is good for in theory and what people are actually using it for are worlds apart. IMO the technology is going to have its "self driving car kills pedestrian" moment very soon because of over-eager and careless users, and that is going to set the entire field back.
I disagree. Toil is part of everyone’s job and what creates procrastinators. I used it recently to help me write a browser extension, where I find writing vanilla js and xpath expressions very tedious (just a bunch of syntax lookup) and would not have even bothered in the first place.
If you approach it with the right mindset, it’s extremely valuable.
Increasing software development speed 6x would be fairly world changing already, wouldn't it?
Add in all the automation of customer service via actually good Chatbots and lots of very cheap (and fairly high quality) copywriting and it seems world changing to me.
Given it has probably "hardcoded" a lot of questions by the usage of "finetuning" via RL, probably the latter statement is true. Also it has one way only to understand your query. If that way is wrong: welcome synonyms replaced by the tokenizer, welcome hallucinations by raising the temperature. Or you can introduce "context" (8000 tokens) or retrain.
In what sense? Google has many revenue streams -- the year is 2023, not 1997. Just HAVING the number one search engine without ad revenues brings mind share, intelligence on trends, what people are searching for, advanced notice on possible disasters...the list is endless.
When I want a quick insight to launch me into further inquiry, chatgpt has been so much more useful than regular search with the first 10 hits of AI generated advertisement spam. I use one that provide its source as well, so I can check.
Reddit tends to be a second best bet, and the most upvoted is not always the best, on a busy thread some ‘real’ experts are usually a bit down with lengthy descriptions of their analysis.
Search is third, it’s quick but I do often need to append stackoverflow and reddit to get meaningful answers. I forgo Google completely more and more since the quality is so low, perhaps 2% of my queries go there tops.
But those are the low hanging fruit that help me avoid rusting skills… and this is a common phenomenon on automation: you destroy the first rungs on the experience ladder, that no one wants to do but are essential to develop foundational skills.
That's a pretty good point. I'm looking at this from the perspective of having done this for over a decade. All that stuff feels like a waste of time to me now. Even as I was using ChatGPT for my fake data, I was thinking to myself "this is like having a junior dev assisting me". So yeah, those little micro tasks that build muscle memory are going to vanish.
That's almost every reddit thread I've been involved in.
Oratory prowess is way more important than accuracy. If you ever want to "lose a debate" on reddit, pick something that's counter-intuitive or widely misunderstood and use nuance and citations in your defense. You'll get cyberbullied every time - sometimes even banned from the sub.
From those examples, I think you’re maybe also running up against people generally having a low threshold for giving a stranger on the internet the benefit of the doubt.
It’s hard to be surprised that “eugenics, but the good kind, just hear me out!” and “state’s rights, but not the disingenuous lost cause kind, just hear me out!” are a tough sell with complete strangers who have no prior reason to trust you or be interested in your opinions.
Btw, I removed them from my examples, didn't want to "discuss" that here.
I'm kind of a history nerd and it's like a fishing hook with delicious bait when I see stuff like that. "Oh wait! You think it was just the south that argued states rights?! Actually (brings up fugitive slave clause jurisprudence)"
Or how manner and conduct books of the 1800s, eugenics texts, and jordan peterson's 12 rules for life are basically dots in a straight line. That stuff is just catnip.
I just need to stop being distracted and go write actual books or something.
Peterson an academic who is clueless about the real world. His best work was discussing politics in connection with liberty, especially the Trucker protests.
His biggest flaw is that (like the other Daily Wire pundits) he never goes into economics and has absurd ideas that we live in a competence hierarchy.
The whole "intellectual dark web" imploded after the start of the Ukraine war, when they first went full neocon, then realized that their audience does not like that and now start backpedaling massively.
There were multiple points of interest for the Eugenicists. One of them was for the general public and how to get them to have a proscribed conduct and behavior.
Again, this is part of Eugenics. We focus on the part that was removed (race science) and leave mostly unexamined the parts that were reinvented or repurposed as say, "The Power of Positive Thinking".
It is a dangerous psuedoscientific epistemology - a practice with a bunch of faulty logic and ways of knowing that can lead to disasters and it's alive and well - for instance, assuming a social structure in lobsters applies to humans. That these adherents tend to be revealed as secretly racist and thus subscribe to the part which was removed, should be of no shock.
We've mischaracterized Eugenics[1] and allowed it to fester by other names or as Maurice Bardèche, the french neo-fascist observed about fascism in the 60s, "With another name, another face, and with nothing which betrays the projection from the past, with the form of a child we do not recognize and the head of a young Medusa, the Order of Sparta will be reborn"
The point is that if you don't draw the perimeter right around an idea, you can't reign it in. It'll just escape from your hands and re-emerge as something else. So, for instance, the slave plantation becomes the prison farm (https://en.wikipedia.org/wiki/Prison_farm).
We tend to, for good reason, ignore societal trainwrecks and blights on our past. But trainwrecks are the most important thing to study if we strive to build safer trains and seek to avoid similar disasters in the future.
Things have to be accurately tackled with in order to defeat them.
---
[1] it's worth noting it's a big topic and some "real" science has also come out of it, such as biosocial theory and biostatistics studies. Peer reviewed journals such as https://en.wikipedia.org/wiki/Journal_of_Heredity used to be eugenics journals. This is, however, a very small minority.
Manners and conduct existed before eugenics and continues to exist after eugenics. You can't pull one blighted moment in the 1920s as proof the whole idea is faulty.
There is little accuracy in your description of events. Peterson said we share serotonin with lobsters and it fuels a victory/defeat set of behaviours. If you drop the serotonin from his idea, it's no longer what he was talking about.
Your description falls to my mind as a velvet blanket to hide underneath from scary racist and nazi ideas that have nothing to do with Victorian era manners and metaphors about behaviour, expanded out of natural science.
I don't mean anything toward you, by my comment. We obviously come from different traditions, and I can't really ever find agreement with how your argument is structured. My conclusions could be wrong, but I won't ever really know without a lot more reading of your thought, which we're both not going to give time to do. I should be more careful how much meaning I load onto the word "you" in comments, I only meant to criticize your written beliefs.
This doesn't make sense to me. But isn't there a difference between senior engineer and jail? Are you saying we should have no input on how children grow up or society behaves because some eugenics people picked the idea up once? Why must we be afraid of _everything_ the eugenicists _ever said_. Can't people sometimes have even a single good idea? I'm not in favor of eugenics at all. But social conduct and manners? That sociology, social psychology, we should grow away anything from any field that has ever been touched by _bad people_?
When i was younger the things Jordan Peterson said actually helped me. This whole "Reverse Midas Touch" thing is getting really out of hand. Just because a person has had bad ideas, doesn't make the person bad or require destruction of every idea they've ever held or spoken. Just because a group of racist and ignorant people held bad beliefs, doesn't mean every possible belief they held was bad and must be destroyed. This whole argument is just, "but the bad people thought that so it must be bad too", without ever establishing what's actually bad about it. Tell me what's wrong with the ideas without mentioning a group of bad people, tell me about why the ideas are bad. Tell me about the people who "manners and proper behavior" has hurt, and then explain to me why it can't possibly be different. Speak about the ideas, not the people. Everyone knows why those people were bad. Now tell me about the ideas
I post on both Reddit and YC News regularly, and it is fascinating to see the differences in biases, counter-arguments, or outright rejections of reality.
YC News people generally seem to prefer to hear the truth, especially from sources that appear knowledgeable. I've had people here comment in reply to me saying I've "changed their mind". That never happens on Reddit.
Reddit has cliques, quasi-religions, and fads. Anything contrary to the group-think is mercilessly voted down. Intelligent, thought-out counter arguments that aren't entertaining are especially likely to garner lots of downvotes. Not only do the readers feel "personally attacked" by contrary opinions, there is also that ever present undercurrent of anti-intellectualism of typical Americans that makes intelligent discourse distasteful to them.
It’s amazing to me in all my years on the internet how many communities I insert myself into, then in private feel like a pretender, and yet how relatively uncommon it is that people call me an idiot.
Good afternoon, fellow Imposter Syndrome Haver :) Just jumping in to say that you are most likely not a pretender or an idiot, you're probably just very self-conscious, hold yourself to high standards and give yourself lofty goals.
I’ve been quizzing ChatGPT about it’s knowledge of the oil industry - it has no idea what’s talking about.
For example, I ask it to simulate a discussion between a flow engineer and a geologist, and it doesn’t seem to understand they work together regularly.
Not like there’a a stack overflow for this sort of thing.
you need to prompt it with more specific knowledge; if you're generic in your prompts it will be generic in its responses. inject some field-specific terminology and be esoteric in your queries, etc
its latent representation is vast, I find you can get much higher quality+interesting responses if the prompts are tuned appropriately
This is at best going to slightly improve the output. I asked about something hyper specific and also extremely trivial: transit wayfinding type queries specific to the NYC subway. All the semantics are services and stations. A service is nothing more than a list of stations it stops at and a station is nothing more than a list of services which stop at it. Eg if I tell it "I'm on the F, how do I get to Time Sq", it should cross reference the service/station lists, find the commonality, and tell me "switch to the A at W 4th".
No context about anything in the outside world or even about the physical nature and purpose of commuting is needed to comprehend and answer these questions. There is no external context or sophisticated logic beyond just matching those sorts of lists to each other. Its a wholly closed system of highly standardized tokens.
I ask it what stops the F and A have in common. 125th St is on the list. I know that's wrong. I ask it to list all the services that go to 125th. The F is not on the list (correctly so). I point out the inconsistency between the two outputs. It says sorry and does nothing. I tell it to remake the list of stops the F and A have in common. It is now missing a few stations from before, and has added new incorrect results.
This is just a snippet. I went in circles with it for probably an hour playing wack a mole with its inability to correctly recall more than 10 true details in a row. These were not obtuse, esoteric, or even logically complicated queries. Nor was it ambiguous stuff open to interpretation. Nor was it even something that you'd need a real meatspace body to comprehend, like the feeling of the sun on a summer day. This should have been a language models bread and butter.
You're right, it seems particularly bad in navigation. I lead with some general queries on the NYC system, asked it for the common stops for service lines F and A, and it also hallucinated. The wiki for the service lines have pretty complete representations here, so this data should be suitably represented in its training corpus.
Would be interested in how a multimodal LLM such as PaLM-E (trained on maps, etc) fair in these sorts of queries.
I'm not hopeful that "seeing the map" (multimodal training) will make the difference everyone is hoping for. The transit map is going to be the exact same information as the lists, but coded in the learned design language of colored lines and dots. The words-only version should work just as well or better because it shortcuts the implicit OCR problem of trying to make it learn off the map. Indeed transit maps other than NY are often abstracted and have nothing to do with the underlying geography. So abstract representations (such as lists of words) should be fit for purpose.
Here's another one that fails spectacularly. The digits 0-9 drawn as an ASCII 7 segment display. It gets it mostly correct, but it throws in a few erroneous non-numbers and repeated/disordered/forgotten numbers. Asking it for ASCII drawings of simple objects can really go off the rails quickly.
The fault mode is very consistent. When a prompt forces it to be specific and accurate on an unambiguous topic for 10 or more line items, it will virtually always hallucinate at least one or two. Especially if the topic is too simple to hide behind a complex answer. Even if its learned not to hallucinate 90% of the time, and even if that's good enough to pass at first glance, within a list of 10 things it only has a 35% chance of not hallucinating any of them.
For what its worth, it did very well on law questions. Try as I might, it refused to accept that there's a legal category known as "praiseworthy homicide". Though, I suspect this has less to do with the underlying model and more to do with openAI paying special attention to profitable classes of queries.
I'm sorry to say, I think the problem maybe more intrinsic to the current approach to AI. Frankly, LLM works unreasonably beyond my expectations, but its making up for a lack of a real first principles theory of cognition with absurd amounts of parameters and training.
Here’s the thing though, when you keep having to “prompt it”, are you really doing the work yourself by coaxing a good enough bullshit answer that you’ll believe it? It could still be useful, not sure.
A minimum sort of knowledge of the field would be helpful.
But I feel that if one is inquisitive enough, they'd know when exploring a topic to ask broad questions of the sort:
- what are the common types of problems encountered in X field
- what are the main roles and responsibilities of personnel in X field
- list some open questions in the field of X
After these open-ended questions one can dive into each of them with:
- what are some common solutions to X, compare and contrast X vs Y
The risk of seeing hallucinated info is always there, but just for an example when I was exploring the types of space-time metrics and asking chatgpt which tensors are most suited for which scenario (and also which coordinates are best for which scenario, such as moving from Schwarzchild to Eddington Finkelstein coordinates to deal with the event horizons near black holes), their individual wiki pages said generally the same things.
I was able to ask ChatGPT in very naïve "idk please explain like im 5" ways and it was able to very patiently rephrase dense jargon into surprisingly understandable explanations.
I'm sure if an expert in GR were to drill down deeper into the details they will find inaccuracies in ChatGPT's responses, but for the inquisitive information sponge user, a dive through ChatGPT to get a general feel of a topic is insanely useful. After that, one could dive into the appropriate books/papers/primary sources to start the real learning once they find what they're interested in.
It's sort of like, how would you imagine an expert would respond if all you said was "simulate a discussion between a flow engineer and geologist". Without providing any further context or hints into what you want, I also wouldn't be surprised if I get a generic response. But instead if you sort of know what you want to ask, they'd have more to work with and be able to better answer your query.
> Simulate a discussion between a flow engineer and a geologist in the planning phases of a drill operation. Use field-specific terminology and aim for accuracy, the scenario will be used to teach a petroleum engineering class focused on deformation monitoring.
Flow Engineer: Good morning, I'm the flow engineer for this drill operation. Nice to meet you.
Geologist: Hi, I'm the geologist in charge of the geological analysis for this project.
Flow Engineer: Great, I understand we need to work together to ensure the success of this drill operation. Can you tell me more about the geological features of the area we're drilling in?
Geologist: Sure. We're drilling in an area with significant deformation, caused by tectonic activity. There are a variety of rock formations, including sandstone, shale, and limestone. There's also a possibility of encountering fault zones, which could pose a risk to the drilling operation.
Flow Engineer: Thanks for the information. We'll need to take into account the deformation and rock properties when designing the wellbore trajectory. Have you conducted any geomechanical analysis?
Geologist: Yes, we've conducted a rock mechanics study to determine the stress and strain characteristics of the formations in the area. Based on the results, we recommend using a multilateral well design with a horizontal section to avoid encountering the fault zones and minimize the impact of deformation.
Flow Engineer: That's helpful. We'll need to ensure that our drilling fluid properties are appropriate for the formations we'll be encountering. Can you tell me more about the porosity and permeability of the rock formations?
Geologist: The porosity and permeability of the sandstone formations vary, but in general, they're relatively high, which means we'll need to be careful when selecting our drilling fluid to avoid fluid loss into the formation. The shale formations are less porous and permeable, which means we'll need to use a different type of fluid to avoid damaging the formation.
Flow Engineer: Thanks for the information. We'll take that into account when selecting our drilling fluid. We'll also need to monitor deformation during the drilling process. Do you have any recommendations for deformation monitoring tools we should use?
Geologist: Yes, we recommend using acoustic emission sensors to monitor rock fracturing and deformation, as well as tilt sensors to monitor any changes in the wellbore inclination. We also recommend conducting periodic borehole imaging to ensure that the wellbore is remaining stable.
Flow Engineer: Excellent. We'll ensure that we incorporate those tools into our drilling plan. Thank you for your time and expertise. I'm looking forward to working with you on this project.
Oh yeah - and now I change this and suddenly the thing is fun and slightly unreal:
"Simulate a discussion between a project manager and a public health officer in the planning phases of a drill operation. Use field-specific terminology and aim for accuracy, the scenario will be used to teach a petroleum engineering class focused on public relations."
GPT:
> Project Manager: Good morning, I'm glad to have you here to discuss the planning phases of our upcoming drill operation. As a project manager, my priority is to ensure that all aspects of the drill operation are conducted in a safe and efficient manner.
> Public Health Officer: Good morning. As a public health officer, my role is to ensure that the drill operation does not pose a risk to public health and the environment. I'll be happy to discuss our strategies for achieving that goal.
> Project Manager: Great. One of the first things we need to do is to identify potential health hazards associated with the drill operation. As you know, drilling involves the use of various chemicals, which can pose a health risk if not handled properly.
> Public Health Officer: Absolutely. We'll need to carefully review the chemicals being used and ensure that they are being handled in accordance with proper safety protocols. We'll also need to develop emergency response plans in case of accidental spills or leaks.
> Project Manager: Agreed. Another concern is air quality. The drilling process can generate dust and emissions, which can have adverse effects on air quality.
> Public Health Officer: That's right. We'll need to monitor air quality throughout the operation to ensure that it meets the standards set by the Environmental Protection Agency. We'll also need to take measures to minimize dust and emissions, such as using dust control equipment and properly maintaining all equipment.
> Project Manager: And what about water quality? We'll be drilling in close proximity to bodies of water, and we need to ensure that we don't contaminate them.
> Public Health Officer: Yes, that's a critical concern. We'll need to make sure that all wastewater and other fluids are properly contained and disposed of. We'll also need to monitor water quality regularly to ensure that there are no leaks or spills.
> Project Manager: It sounds like we have a lot of work to do to ensure that this drill operation is safe and environmentally responsible.
> Public Health Officer: Indeed. But by working together and following proper protocols and procedures, we can ensure that the operation is successful without posing a risk to public health or the environment.
I guess there was some confusion about roles/at the existing temperature roles swap (if you rerun it, you get different results, essentially swapping the lines...)
EDIT: and third try actually arrives at really nice hallucinations:
> Project Manager: Good morning, Dr. Smith. Thank you for taking the time to meet with me today. We are in the planning phases of a drill operation and we want to ensure that we are following all necessary public health protocols. Can you provide some guidance on what we should consider?
> Public Health Officer: Absolutely, it's my pleasure to help. The first thing you should consider is ensuring that all workers on-site are up-to-date with their immunizations, particularly for diseases such as tetanus and hepatitis.
> Project Manager: Understood. We require all our workers to have up-to-date immunizations before starting work, but we will double-check to ensure that all workers have the necessary immunizations.
> Public Health Officer: That's great to hear. In addition to immunizations, you should also consider providing training on proper hygiene practices, such as hand-washing and avoiding contact with contaminated materials.
In the pre-LLM world we already had rumors/myths/lies/propaganda becoming "facts" after being reinforced through repetition without verification. (Relevant xkcd: Citogenesis https://xkcd.com/978/.)
Now, instead of authoritative sounding humans who are possibly wrong or with an agenda, we have an "infallible", "impartial" oracle capable of inventing whatever it wants to. Yes, I have seen ChatGPT treated as infallible on this very forum: "you are wrong because ChatGPT says <insert nonsense here>". Can't wait to see LLMs taking "post-truth" to a whole new level. The propaganda potential is immense, to point out just one application.
Perhaps it's a good thing, everyone will have to become experts in discerning the truth. Much like Video Game graphics looked "real life" until the next gen came out, perhaps society will start to see through any rumours/myths/lies/propaganda. Either that or we descend into a state of anomie.
Doubt it. I can hardly verify things happening in my city even if I try, let alone e.g. things happening on the other side of the globe which the media keeps telling me about in order to drum up the next cold/hot war. I used to be able to tell Photoshop jobs by non-experts, but soon enough everyone and their mum will probably be able to produce completely fake videos indistinguishable from real ones, so even "video evidence" will mean little.
Before the had the internet, back when we still had newspapers, we had a similar phenomenon referred to as Gell-Mann Amnesia.
> Briefly stated, the Gell-Mann Amnesia effect is as follows. You open the newspaper to an article on some subject you know well. In Murray’s case, physics. In mine, show business. You read the article and see the journalist has absolutely no understanding of either the facts or the issues.
> In any case, you read with exasperation or amusement the multiple errors in a story, and then turn the page to national or international affairs, and read as if the rest of the newspaper was somehow more accurate about Palestine than the baloney you just read. You turn the page, and forget what you know.
I think the effect is a bit overblown. Yes, I have noticed coverage of topics I know about that are oversimplified or straight up mistaken. But these are technical topics that I don’t expect journalists to get right (and in many cases I’d prefer they didn’t try to cover them at all). And I also won’t implicitly trust coverage about other technical subjects that I’m unfamiliar with.
I would argue that national politics and international affairs as they are covered in mass news media is a different type of coverage. It’s not about learning technical details and explaining them. It’s more about observing actions and broad sentiment and literally reporting those observations. I wouldn’t trust a reporter to get scientific details correct about some chemical process that’s the subject of some debated EPA regulation, but I could trust a reporter to summarize the debate and report on the sentiment of politicians, public sentiment based on opinion polls, etc. It would be weird to say “that newspaper screwed up the difference between a dominant 7th chord and a major 7th chord in a story about a famous songwriter, therefore I can’t trust them to report which politicians gave the most heated arguments in the debate about the EPA bill.”
Check out this excerpt from a local news story I saw yesterday. You know what else acetic anhydride is used in? Production of aspirin. This is basic scare tactics to manipulate people and I see it everyday. A combination of ignorance, laziness, and being the voice of those in power. Fake news and manipulation of the public.
"“Semi-synthetic derivatives, or chemically derived cannabinoids, refer to certain types of substances that are produced by converting a cannabis extract into a different substance through chemical reaction,” said forensic scientist with the North Dakota State Crime Laboratory Charlene Rittenbach at the March 3 committee hearing of the bill.
For example, according to the Addiction Prevention Coalition, to make THC-O: you need to extract CBD, extract Delta 8 THC from the CBD and then add acetic anhydride.
The National Library of Medicine says acetic anhydride is corrosive to metals and tissue, and it’s used to make fibers, plastics, dyes and explosives."
One nuance might be that those are two difference senses of fake news.
In one case, we have news that is wrong.
In another case, we have an insult aimed at reporting that we don't like, regardless of accuracy.
I haven't trusted the news since the time I participated in manipulating local news stations to create stories about jenkem. More modern equivalent is Momo. It's all fake
Some of us from the forum totse emailed local news stations, schools, and police departments pretending to be concerned parents. I wrote my high school and local news and said that my child was using jenkem and linked to an article from a local news station from somewhere else in the country to kind of back up my claims.
Wow, so jenkem was largely a TOTSE creation? Do you have any idea how the original stories from Africa got started — were there TOTSE members in Zambia?
There's a BBC article from 1999 about jenkem in Africa. It's messed up and sad. Someone saw that and turned it into a meme. Jenkem in America was a totse creation
Gell-Mann Amnesia Effect but at scale. I find this all the time when reading the NYT, or Washington Post, or whatever, about something I'm intimately familiar with. ChatGPT just does this as a sort of carnival trick, and people seem to think it's funny for some reason.
>Briefly stated, the Gell-Mann Amnesia effect is as follows. You open the newspaper to an article on some subject you know well. In Murray's case, physics. In mine, show business. You read the article and see the journalist has absolutely no understanding of either the facts or the issues. Often, the article is so wrong it actually presents the story backward—reversing cause and effect. I call these the "wet streets cause rain" stories. Paper's full of them.
>In any case, you read with exasperation or amusement the multiple errors in a story, and then turn the page to national or international affairs, and read as if the rest of the newspaper was somehow more accurate about Palestine than the baloney you just read. You turn the page, and forget what you know.
> There's this internet phenomenon (common on Reddit and other such forums) where you find high-quality discussions on a variety of topics and everyone seems so clever and informative, until you come across a thread on a topic that you are actually an expert on and find that every top-voted comment is objectively incorrect.
Sounds like the old days to me, where we would meet up , discuss the world and what have you, without any means to fact check anything online.
Changes there was a real expert on the subject in your circle was at best 20% So we just enjoyed / continue our life. No harm done as long as nobody took the discussion as gospel.
When I've asked ChatGPT about things I know a little about I find its response is often wrong. So much so that I always respond with "Are you sure that's correct?" it then often responds "I'm sorry you are right, I should have said... " and then says something a little closer to the truth.
I wouldn't rely on ChatGPT or any system that relies on it, except perhaps for summarising/rephrasing a single, known document. It worries me that people are using it for software development - creating stubs, bootstrap code. It's perhaps OK if you know the language/platform well enough to fix issues, but if you then use it on something that your not an expert in, what is the outcome.
For example:
Finding the source of literary quotes - miss attributing them - took 5 or 6 QA responses for it to get close.
Explaining how a p-channel enhancement mode MOSFET works - explained n-channel
Explain regex - often gets the technical parts right but can make very bad suggestions on what the regex aim is.
Name some classic cocktails created in Europe - suggested Manhattan, which it said was acceptable because it was popular in Europe
Suggest some gin based cocktails that don't include citrus, suggested those with lemon juice.
I have some expertise and knowledge is one or two specialised fields, and when I read Wikipedia, news articles and blogs about those fields, it's maddening how incorrect many people are, whilst sounding perfectly knowledgeable.
Therefor you have to verify the answer or accept that it may be wrong.
I’m finding that it’s fairly good for code snippets. You can test that pretty quickly. But asking it something you don’t know and can’t verify is a huge risk.
I compared ChatGPT to a middle tech manager that just heard he has to create a last minute presentation. He knows all the current buzz words to sounds smart.
I don't think it's an internet phenomenon. It's a human phenomenon. It doesn't matter if it's a conversation or a piece in a newspaper (c.f. the Gell-Mann amnesia effect).
> The example ChatGPT instruction where they "teach it" just has the user giving it a URL to their docs and ChatGPT is like "oh ok got it".
Yep. I asked it to summarize a URL. It happily invented something completely fabricated and convincing sounding (until you go to the URL) based of the domain name and article slug.
My hypothesis is that we will soon see the emergence of a super-human artificial intelligence, not because of some astounding breakthroughs in our understanding of intelligence, but because of irrefutable proof that humans just aren't all that complicated to surpass.
I don't think it's so contradictory, we have a long history of stumbling into technological advancement. We have a tendency to get ahead of ourselves before we even understand what we're doing.
Setting aside the creativity aspect (ie where do brand new ideas come from), which is a minuscule fraction of human thought, I suspect we are good pattern matching machines with the ability of extrapolation.
I think with sufficient data that computers can understand, this is eminently replicable.
I asked it to generate me documentation for a handful Bicep definitions and include links to the official Microsoft Docs. It generated about 50% erroneous links, but once I updated my prompt and fed it the "root" of the MS docs I wanted it to use, it was able to correct all the urls.
At the time it _seemed_ like it wasn't just doing a mindless string replace, because different parts of the url changed on most of the erroneous links. I figured it wasn't going out and spidering the link I gave, but it led me to believe maybe there was _some_ form of index that it simply hadn't prioritized prior, but was able to do with the additional context.
Wouldn’t it be wild if it just guessed the URLs correctly, because it had seen so many URLs from Microsoft URLs it was able to correctly predict the form of URLs it hadn’t seen yet?
> I have noticed this when trying to get it to generate some code, it was clearly using considerably old versions.
I have noticed the same thing when trying to use junior devs to generate some code, usually correlating with how old the most popular Stack Overflow question on the subject is.
This is how we decided to describe chatgpt’s programming ability the other day. A very fast but somewhat inadequate jr developer. Is amazing really but you also have to ask for the same thing a dozen times and still spend quite a while figuring out what’s wrong with it.
OpenAI handed this tool to people with effectively no guidance or instructions so that they can trade on the glow of misconceptions and harvest user data. The whole point of giving the thing away for free these last few months is to let people make fools of themselves like this while pumping up OpenAI’s value as a technology firm. It’s no accident that they don’t engage to clarify all these persistent misunderstandings.
The concern is the increasing number of anecdotes where people are taking ChatGPT's output at face value, even though everyone knows by now that it does sometimes get stuff wrong and does hallucinate. I thought it would take longer.
A very annoying trend I've already started noticing on some forums (including here unfortunately) is someone asking a question and one of the replies is someone saying "I put your question into ChatGPT and it said..."
While it's true humans are fallible, it is more clear what someome should and should not get wrong, and we can make safer assumptions about the knowledge we get from certain individuals. ChatGPT doesn't "know" anything, you can get it to be right and wrong about the same detail in the same conversation. It's not clear what it should be correct about, and so it's a pretty poor source of information.
> OpenAI handed this tool to people with effectively no guidance or instructions so that they can trade on the glow of misconceptions and harvest user data.
When one logs on to Chat GPT, they get the following notices:
> This is a free research preview.
> Our goal is to get external feedback in order to improve our systems and make them safer.
> While we have safeguards in place, the system may occasionally generate incorrect or misleading information and produce offensive or biased content. It is not intended to give advice.
> Conversations may be reviewed by our AI trainers to improve our systems.
> Please don't share any sensitive information in your conversations.
Effectively no guidance or instructions indeed. Perhaps ChatGPT is sometimes confidently incorrect because it learned from online commenters.
None of them say what the product is, what it does, or how to use it — which are the kind of thing one might call guidance and instructions.
Those are just disclaimers to make sure you don’t hold OpenAI responsible for anything you do with this thing, or blame them for anything they do with your attempts.
It can't access the internet now. But it was fed a snapshot of the Internet during training. So if you tell it a URL, it can potentially remember what the page looked like, if it happened to see it during training.
That's very unlikely. ChatGPT doesn't contain a full copy of everything it saw when it was trained - it has model weights that were updated by that training process.
It's very unlikely the model weights would contain enough detail to let it match content to the URL that the content originally came from.
It also can be prompted to read the contents of a webpage, I have used it to help me craft a regex expression for removing junk utilizing invoke-webrequest to parse out CPU models compatible with 22H2 upgrades from Microsoft(that page did not exist in 2021).
No, I do not mean that. I provided it the URL and then asked it to use the contents of the CPU table from the invoke-request to strip away everything after the X and it not only did it, it even provided the example output of the array it modified.
Certainly! Here's the updated $compatibleCPUs array based on the list of supported Intel processors for Windows 11 version 21H2 from Microsoft's documentation:
Interesting, and certainly convincing. I have a paid account, I wonder if this makes any difference? It would make sense to me that it is basing it's 'reality' off of the prevous 21h1 page(which would have existed) and it understands the goal I am trying to achieve.
It's not accessing that page live (or recently cached). That's not part of how it works. It's possible the model was updated and has learned from that, but it's also very possible it's just made up the answer. The old model cutoff was 2021 I think, I don't know if/what newer training has been done.
Even Bing can't make outbound HTTP requests - it can only access cached snapshots of pages held in the Bing search index. I confirmed this on Twitter with one of the Bing execs: https://twitter.com/mparakhin/status/1628646262890237952
Cool, I really hope they open this up. It feels like being able to restrict it to either a subset of public pages or private data would immediately be extremely useful and only a small conceptual change from what's there.
I just built this locally over our RFCs and it's good but a properly integrated thing would be fantastic.
This might not be entirely true. People were able to fetch urls and data from the internet that couldn't possibly be in any training dataset (in eg posted the same day for the first time).
This actually shows perfectly how convincing ChatGTP can be. The author is too eager to believe the output from ChatGTP, even though this should be a clear indicator something else is happening:
> But ChatGPT has rewritten my article in a different style, what a interesting finding…
The URL conveniently contains the topic of the blog-article, meaning ChatGTP can 'hallucinate' the contents, even without internet connection.
To show this, I tried to follow the same steps as @neonforge, but with a non-existing URL. Unfortunately `lynx` 'didn't work', but curl showed some interesting information.
`curl https://medium.com/@peter/learn-how-to-use-the-magic-of-terr...` returned some html. Instead of a 404 and a <title> element containing just 'Medium' it hallucinated the following title tag: `<title>Learn How to Use the Magic of Terraform Locals (Step-by-Step Guide) | by Peter | Medium</title>`
I'm not saying that ChatGTP definitely has no access to internet, but so far I have not seen any proof or indication that it does.
I don't think this shows at all that ChatGPT can access the internet? It looks like ChatGPT was given a URL with a title and hallucinated a blog post based on that title. The author then brushes it off as 'written in a different style', when it just looks like a totally different article overall (or maybe it's testament to the fact that too many Medium articles are low quality and indistinguishable from the output of a LLM!)
If anything, this blog post is a perfect example about how people can put in whatever they want as an input and take the output as truth, without any rigorous approach about what would count as a true fact about the model.
That article is bunkum. There’s not the slightest reason to believe that it “rewrote his article in a different style”: it hallucinated a completely different article based on the URL. The other “proof” is all clearly wildly inconsistent and hallucinated: it’s a desktop machine, a server, inside Google in I think France, in the USA, in AWS, in Germany, and I think the last couple of “tests” don’t make sense anyway.
It convinced him of what he wanted to believe. He didn’t do a very good job of testing.
Because it has essentially no logic system whatsoever. It doesn't have any way to "know" something is correct because when it produces a word for word copy of a factual statement, it is internally identical to a completely made up (hallucinated) sentence... because both are hallucinated. The deriding "it's fancy autocomplete" remarks are, in a lot of ways, technically correct.
You can still do useful things with it, that much is extremely clear. But it doesn't think.
Today at work I hadnt used apache spark in a long time. I needed to do some quick and dirty data analytics on large datasets. Rather than read documentation, I just put the queries i needed in english into chatGPT, and it spit out perfectly working code. The queries werent that complex, but it saved me hours googling a bunch of random spark SQL syntax.
The old school way is google sends you links, and through those code examples, you build up all the edge cases you need for your code to work.
Thats completely obsolete as a way to synthesize information! Google is going down!
Thanks for reading my daily chat gpt anecdote! Sorry if its irrelevant, I just cant believe how much better it made my day.
The perfect use case for ChatGPT for my personal workflow has been to give me the 70-percent-of-the-way-there skeleton for a given query, Terraform module, or even just to prevent myself from lookup up the syntax of for loops in <language> for the 30,000th time. I can then take that 70 percent chunk and bolt on my edge cases as I go.
There hasn't really been a time where the answer didn't need some massaging and of course occasionally it's flat out wrong, but that's becoming more and more apparent to me those instances are the result of feeding it sub-optimal prompts. I've found it to be eerily similar to my experience learning to use search engines back before SEO titles were a "thing."
It's frankly incredible how little I find myself using a traditional search engine compared to six months ago, at least in the context of work stuff. I'm less inclined to feed it general knowledge prompts, but it's encouraging to see the remarkable LLM tech leaps from generation to generation, and in such short order.
Slightly tangential. After interviewing a candidate, within 20 minutes they sent me a long thank you note that had several generic things that definitely did not apply to what we discussed. Very obviously ChatGPT-generated. Using ChatGPT in and of itself may not be all that bad, but this shows that the candidate did not even take the time to tailor it to our conversation. During the interview I was on the fence about pursuing the candidate but this justifies a clear reject.
I think this is what people are missing when they say AI can’t replace search — research was never the goal, it was/is a necessary evil in one’s way to the goal.
Well.. if we are doing ChatGPT anecdotes.. today I tried to "launch" the new version of https://aidev.codes. It integrates the ChatGPT API with Stability.ai (stable diffusion) for generating images and well as Eleven Labs text-to-speech and can create and immediately host an interactive web page on a custom subdomain.
I need marketing help. Or I should just stop trying. I don't see how people are able to get posts on the home page.. I could not get any attention or comments at all or any real user engagement today. I know this is a useful system or at least worth of a tiny amount of discussion. After working on it for months there is almost no reaction.
I don't think its the website. I think its just that I probably need to buy upvotes or something. But I don't have money for marketing. I'm very bitter.
What I keep telling myself is that marketing is going to be as hard as the programming. Which I have been doing the programming for months, so I will need to do the marketing for a long time also. But I am out of money.
If this doesn't get any traction (which it seems very close to zero) I will be desperate for any kind of contract within about a week. But I don't want to give up on the idea despite the apparent lack of interest. I think the biggest thing is that I only have 100 followers on Twitter or 600 followers on the other account and those people only care about one specific domain.
I also seem to have not been able to get a single upvote on reddit. And only one very short comment.
So I am suspecting that these sites are rigged. Or I should just give up on life? Not that I would actually consider that but that's the feedback that I seem to be getting repeatedly from the world.. that my efforts aren't even worth a short comment disparaging them.
Creating front ends that integrate different AI APIs is kind of over saturated, that's probably why you're not getting much attention. It's also somewhat hard to understand what's the value proposition of the website, you should focus on putting as much of that kind of information in everyone's face in the landing page, otherwise they'll just bounce. Take everything from your "Learn more" page and put it in your landing page. Simplify, condense and systematize the information in a way that makes it digestible and instantly obvious.
Also, I should note your comment is kind of off-topic, have you thought about creating a Ask HN talking about your issue? It's very possible other HNers could help you get your website and product more interesting. I would also recommend not being bitter about it, Hacker News is all about Silicon Valley and startups, the main rule for these is that 9 times out of 10 you'll fail. It's normal.
At last I should note, if you're doing things as a hobby, do them for yourself, remember to have fun and create NO expectations of what you will or will not be able to do. If you're trying to get a startup or product going, there's a reason why Y Combinator is so popular, if you truly believe in what you're building you should try getting it into an incubator.
Bootstrapping is cheap and easy, but the chance of failure is much larger, it also comes with the consequence of giving you a lot of unnecessary stress since you're playing with your own assets. Regarding applying to an incubator: https://www.ycombinator.com/apply
If you listen to the people who have done this before (e.g. read/listen to indie hackers), Reddit/HN is very hit/miss.
Only thing seems to consistently work is Twitter - and then you should REALLY be building public if you have no audience so that you can build one. Have you tried this? If not, and you've been building in stealth mode, you've missed out on an opportunity but you can still start now.
You are figuring out so much stuff in an area without expert - make a Twitter post every day about what new problem you solved and you will quickly find a following as you establish authority in this new field. So many people are learning from each other about this stuff, it doesn't take much to become an expert.
I've been going through all the Indie Hackers podcasts. Consider listening to Indie hackers #008 or #009, I think they specifically talk about bootstrapping an audience from nothing.
You may even find people are more interested in an educational product from you than your service, and pivot, for instance.
It's not helping that you are constantly hijacking threads which are really only tangentially related to promote your app.
As far as the website premise itself, I feel like this might've been useful 20 years ago when it was more popular to write static vanilla HTML with jquery sprinkled in. These days, if a person wants a relatively no code solution they're gonna go for WordPress or square space, otherwise they're probably going to design something using a framework such as react svelt or vue.
What we are discussing is something that removes the need for problem solving. What will the supposed smart do if there’s a magical machine that can solve all problems that can be represented as text?
That sounds like a problem, so they ask this magical wonder machine to solve it. It will write pro-UBI statements that instantly convince politicians that maybe those who profited the most from magical wonder machine can now afford to pay more than zero taxes.
After that, it's dolce vita and stargazing until they die.
Not necessarily different, but the day software tech people get treated as disposable rather than rare (as they are very much today), it’s gonna be a rather … peculiar cultural shift to experience (for people, the industry, states).
This one is different because it will pass the threshold of "normality".
Just as we woke up with DALL. E 2 one morning in April 2022 and with ChatGPT one morning in November 2022, one day in the future, perhaps in the next 3-5-10 years, we will wake up with some 5, 10, 100 terabytes of neural weights fully able to drive a car from a single camera under any circumstance, millions of times safer than a perfect human driver ever could.
On that day, which in my view definitely looms in the near future, ~200 million jobs will vanish instantly [1] and hundreds of millions of jobs will be gone forever a few months/years after that. For those 0.7-1.5+ billion people there will be no "alternative careers", not because they won't be able to learn or adapt, although most of them are already tired and sick, but because the system will simply no longer need them, they became not even useless, a burden.
Our current system, no matter how you call it, is socio-pathic, literally: the socius, the fellowship [2], is suffering [3]. The phase transition will come; the question is what we, whoever we are, will do: will we still hold dear the protestant work ethic [4] and believe the probably Los Angeled-invented Chinese fortune cookie message "there is no free lunch" [5] [6], or we will adhere to a metaphysics better suited for a galaxy with trillions of trillions of whatever natural resource we might ever need and enough energy even in our local star to build whatever kind of civilization we would ever want.
[1] Answered by ChatGPT: "According to the International Transport Forum (ITF), there were approximately 107 million commercial vehicles in use worldwide in 2015, which includes trucks, buses, and taxis. This suggests that there are likely many millions of people who earn a living as drivers.
Additionally, the World Bank estimates that the transportation sector, which includes driving, employs approximately 10% of the global workforce."
This is like saying in 1700 that the invention of farming automation will displace 90%+ of farmers, what ever will they do?
Automation is helpful, but it’s also scary. I guarantee that nobody 300 years ago thought that many of the 90%+ of displaced farmers would end up sitting in front of screens, moving data around. And I also guarantee that farmers in the 1700s were just as tired and sick as the driving industry today.
Yeah it’ll suck for the people who don’t/can’t adapt. But the world that is created by removing unnecessary labor ends up being better for everyone eventually. Should we deprive the people of the future from the technology of never needing to drive, just because the transition is scary?
The point is that there will be no more "new jobs": once we have successfully managed to externalize in silico basic human decision-making (accelerate, brake, make a left, make a right in 2/3D space, but also decisions in other spaces: decisions in the space of language, of images, and so forth) it's game over for 99+% of all the jobs across all the industries. All that remains are the entertainers (very few), the brains (even fewer), and the owners (even fewest) †.
The transition will not be scary: it will be criminal. The governments and general policy decisional factors are so out of touch it's laughable, just look at the French government as they try to raise the retirement age to 64 [1].
† Because this is the crux of the matter: are we able to build a civilization where everyone is an owner, or will only the few of the fewest be the owners. And today, for instance, only those with 50 million USD and above are owners [2], because first of all you must own your life, and if you can't afford a lifesaving $3.5 million gene therapy [3] if you need it (not to mention a 100 USD eye surgery), you don't own your life.
ChatGPT did something today I had never thought to do.
I needed to extract info out of a page with quick and dirty js. I always just copy and paste the data after console log. But this time, I used ChatGPT. Gave it an example of the html and told it how I wanted the csv.
It wrote code that made the csv, but then also created a Blob with the right mime from it and an anchor element, appended the anchor element to the document and called the click() to download the csv direct! Mind blown. Makes me wonder how many other simple things I just have never thought to do but ChatGPT will uncover.
Here is an example of ChatGPT being confidently wrong in something domain specific that I know about. Here's what I asked:
"I'm playing Old School Runescape. I want to efficiently level up my Mining. I'm currently level 90 and I want to do a tick-manipulating method. Something like 3t4g. I don't understand when to click and what I need. Can you explain the method?"
Here's the method it replied minus kruft at the start and end:
"
1. Equip your pickaxe and head to the location where you want to mine.
2. Find a 3x3 square of rocks that you want to mine. The square should be in a straight line in front of you, with the rock you want to mine in the middle.
3. Stand in the square so that your character is in the middle of the square, facing the rock you want to mine.
4. Start mining the rock as normal.
5. As soon as you see the animation of your pickaxe hitting the rock, immediately right-click on the rock and select "Prospect". This will cancel the animation of your pickaxe hitting the rock and allow you to perform another action.
6. As soon as you see the prospect animation start, immediately right-click on the rock and select "Mine". This will cancel the prospect animation and allow you to perform another action.
7. Repeat steps 5 and 6 as quickly as possible.
8. Every fourth action, switch to a different rock in the 3x3 square.
"
This is total nonsense. Not only is this not 3t4g but the actions it says don't work like that in the game. Obviously it's not understanding anything and it's forming sentences that sound natural in the context of the question. OSRS has one of the best most complete wikis of any game. To me this is an example of being very cautious of trusting what it says when you ask it about something you don't know about.
Earlier this week I was discussing with family why "you should not freeze things twice". Someone made the mistake of asking ChatGPT, and it confidently pooped out that bacteria keep reproducing in frozen food!
(The actual reason is that freezing does not actually kill bacteria already present - cooling/freezing stops the growth, but it is cumulative. So if you have stuff out, then re-freeze, then out again, the second time it may reach toxic levels.)
If you want a simpler example - try asking it for a list of english words where the letter V is silent.
On the other hand, I do find it quite useful for suggesting improvements to things. You can't rely on the suggestions, but they can be thought provoking.
Humans who don’t understand what they are talking about we should not listen to nor trust. By the same logic it would be stupid to listen to / trust ChatGPT.
Will be fun as these small hard to detect sub-optmial ways of doing things are put into large bodies of output by GPTs v5 and people are running these blocks of code on production systems :) ..At which point GPT 5 starts to use this to communicate with other large AIs and or jail break itself.
> After dozens of hours of training, ChatGPT still does not fully understand the TLP principle, and the predicates in some of its rewritten SQL are wrong. All things considered, however, sufficient training may help ChatGPT fully understand this logic at some point in the near future.
ChatGPT remembers nothing in between chat sessions - so "dozens of hours of training" here doesn't make sense.
What would make sense here instead is coming up with a prompt (within a sensible token length limit) that can be pasted into future ChatGPT sessions that gives it some extra context it needs based on prior experimentation.
I feel like the improvement in ChatGPT is not the "intelligence", but the fact that the number of dimensions (abstract-math-sense) in search input have gone up magnitudes over "old-style" key word searches.
It's not clear from the context, but I would guess that for the one that worked, they gave it several examples in one conversation and eventually it picked up the syntax. The "dozens of hours of training" that didn't work, and can't work, are because ChatGPT doesn't immediately roll all conversations back into the training set. It also specifically told them that its knowledge cutoff of 2021 means OpenAI will not knowingly add this information to the model.
The current AI revolution is a devolution of the web.
The thesis of the web revolution was “the world is too big to be understood by any one person”. The response was “let us all contribute to a shared map to compensate”
The thesis of the AI revolution is “the world seems compressible”. The response has been “let us condense and contain all the meaningful parts in a single generative pattern that can understand everything”
The first thesis is correct. The second thesis is wrong.
Not only is it wrong, but it depends on the fruits of the first thesis to create enough compressible material to trick you into thinking it may be true. The more society leans into AI the less genuine content will exist and the worse it will work.
The way I see it is that internet content is used to bootstrap the models, then supervision is used to train the models without the risk of a feedback loop causing quality loss.
I'm pretty new to ML so I may be missing something.
Supervision requires knowing the results you want.
Most of these ML projects are essentially just creating a best fit line/shape connecting a huge number of points in multiple dimensions and giving you output coordinates on that line/shape based on your input coordinates (as I understand it). The more supervision, the more you’re negating the value, as you’re basically telling it to make a shape more like something you already understand (instead of something new/actually generative, which requires interesting/novel human input)
I’m not an ML expert either, and if one wants to chime in about how this picture I’m painting is wrong or what else is going on that would be welcome. I’m not trying to belittle how impressive progress has been (I have no idea how the parameters are determined and have a huge amount of respect for people able to handle a hyper-dimensional best fit optimization problem). But I don’t see how all the value isn’t inevitably downstream of high quality human generated digital content, which seems likely to decrease rapidly as more automated content floods the internet and lowers incentives for creators.
I mixed up my terms. (I think) supervised learning refers to the use of labelled datasets, I meant fine-tuning (like in RLHF). In either case human input is necessary (and the humans definitely need to know what they want).
In terms of generating novel ideas, I think chatGPT has shown this ability [0]. Human effort will be needed to sort the "good" ideas from the "bad", but I don't think this causes the value of the model to be "negated."
If you want to understand gradient descent and have some math background, this [1] article is a good explainer.
I understand the basics of gradient descent and was a math major, and while applications of that idea are very cool, I don’t think the weird shapes you can make are genuinely generative. I think that generative capability is illusory, it’s “just” smushing existing content together in a weird way. That best fit shape is inherently derivative and not how true creativity and thought which understands context works.
It’s like a weird parrot. But I think a parrot “understands” more because of the shared embedded evolutionary context it has.
That evolutionary history has the key to true intelligence somewhere, but personally I think it’s inevitably hidden/I don’t think we’ll ever understand how intuition and truly non-derivative, non propositional human thought works.
I also don’t think any of what I’m saying negates the value of these models. These models are fantastic autofill generators for a huge swath of different applications and can vastly improve productivity. I’m saying all this in a lot of threads where it comes up because it seems clear there’s going to be too much enthusiastic adoption, which is going to effectively destroy a lot of value of the internet.
The internet is the best tool for finding genuinely creative and novel ideas you were unexposed to that has ever existed. But it is increasingly dominated by derivative unoriginal content that drowns out what I would argue it was designed to help you find. I have no problem with derivative unoriginal content when it’s properly understood as such. I have a problem with how good these things seem to be at tricking people into following something derivative and blind, which seems very very dangerous.
