No, it's completely useless, and puts the entire rest of the analysis in a bad light.
LLMs have next to no understanding of their own internal processes. There's a significant amount of research that demonstrates this. All explanations of an internal thought process in an LLM are completely reverse engineered to fit the final answer (interestingly, humans are also prone to this – seen especially in split brain experiments).
In addition, the degree to which the author must have prompted the LLM to get it to anthropomorphize this hard makes the rest of the project suspect. How many of the results are repeated human prompting until the author liked the results, and how many come from actual LLM intelligence/analysis skill?
By saying that's its gold mine, I think OP meant that's it's funny, not that it brings valuable insight.
ie: THEY KNOW -> that made me laugh
and as the article said
"an LLM who just spent thousands of words explaining why they're not allowed to use thousands of words", its just funny to read.
The fact that they produce this as “default” response is an interesting insight regardless of its internal mechanisms. I don’t understand my neurons but can still articulate how I feel
It's sure phrased like one, but I'd be careful to attribute LLM thought process to what it says it's thinking. LLMs are experts at working backwards to justify why they came to an answer, even when it's entirely fabricated
I would go further and say it's _always_ fabricated. LLMs are no better able to explain their inner workings than you are able to explain which neurons are firing for a particular thought in your head.
Note, this isn't a statement on the usefulness of LLMs, just their capability. An LLM may eventually be given a tool to enable it to introspect, but IMO its not natively possible with the LLM architectures today.
There's a slight exception to this, in that LLMs are able to accurately describe portions of the buffer that are arbitrarily hidden from the user.
An LLM that says "I said orcs are green because I recalled a scene in lord of the rings..." is fabricating*. An LLM that says "I talked about white genocide because my system prompt told me to" is very likely telling the truth because it can literally see the system prompt as it generates the output. Even though in the situation I'm referring to the system prompt was hidden from users. It's a logical conclusion from the combination of the system prompt and its previous output that that is why its previous output is what it is (that anyone could make with the same degree of confidence if they had access to the full buffer).
* Unless it's reading back from a <thinking> section of the buffer that was potentially hidden from the user.
Right… because these things are trained on sci-fi and so when asked to describe an internal monologue they create text that reads like an internal monologue from a sci-fi character.
Maybe there’s genuine sentience there, maybe not. Maybe that text explains what’s happening, maybe not.
Very useful guide. I'm incredibly frustrated by how Libre shows more effort put into preventing others from building a similar app instead of just building a good app in the first place.
They could make so many people so happy if they just gave us a damn good app.
I also can't see how this benefits them at all - their app isn't tied into monetisation, it's free to download (replacing their own chargeable reader!), and it doesn't work at all without a sensor (the paid component).
The only reasons I can think of are:
- legal restrictions on them as a healthcare provider?
- preventing users from jailbreaking it to work longer (honestly people only do this if they legitimately can't afford a new one)
- they want user data for monetisable purposes
I've used both systems. Each have annoyances. The Dexcom system can't be silenced and has an insane unstoppable firm "your sensor will expire in 6 hours", so unless you reset your sensor in the afternoon or evening this will wake you up and has woken me up at 4AM. I had to hide the Dexcom sensor in a watertight cigarette case to silennce the alarms. Both brands mobile apps hold you hostage and will not allow going do not disturb. I'm currently on Freestyle 2 with their own reader, which is silencible and I use a third party app Juggluco from some Dutch Diabetic. That app is quiet and gets realtime readings via bluetooth, it flakes out some times but it is great to have a second reader. Juggluco also gives me a bonus extra 12 hours at the end of the sensor life, so I'm slowly building up a reserve in days and eventually will have an extra sensor in que. https://www.juggluco.nl/Juggluco/index.html
The one really helpful use for a massive nodegraph with way too much data? Convincing people that something is complicated. Eg: illustrating to non-technical people that your codebase is a massive mess.
I later used d3 to do pretty much the same thing, but with a much larger graph (still only 100,000 nodes). That was pretty fragile though, so I added an `export to svg` button so you could load the graph, wait for it to settle, and then download the full thing. This kept good quality for zooming in and out.
However my nodegraphs were both incredibly messy, with many many connections going everywhere. That meant that I couldn't find a library that could work out how to lay it out properly first time, and needed the force-directed nature to spread them out. For your case of 1 billion nodes, force-directed may not be the way to go.
This is super interesting! I've always had a strong preference for schema on write, both in databases and in life - prefer to organize the cupboards when I first set up house, rather than dumping everything in and hoping for the best. So I'm definitely very inclined to immediately accept your basic premise: schema on write is substantially better.
But I've ended up in several headed discussions on this, in the context of SQL vs NoSQL. The one argument I've been slightly persuaded by is: schema-on-read is significantly more flexible than schema-on-write. In most of my actual programming applications I still use SQL, because in coding I think it's better to prioritize planning and structure over flexibility.
If there's one area where flexibility is necessary, it's real life. When I first start researching something new, I don't usually have enough knowledge to actually structure my schema effectively, and the usefulness degrades. Of course, you can update the schema. This isn't always great. If your schema needs to change constantly, it wastes a lot of time. Depending on complexity, it can also just be a massive cost. Eg: when I have the kitchen set up all nicely, but then we get a new blender with five different attachments and now I need to find an empty shelf for them. Took about an hour to reshuffle everything satisfactorily.
All that said, I'd still say that schema-on-write is better than schema-on-read. Some structure is typically always better than no structure.
However, I've recently been reading a book that I think gives an interesting different insight to this problem - Designing Data-Intensive Applications, by Martin Kleppmann. I've always considered the main categorization of databases to be schema-on-write vs schema-on-read, but this gives a completely different method: databases are either document-based, relational, or graph. Relational databases we're all generally familiar with, while document-based is similar to today's NoSQL.
Graph databases have fallen out of favour, but I actually think that they might be the best at representing the human information gathering process. They have a structure that's provided on write, but isn't always consistent across entries - because it's flexible, and can be added to very easily. This lets us expand the schema as we gather more information and our view of the world changes, without needing to rearrange our past knowledge. I also feel like a graphlike structure better represents how we think.
Honestly, the main useful point I got from the book is that schema-on-write doesn't need to mean lack of flexibility. That is the case with most of the RDSs we use today, so it's what I've come to expect. But that shows a lack of imagination on my part, rather than any inherent restriction.
I agree completely about Graph DB's. I used a third party Graph database that was schema heavy on Nodes and edges and it was amazing for mapping business processes as the customers can actually understand the data model and we did not need any DBA's. I wrote several custom enterprise systems based on this graph backend that were very successful in their deployment and upkeep.
The company that has this DB never sold it on its own and its only available with their $$$$ CAD / CAM / 3D / PDM / PLM software packages used by the big guys (E.g. Boeing, Honeywell ...)
Great book! Graph databases are useful, but I'm not sure if they're still a great interface for a direct human interface. Problem is listing links and finding connections is tiring for us to do.
That said, with enough automation and context-aware software, graph dbs start looking like a great idea.
SQL vs NoSQL is a perfect example of schema-on-read vs schema-on-write, even though with jsonb in postgres and more templates in Mongo you can use both either way.
That's my next addition planned! Good idea on Tolkein and Star Wars - I'd been thinking smaller scale (eg Harry Potter) but those are probably a lot more interesting, and also better test cases.
Oh wow, hadn't thought of that. Could be very possible! I'd say the main drawback is that would probably require self-hosting it instead of making use of Binder to have it available for free.
But you're right, the speed is definitely the biggest problem. I'll definitely take that idea seriously if I do anything else with this.
Thank you so much!!! It's a project I've been wanting to have for awhile, so I'm glad that other people like it.
Fanwikis are my next target - probably a bigger amount of interest, and I'm hopeful that there will be a similar amount of consistency. I'll also need to work out what the interesting relationships to trace would be for fandoms - or see if I can parameterize that, since I suspect it varies a lot.
I now understand a bit better why Wikipedia isn't that popular as a base layer - it's built by humans, for humans. That can make it a bit tricky to parse for computers. There's no consistency to naming, labeling, or even organisation. There are some cultural standards, but they're not binding and tend to vary between sections. For instance, children are labeled as 'children' for most people, but 'issue' for a lot of monarchs, and 'offspring' for mythological characters.
Yeah you're right, but it's still the biggest information resource we have. Some sort of effort towards making Wikipedia more machine readable - maybe through parsing or automated synonyms - would really go a long way there.
