I recently started diving into LLMs a few weeks ago, and one thing that immediately caught me off guard was how little standardization there is across all the various pieces you would use to build a chat stack.
Want to swap out your client for a different one? Good luck - it probably expects a completely different schema. Trying a new model? Hope you're ready to deal with a different chat template. It felt like every layer had its own way of doing things, which made understanding the flow pretty frustrating for a noobie.
So I sketched out a diagram that maps out what (rough) schema is being used at each step of the process - from the initial request all the way through Ollama and an MCP server with OpenAI-compatible endpoints showing what transformations occur where.
Have you tried BAML? We use it to manage APIs and clients, as well as prompts and types. It gives great low level control over your prompts and logic, but acts as a nice standardisation later.
I found this really helpful. I've read a few different bits around this area, and being able to quickly click and scroll around this has confirmed my understanding of it now - thanks!
I thought it funny to think how this is all to give the impression to the user that the AI, for example, _knows_ the weather. The AI doesn't: it's just getting it from a weather API and wrapping some text around it.
Now, imagine being given a requirement 5 years ago like: "When the user asks, we need to be able to show them the weather from this API, and wrap some text around it". Imagine something like your diagram came back as the proposed the solution:| Not at all a criticism of any of your stuff, but it blows my mind how tech develops.
I think we fit in that bucket [1] - open source, self-hostable, based on OpenTelemetry and backed by Clickhouse DB (columnar, not time-series).
Clickhouse gives users much greater flexibility in tradeoffs than either a time-series or inverted-index based store could offer (along with S3 support). There's nothing like a system that can balance high performance AND (usable) high cardinality.
Companies like https://signoz.io/ are Opentelemetry native and have very transparent approach to predictable pricing. You can self host easily as well.
It wrote a sad song for me in A7m that was actually quite good.
So far I've been blown away by its ability to not only understand what I'm asking, but provide decent answers and clarify something or update its answer when it's wrong.
In the context of music, the letter "A" refers to a musical note, and the number "7" refers to a chord. The chord is called a dominant seventh chord, and it is built by taking the first, third, fifth, and seventh notes of the A major scale. The letter "m" stands for minor, which means that the chord is a minor chord rather than a major chord. So, the chord "A7m" is an A minor seventh chord.
Which is wrong - that's Am7! (BTW without any further qualification a "7" is assumed to be a minor 7th, even for a major key - and you can even have Am maj7, i.e. an A minor chord with a major 7th - i.e. a G# - added). A7m doesn't mean anything to me - even if it were Am7, it's not a key a piece can be "in"...
Oh and also "a dominant 7th" chord is specifically the chord formed on the fifth note of the scale, so a dominant 7th in Am is an E7 chord (and a dominant 7th is always a major chord, with a minor 7th!).
Edit: I asked it whether it thought A7m made sense in that context...
"A7m is not a meaningful description of a key that a piece of music could be in. In music, the key of a piece refers to the tonal center around which the melody and harmonies are built. Keys are typically designated by a letter name, such as C, D, or E, and may include additional information about the type of scale (major or minor) and the presence of any accidentals (sharp or flat notes that are not part of the key's natural scale). For example, a piece in the key of C major would be indicated as "C major" or simply "C," while a piece in the key of A minor would be indicated as "A minor" or "A.""
(there was a second paragraph attempting to convince me that A7m was a description of chord, similar to what you posted.
Note that the above description is basically correct, though the way it describes accidentals is not quite how I'd put it, plus it would be unusual to ever say something was just in "A" if it were actually in A minor).
(I then got it to give me an example of a well-known song in Am and it replied "Love me to death" by Type O negative, of all things. That's actually in G# minor - interesting the web in general does a really bad job of guessing what key that's in too, most of the responses from a google search say F# major! I'd think "Stairway to heaven" would be about the best known song in A minor - looking at some of the Google results I suspect part of the problem is interpreting the "A" as the indefinite article rather than the note name?)
Does this type of explanation actually help the neural net do anything? If ChatGPT is just combining other instances of A minor music it has been trained on, and has no ability to "understand" what "letters referring to musical notes" actually is, why bother explaining this? Have you tried simply asking it up front to use an am7 chord as the basis for a chord progression or song?
That’s correct. It already knows what “works” musically because it’s likely seen many thousands of harmonic progressions in its training data. The input we give it just constrains it rather than teaching it something new. At least that’s my guess.
I found myself in a similar boat (I'm a software engineer with no college degree).
I did well in school with geometry, algebra, and pre-calculus but I did so by memorizing not by understanding.
A decade later I ended up going through a lot of Khan Academy videos to refresh myself and then diving into discrete math and linear algebra textbooks. It really helped me to finally understand the core mathematical concepts that we use in programming algorithms.
> In fact, in numerous studies conducted throughout the world, consuming four or five eight-ounce cups of coffee (or about 400 milligrams of caffeine) a day has been associated with reduced death rates.
That seems like an absurd amount of coffee to drink in one day, no?
That's the way to do it. No sugars or anything, just that great taste.
I saved a huge amount of money going from pods to a manual Breville Barista, which whilst making coffee shop quality brew is also far better for the environment. Price of purchase was recovered in 6 months, and all savings from there.
Heads up, this depends on what you mean by double shot of espresso. One ‘payload’ of espresso as regularly consumed is usually 15-18g of ground coffee and has about 75mg of caffeine. Confusingly, traditionally and especially in Italy this is called a ‘double’ espresso. Because of that reason it’s better to talk about caffeine in terms of grams in.
Unless you are pulling two shots of espresso into the same cup (which will likely overflow your average espresso cup) your caffeine intake per shot of espresso is 60 to 75mg. I don’t know of any portafilter basket that would let you get to 150mg in one pull, that would be a quadruple espresso.
I just went an measured, about 18g of finely ground coffee fit into my 'double' espresso shot... However I could easily power through 5 of those a day.
I further fell from grace when I started packing as much as possible into a massive percolator for rocket fuel by the filled cup.
But you are right, 3 shots is probably not that much caffeine. It's somewhat hard to get good numbers on it.
Yes, and honestly you’re doing fine, because your 3 pulls of espresso is about the same as a Starbucks venti latte at 180-190mg. I think in Europe the caffeine content is even higher at something like 225mg for the same latte.
If you really, and I mean really want to have a go at the rocket fuel territory, you can do what many cafes in Istanbul now offer: Turkish coffee… at a regular coffee size. I think it takes about 150g of ground coffee per cup. That’s like pulling repeated shots of espresso till you fill a regular mug, except Turkish coffee is finer ground so has more caffeine than your average espresso per gram of coffee in.
Obviously, don’t try this if you have any medical conditions or sensitivity to caffeine, but this is what some Turks often drink on their days off.
Yes, that used to be the case I think. The espresso world backed off of that in the last few years. Nowadays the average dose is trending lower. Possibly, half the reason is that a larger dose is more inefficient (i.e. it drops your extraction rate) and the other half is that coffee costs money and for coffee shops that operate at 5-10% margins, 14g doses (standard Italian double espresso) vs 21g (the ‘third wave’ dose up till a few years ago) does make a meaningful difference in whether the store is under water or not. I think it’s settling around 16-18g.
Yes. James Hoffmann on YouTube talks more about the margins of coffee business if you'd like to learn more, that's where my knowledge comes from for the most part.
Holy shit, I would be bouncing off the walls if I drank that much coffee. I can't even handle one in the morning and one in the afternoon or I will be awake until 4 in the morning
People metabolize caffeine at wildly different rates depending on their genes. One cup every other day in the morning is my limit until I see significant effects on deep sleep (as measured by a Dreem 2.) :/
Different sources quote a half-life of 3-5 hours in healthy individuals who are not pregnant.
https://sleepeducation.org/sleep-caffeine/
Caffeine begins to affect your body very quickly. It reaches a peak level in your blood within 30 to 60 minutes. It has a half-life of 3 to 5 hours.
https://www.ncbi.nlm.nih.gov/books/NBK223808/
The mean half-life of caffeine in plasma of healthy individuals is about 5 hours. However, caffeine's elimination half-life may range between 1.5 and 9.5 hours,
https://www.news-medical.net/health/Caffeine-Pharmacology.as...
The half-life of caffeine (time taken for the body to eliminate one-half of the caffeine) varies widely between people, depending on factors such as age, body weight, pregnancy status, medication intake and liver health. In healthy adults, the half-life is approximately 5 to 6 hours. Heavy cigarette smoking can decrease the half-life of caffeine by up to a half and in pregnancy, the half-life may be increased by as much as 15 hours. [...] The stimulatory effects of caffeine may begin as early as 15 minutes after ingesting the drug and last as long as six hours.
My IBS looks at this and thinks "No shit, Sherlock! Actually is gonna be tons of shit after this".
Stepping aside myself trying to make a Brazilian's joke in English, a single cup of coffee (around 80mg-120mg) really give me cramps and sometimes diarrhea.
I always start the day splitting a 12 cup pot with my husband, frequently we make more later in the morning. It's really nothing if you are habituated to it and not especially caffeine sensitive. I know many other similarly heavy coffee drinkers.
It tastes good, especially with breakfast food, wakes me up a bit, and is hydrating. I experience no negatives, and nothing in particular happens to me if I skip it either. It seems implausible to frame as an addiction unless you start from the preconception that caffeine must be a vice.
And according to this article, and others, I apparently gain various health benefits from my preferred beverage to boot :)
Seems average for a coffee-drinking office worker. One for breakfast, one for lunch and one between each major meal (assuming 100mg per cup which would be normal for a middle-roast brew coffee).
But as others have mentioned, it’s hard to control for other factors.
I imagine 300+ mg of caffeine is also associated with a less sedentary lifestyle, for example.
Want to swap out your client for a different one? Good luck - it probably expects a completely different schema. Trying a new model? Hope you're ready to deal with a different chat template. It felt like every layer had its own way of doing things, which made understanding the flow pretty frustrating for a noobie.
So I sketched out a diagram that maps out what (rough) schema is being used at each step of the process - from the initial request all the way through Ollama and an MCP server with OpenAI-compatible endpoints showing what transformations occur where.
Figured I'd share it as it may help someone else.
https://moog.sh/posts/openai_ollama_mcp_flow.html
Somewhat ironically, Claude built the JS hooks for my SVG with about five minutes of prompting.