I highly recommend Build a large language model from scratch [1] by Sebastian Raschka. It provides a clear explanation of the building blocks used in the first versions of ChatGPT (GPT 2 if I recall correctly). The output of the model is a huge vector of n elements, where n is the number of tokens in the vocabulary. We use that huge vector as a probability distribution to sample the next token given an input sequence (i.e., a prompt). Under the hood, the model has several building blocks like tokenization, skip connections, self attention, masking, etc. The author makes a great job explaining all the concepts. It is very useful to understand how LLMs works.
But this is missing exactly the gap which OP seems to have, which is going from a next token predictor (a language model in the classical sense) to an instruction finetuned, RLHF-ed and "harnessed" tool?
hey sry didnt saw ur comment :) well actually there is a installer allready includet you can just download the exe. virustotal hash included. otherwise : yes you have to build the exe dfepending on the package json
"Who art Henry?" was never grammatical English. "Art" was the second person singular present form of "to be" and it was already archaic by the 17th century. "Who is Henry?" would be fine.
In some languages you can put a second person conjugation next to a noun that might otherwise use third person verbs, and it serves as implying that you are that noun. I'm not sure if older forms of English had that construct. I think many Indo-European languages do.
The part of the lord's prayer that says "our father who art in heaven" is kinda like this - father is linked to a second person conjugation. You could remove some words and make it into "father art in heaven", which you claim is ungrammatical. I'm skeptical that it was.
“who art in heaven” is a grammatical relative clause because the subject of the verb is the relative pronoun “who” which is second person in that context. You can still get this kind of thing in modern English, for example “I, who am a farmer, will be happy” is grammatical because the relative pronoun “who” is first person there. That doesn’t mean it would be grammatical to say “*A farmer am happy” and it wouldn’t have worked with art either.
Conceivably it’s grammatical if Henry is vocative and the pronoun is dropped colloquially, like “Who art [thou], O Henry?” but it’s a stretch.
I think the further back you go in Indo-European grammar, the more common the thing you are describing becomes. For me it's less of a question of if English did this, and more like how far back you need to go.
Today, even ignoring the dated conjugation, "who art in heaven" or "who are in heaven", does not make sense. We would switch it into the third person.
The problem is the subjunctive mood of the word "art".
"Art thou" should be translated into modern English as "are you to be", and so works better with things (what are you going to be), or people who are alive, and have a future (who are you going to be?).
Those are probably the contexts you are thinking of.
I gave it a try a few months ago. The first thing that I searched for was the REPL. Comparing it with Python, I prefer ipython over irb. It would be nice to see some improvement on DX on the REPL side
Open a requirements.txt and a package.lock.json next to each other and compare. Then you will know the answer to the question what npm, cargo, and others are doing better than pip. Oh, did I sneek a ".lock" in there? Damn right I did.
[1] https://www.manning.com/books/build-a-large-language-model-f...
reply