Ask HN: Textbook Regarding LLMs

[leoh]

Are there any textbooks that cover LLMs in depth? I’m looking to read papers (especially review articles) or a textbook instead of trying to piece everything together from papers or blogs or watching YouTube videos, despite there obviously being some good stuff out there.

[mindcrime]

I haven't seen any yet. Closest thing to that that I've seen would be something like one of these:

https://www.amazon.com/Transformers-Natural-Language-Process...

https://www.amazon.com/Natural-Language-Processing-Transform...

https://www.amazon.com/Generative-Deep-Learning-Teaching-Mac...

I wish I could give you a detailed review of all of those, but sadly while I have all three, I've been too slammed with other stuff to make much progress on any of them. :-(

And while it's not a book, there's that video series by Andrej Karpathy:

https://www.youtube.com/playlist?list=PLAqhIrjkxbuWI23v9cThs...

[kadushka → mindcrime]

I've read the second book (by Tunstall et al). It's excellent.

[nborwankar]

See Large Language Models - a Survey. Minaee et al https://arxiv.org/abs/2402.06196

[p1esk → nborwankar]

Thanks! Big names among authors, I’m going to read it.

[nborwankar → p1esk]

Here’s another one - it’s older but has some interesting charts and graphs.

https://arxiv.org/abs/2303.18223

[sandwichukulele]

someone here is writing such a book in public, here's the thread with the discussion

"Implementing a ChatGPT-like LLM from scratch, step by step"

https://news.ycombinator.com/item?id=39156778

you can view the github for the book as they work on it https://github.com/rasbt/LLMs-from-scratch

[TowerTall]

I have pre-ordered this book: Build a Large Language Model (From Scratch) https://www.manning.com/books/build-a-large-language-model-f...