Tests can't catch race conditions in multithreaded code. Now that I told you what the tests can't catch, can you imagine writing tests for that specific thing?
I've written tests around multithreaded code, but they typically catch them in a statistical manner - either running a bit of code many times over to try and catch an edge condition, or by overloading the system to persuade rarer orderings to occur.
tsan will catch a bunch of potential race conditions for you, under the condition that you run it somehow. How to make sure it's run? Well, add a test for the relevant code and add it to your tsan run in your CI and you'll certainly catch a bunch of race conditions over time.
This has saved me a bunch of times when I've be doing work in code with proneness to those kind of issues. Sometimes it will just lead to a flaky test, but the investigation of the flake will usually find the root cause in the end.
I’ve written tests to do exactly that, by adding carefully placed locks that allow the test to control the pace at which each thread advances. It’s not fun but you can do it.
MAI Basic Four Inc. was a relatively succesful computer manufacturer that sold business computers that used BASIC as the development language for business applications: https://en.wikipedia.org/wiki/Basic/Four
In 1983, when I started using Unix, I went through the manuals for Unix System 7 trying to learn about every program that came with the system. So I tried SCCS, but at that time I wasn't developing software as part of a larger team so I decided it wasn't useful to me.
Many years later I bought the book "Applying RCS and SCCS: From Source Control to Project Control", half-read it and decided I still didn't feel it was something useful to me. It wasn't until 1998 or 1999 that I started using CVS, then a few years later switched to SVN, an finally to Git (between SVN and Git I spent a few years using Unity's Asset Server).
"...can't reproduce the human mind because it's obvious that is not how we work" <-- that's a very strong assumption, without any concrete evidence to support it
"stole a huge quantity of copyrighted material" <- nobody stole anything, even if it's eventually determined that there was some form of copyright infringement it wouldn't have been stealing
I read that post recently and it felt prescient to someone who has not been deeply involved in ML
Even the HN discussion around this had comments like "this feels my baby learning to speak.." which are the same comparisons people were saying when LLMs hit mainstream in 2022
I had forgotten it's existence by now, but I remember reading this post all those years back. Damn. I also remember thinking that this would be so cool if RNNs didn't suck at long contexts, even with an attention mechanism. In some sense, the only thing he needed was the transformer architecture and a "fuck, let's just do it" compute budget to end up at ChatGPT. He was always at the frontier of this field.
I tried to find the where I heard that Radford was inspired by that blog post, but the closest thing I found is that in the "Sentiment Neuron" paper (Learning to Generate Reviews and Discovering Sentiment: https://arxiv.org/pdf/1704.01444.pdf), in the "Discussion and Future Work" section they mention this Karpathy paper from 2015: Visualizing and Understanding Recurrent Networks https://arxiv.org/abs/1506.02078
The book that just came out, "Understanding Deep Learning", is an excellent overview of the current state of AI: https://udlbook.github.io/udlbook/
Read that first, then to keep up to date you can follow up with any papers that seem interesting to you.
A good way to be aware of the interesting papers that come out is to follow @_akhaliq on X: https://twitter.com/_akhaliq
My current fave book to introduce DNNs is "Deep Learning: A Visual Approach" by Glassner. He's crystal clear, covers a lot of ground, and the book is up-to-date on everything but LMMs, which is moving so fast that no book could keep up.
Another book which also seems to be very good is "Deep Learning, Foundations and Concepts". It is coming out soon and you can already preview it on-line at https://www.bishopbook.com/
Prince's other writings have been outstanding. Based on the relative opacity of Bishop's venerable PRML, I'd turn to Prince's book before I would Bishop's newest.
