
Ask HN: Best books on AI? - mavmak
Which are the best books on AI, for somoone who doesnt have much knowledge on the topic. Mostly practical, and not academic.
======
arihant
You cannot really go practical in AI without academic rigor. You can do
recipes of what's already been done using a TensorFlow book, but that's how
far one can go. If one is serious in getting in AI today, a great way to do is
read the following books, in order:

1\. AI: A Modern Approach by Stuart Russell and Peter Norvig.

2\. Deep Learning by Ian Goodfellow and Yoshua Bengio.

It is amazing how approachable both books are for beginners, but you will be
diving a lot into academic stuff as you go along.

~~~
nextos
I love Paradigms of Artificial Intelligence by Peter Norvig much more than
AIMA, which I found excessively encyclopedic and shallow.

While some will argue it is dated, I think it presents many timeless ideas
that will get in vogue soon with little tweaks to their inference schemes.

Same for The Art of Prolog.

~~~
nabla9
It's Paradigms of Artificial Intelligence Programming (PAIP)
[http://norvig.com/paip.html](http://norvig.com/paip.html)

AIMA provides better introduction for wider area of subjects but PAIP is one
of most elegant and timeless books for both programming and old school AI.

~~~
jaddood
Why not both?

------
krosaen
Some on this thread have recommended Norvig's PAIP, but that's kind of an old
school AI book in that it focuses on heuristic search and logic (implementing
prolog in lisp at one point, very impressive stuff actually); but is lacking
any coverage of statistical machine learning, which is the approach that
underlies most of the cool stuff these days. It's still a great book, but I'd
instead recommend a path that focuses on machine learning:

\- The Master Algorithm: made for a general audience, gives you a lay of the
land

\- Python Machine Learning by Sebastian Raschka: gives you practical skills
using python, scikit-learn, numpy, jupyter notebooks, pandas etc. From zero to
kaggle in 4 chapters, goes deeper after that. Also goes into enough theory you
aren't flying completely blind.

After that, I'm afraid I think you do need to go "academic", if by that you
mean learning some of the underlying math to approach AI / ML from a more
rigorous probabilistic perspective. I'd recommend studying probability theory
and then working your way through Bishop's Pattern Recognition and Machine
Learning. After that a lot more doors open up too more specialized topics like
computer vision, reinforcement learning etc.

I've written up a lot more about this here:

[http://karlrosaen.com/ml/](http://karlrosaen.com/ml/)

~~~
JamilD
You don't need to focus on just one side or the other. AlphaGo wasn't made by
just hacking together a couple neural nets, it built on heuristic search,
MCTS, and the "old-school" AI.

I agree that basic statistics + Bishop's book is a great way to start getting
into machine learning -- but AI is a much broader field than that.

~~~
taktoa
"old-school" AI is just a pretentious name for algorithms/graph
theory/combinatorics, plus a bit of (now very outdated) PLT mixed in.

"new-school" AI (machine learning) is just a more pretentious name for
statistics/control theory/randomized algorithms.

~~~
nojvek
What is to say the human brain doesn't engage in graph combinotrics with some
learned heuristics.

Some of the old ideas with enough computation power are actually pretty
amazing.

~~~
goatlover
Because the human brain isn't a symbolic machine. It's a living organism made
up of cells trying to survive and reproduce in the real world. All our symbol
manipulation capabilities are built up on older abilities based on feeling,
experiencing, socializing, and moving about.

------
j_s
_First, read fucking Hastie, Tibshirani, and whoever. Chapters 1-4 and 7. If
you don 't understand it, keep reading it until you do.

You can read the rest of the book if you want. You probably should, but I'll
assume you know all of it.

Take Andrew Ng's Coursera. Do all the exercises in Matlab and python and R.
Make sure you get the same answers with all of them.

Now forget all of that and read the deep learning book. Put tensorflow or
torch on a Linux box and run examples until you get it. Do stuff with CNNs and
RNNs and just feed forward NNs.

Once you do all of that, go on arXiv and read the most recent useful papers.
The literature changes every few months, so keep up. There. Now you can
probably be hired most places. If you need resume filler, so some Kaggle
competitions. If you have debugging questions, use StackOverflow. If you have
math questions, read more. If you have life questions, I have no idea._

source: fizixer
[https://news.ycombinator.com/item?id=13890952](https://news.ycombinator.com/item?id=13890952)

 _FWIW, a "super harsh" guide to (learning) ML [1] was posted on reddit a few
days ago.

[1] [https://redd.it/5z8110](https://redd.it/5z8110) _

Edit: The entire Reddit discussion feels slightly similar to this one, if more
snarky. The first reply there also links all the resources listed above. I
don't really know enough to add anything.

~~~
radicality
Statistical learning is only one part of AI. If the person wants to learn
about AI, wouldn't starting with something like "AI: A modern approach" he a
better start?

------
iamlucaswolf
To iterate on what others said, but what was not emphasized enough from my
point of few:

AI _is_ academic (as a synonym for 'theoretical' and 'math-intensive'). Once
you look beyond purely symbolic AI, which proved to be infeasible as @curuinor
pointed out somewhere here, you will need to build up at least basic knowledge
in probability theory and linear algebra.

The path I'm following at the moment is a quite rigorous one and is outlined
here
([http://www.deeplearningweekly.com/pages/open_source_deep_lea...](http://www.deeplearningweekly.com/pages/open_source_deep_learning_curriculum)).

If you've never had any exposure to probability theory or statistics, I
recommend having a look at the course "MIT 6.041 Probabilistic Systems
Analysis and Applied Probability" taught by John Tsitsiklis at MIT (video
lectures are available through YouTube and MIT OpenCourseWare for free). Both
the course and Tsitsiklis' book are superb learning materials to get into
probabilisitc thinking.

Edit: Link was broken. Thanks to @blauditore.

~~~
colmvp
To preface, I'm currently learning several disciplines in tandem along a route
suggested by the link, so kudos to them for putting together a solid list of
resources.

Now, from the link: "Few universities offer an education that is on par with
what you can find online these days. The people pioneering the field from
industry and academia so openly and competently share their knowledge that the
best curriculum is an open source one."

On the one hand, it is true there are a ton of resources where the largest
cost is the time it takes to go through the learning process. And I'm
awestruck that research papers are so openly available and practitioners are
so willing to share their knowledge to others both in posting their books as
PDFs/HTML files and creating online courses.

On the other hand, how feasible is it for an individual to work on notable AI
companies/projects without a Masters or PhD in a related field? Can that gap
be crossed merely by becoming fluent in the various disciplines involved in
AI, before contributing non-formally academic research/experiments you've
conducted on your own?

~~~
Eridrus
The Google Brain Residency is a cool program for non-academics to get into
deep learning research, and you can always get into AI on the applications
side, but in both cases you're going to have to really try.

------
brogrammer2
Though it is an academic book, _Artificial Intelligence: A Modern Approach by
Stuart Russell and Peter Norvig._

The first chapter in the book provides a detailed analysis of how other
disciplines contribute to the idea of AI - from Philosophy to Psychology,
Biology to Computer Science. Makes for an interesting read, even for a non-
tech reader.

~~~
iamlucaswolf
This.

If you're also looking for a course that goes alongside the book, I highly
recommend UC Berkley's CS188 (you can find it at
[http://ai.berkeley.edu](http://ai.berkeley.edu)).

The lecturer Pieter Abbeel does such a good job explaining stuff and the
programming exercises are really neat.

Edit: Formatting

~~~
flor1s
The course is also quite easy to follow without buying the book. I love the
exercises in which you are programming an intelligent agent to move through a
maze. It reminded me of how we learned programming in university using Karel
The Robot.

This alongside Andrew Ng's Machine Learning course was my first exposure to
the field. [https://www.coursera.org/learn/machine-
learning](https://www.coursera.org/learn/machine-learning)

I can also recommend Sebastian Thrun's Artificial Ingelligence for Robotics
course: [https://www.udacity.com/course/artificial-intelligence-
for-r...](https://www.udacity.com/course/artificial-intelligence-for-robotics
--cs373)

------
dongslol
Get your feet wet:

* [http://cs231n.stanford.edu/](http://cs231n.stanford.edu/) (the course notes are excellent)

* [http://neuralnetworksanddeeplearning.com/](http://neuralnetworksanddeeplearning.com/)

* [http://www.deeplearningbook.org/](http://www.deeplearningbook.org/)

Use Tensorflow to train a few small neural nets. Move on to CNNs and RNNs.
Make sure you actually do this. By this point you'll have read a lot, and
retain none of it if you don't put it to use. Look at reinforcement learning.
Use the book by Sutton and Barto, the new edition:
[https://webdocs.cs.ualberta.ca/~sutton/book/the-
book-2nd.htm...](https://webdocs.cs.ualberta.ca/~sutton/book/the-
book-2nd.html) Read the first 4-5 chapters, then go online and read about Deep
Q learning, policy gradients, DDPG, etc. Then try to solve some problems on
OpenAI Gym.

Once you have an idea of the kinds of problems you can solve, and have a
couple you're interested in, go back and learn the foundational math, and
start reading research papers.

In general, start with modern books that mention deep learning. With older
books or high-level-overview books, you'll get frustrated when you see
something cool on /r/machinelearning and can't find any mention of it in the
book.

------
curuinor
AI is not a field where the practitioners can safely ignore the academic. A
huge number of people got into rabbit-holes in its history and basically
failed completely and wasted decades of their lives, in some cases.

~~~
palad1n
Anecdotal. Do you have specific records of such failures?

~~~
curuinor
Less symbolic failures.

There was a huge and abiding torrent of neural net stuff that dealt with
evolving topologies in late 90's. I see very little of it in any way shape or
form in industry or academia today, because it's a lot of computation for
basically no gain.

They thought that layerwise pretraining of neural nets was the way to go in
2006, before they realized that initializations, normalization, and better
activations was the better way.

A disgusting amount of why Watson won Jeopardy was because it could buzz
faster than Jennings and Rutter. Ain't that nice?

Lisp machines (ok that's symbolic again).

~~~
king_magic
Is pretraining really all that much of a failure? I haven't really found an
authoritative answer on whether or not pretraining is worth it these days.
Hinton's 2012(?) Coursera course still focuses pretty deeply on
generative/layer-by-layer pretraining with RBMs but I'm just not really sure
if that's fallen by the wayside today. Or maybe it's still useful only in
specific circumstances?

~~~
curuinor
Saxe Ganguli McClelland, 2013, about linear nets and orthogonal
initialization. But then, read Li Jiao Han Weissman 2017 (maybe preprint),
"Demystifying ResNet", which makes a nice claim about the niceness being
conditioning of Hessian at init.

Tldr: it's good conditioner but you can do better ab initio

~~~
king_magic
Interesting, I will check these out!

------
shoshin23
Superintelligence by nick bostrom. It's a book that explores how
superintelligence could emerge, the different ways it can take off and what it
means to us as humans. More importantly, the book takes on the difficult task
of figuring out ways to make sure the AI is safe and not land up in the wrong
hands. Pretty interesting read that doesnt really require technical know-how.

~~~
curuinor
There is lots of content in this book but there is no practical technical
content in this book. Interesting philosophy.

Much of AI philosophy is done by extremely non-practitioners. John Searle
can't code. Nick Bostrom came to coding extremely late in life. Geoffrey
Hinton and the other ex-PDP folks wrote some philosophy papers, though, which
are of interest if you like the philosophy.

------
hpvic03
If you want to get into actually implementing deep neural nets (what everyone
is calling "AI" these days), then look at TensorFlow and Keras.

This is a good getting started book for TensorFlow:

[https://www.amazon.com/TensorFlow-Machine-Learning-
Cookbook-...](https://www.amazon.com/TensorFlow-Machine-Learning-Cookbook-
McClure-ebook/dp/B01HY3TC54)

------
davidhunter
The Quest for Artificial Intelligence: A History of Ideas and Achievements by
Nils Nilsson

Gives a great run through of the history of AI research. Understanding the
approaches that have been tried before gives you a sense of why the state of
the field is what it is today. It is worth bearing in mind that AI research
expands far beyond computer science into psychology, philosophy, linguistics
etc.

~~~
flor1s
I think there is a pdf draft of the book legally available from
[http://ai.stanford.edu/~nilsson/QAI/qai.pdf](http://ai.stanford.edu/~nilsson/QAI/qai.pdf)

------
ju-st
"Python for Data Science For Dummies" by Luca Massaron & John Paul Mueller is
a very practical book on _machine learning_. In a real world scenario your
first obstacles will be learning how to use the programming language as well
as preparing data. Both topics are well covered in this book. The "learning
from data" chapter contains an introduction into basic machine learning
algorithms as well as ensemble learning. The book contains minor inaccuracies
but I think it's a good, practical start for a novice. It doesn't include
anything on neural nets however.

------
mnault000
Try this site:
[http://www.allitebooks.com/?s=artificial](http://www.allitebooks.com/?s=artificial)
And the books are free and downloadable :)

------
sehugg
You know, I have never found a "casual" book covering tree-search techniques,
from minimax to Monte Carlo Tree Search. Still relevant for game/agent AI
(AlphaGo used MCTS for example).

You'd think there would have been 100 "How To Make a Computer Chess Engine in
BASIC" books back in the 80s, and continuing to the present day, but I can't
find them. Lots of papers and online tutorials, and some stuff in textbooks,
but no accessible hands-on books.

------
akssri
Graphical models might also be something folks might want to consider. Ideas
from PGMs are often behind many advances in ML.

The canonical text is by Daphne Koller; a course I took used Martin
Wainwright's monograph though - the book is briefer and dives into the math
quicker.

[https://people.eecs.berkeley.edu/~wainwrig/Papers/WaiJor08_F...](https://people.eecs.berkeley.edu/~wainwrig/Papers/WaiJor08_FTML.pdf)

------
JamilD
Firstly, I don't think you can dive straight into coding without understanding
the fundamentals. AI is such a broad and rich field, and there's a lot you
need to know before you start.

It also depends on what you're going to focus on. Are you looking to implement
a game-playing agent? An object recognition algorithm? More of a logic focus?

If you just want Deep Learning and statistical methods, then Bishop's Pattern
Recognition and Machine Learning is a good start. Otherwise, Russel and
Norvig's Artificial Intelligence or Patrick Winston's similarly titled book
are great starting points. For more big-picture stuff,

Marvin Minsky's Society of Mind is great, and Hofstader's Gödel, Escher, Bach
is a classic too. Both are a lot less practical though, which seems to be what
you're looking for.

------
learningtool
A good suggestion seems to be not to read a lot of books, but to put intro
practice what you read and apply it to problems you need to solve, that way
you learn a lot more effectively

------
mrdrozdov
David Rosenberg's Machine Learning course is an excellent intro. Will give you
the foundations that you need for anything else. Has a few links for
additional resources, but the slides are mostly sufficient.
[https://davidrosenberg.github.io/ml2015](https://davidrosenberg.github.io/ml2015)

------
giardini
Sometimes its best to answer a question with a question.

Are you simply curious or is there something more pressing? For example, do
you want some light reading or have you perhaps been asked to implement
machine learning for your company?

Most answers here assume you want to jump into the ML swamp and start
analyzing your trove of "big data" ASAP. But is that so?

~~~
mavmak
I want to have a basic understanding on the topic, and implement something
small, maybe a side project.

------
tvalentius
If you are developing AI for games, then Programming Game AI By Example by Mat
Buckland are one of the best book for that :
[https://www.amazon.com/Programming-Example-Wordware-
Develope...](https://www.amazon.com/Programming-Example-Wordware-Developers-
Library/dp/1556220782)

~~~
LyutD
I can also vouch for this, it's a great introduction to AI in Games, however
it is just an introduction as the topic is very broad.

------
cowpig
AI: A Modern Approach (Russell, Norvig) and Deep Learning (Goodfellow, Bengio)
have been mentioned already.

I'd also recommend:

Godel, Escher Bach: an Eternal Golden Braid

by Douglas Hofstadter. Might not be exactly what you're looking for (it's all
over the place, touching music theory, math, art, philosophy...), but it's fun
and enjoyable to read. Also very dense.

------
danielmorozoff
Once you get your feet wet, the first year PHD course book for good theory is
Pattern Recognition and Machine Learning by Bishop.

But it could be a bit too theoretical - it provides a foundational
mathematical framework and got me thinking about problems in a better way.

------
kruhft
Paradigms of Artificial Intelligence Programming (PAIP)

One of the best books on AI and Programming ever.

~~~
curuinor
Unfortunately, there is not a drop of numerics in that book. It's a good book
for learning about symbolic AI. There is, to a solid first order
approximation, zero symbolic AI in a system like, say, Google speech
recognition.

~~~
kleiba
Not surprisingly, I would say, since I wouldn't count speech _recognition_ as
an AI task.

~~~
Scarblac
Nothing is counted as an AI task after someone finally manages to do it.

~~~
kleiba
I anticipated that argument, but I wouldn't have considered ASR an AI task
even before it was first attempted.

~~~
bstamour
Any particular reason why? I feel that the problem of mapping sensory inputs
like sound etc onto internal concepts for reasoning is an important part of
AI.

~~~
kleiba
My personal reason is that I don't consider the output of speech recognition
to be "internal concepts for reasoning". In a Spoken Dialog System, this task
is typically performed by a subsequent component that does natural language
_understanding_ (rather than recognition).

------
payne92
The challenge I've found with books is the space has been moving so quickly in
the past 10 years. By the time the book is out, the methods described in it
are no longer state of the art.

~~~
endisukaj
That's why it's important to understand the underlying math.

------
tatoalo
I'd spend my money on these two:

\- The Emotion Machine by Minsky

\- Superintelligence: Paths, Dangers, Strategies by Nick Bostrom

------
blojayble
Any recommendations for a book focused on AI in game development?

~~~
fapjacks
I've got a book called "Game Programming Patterns" by Robert Nystrom which
goes into this. It's not advanced, but it's very good.

------
MidoAssran
Bishop, Hastie, Shai Shalev-Shwartz

------
pieterp
The Second Machine Age

------
halis
Just watch the movie with Jude Law. We're close, we're so close!

------
vasira
The society of mind and the diamond age !

------
visarga
Imagine you want to learn the Roman alphabet. Which book should you use? Any
book. There are so many good books and courses that it's almost useless to
select. Don't worry about selecting the initial book, just use any course or
book in the beginning, and later when you will know exactly what fits your
needs, you will be able to fine tune.

