It does not provide answers to every exercise--maybe 10% tops--but a lot of the exercises are small and should not be any problem. These are often in a group, where he takes something that would be one hard exercise in another book and breaks it down almost to the level it would be if were part of the main text, leaving just small thing for you to fill in as exercises.
There are a few recurring themes throughout the exercises, where he applies the material of the chapter to some specific application in several exercises (e.g., error correcting codes if I recall correctly), and subsequent chapters continue with those themes in their exercises.
For Linear Algebra, "Linear Algebra Done Right" -- way better than my college lin alg course.
For "general mathematics", I like books that read almost like novels to really grasp the "why" of mathematics, so these are more to embellish your general understanding of "what is the point" type questions -- so things here like Euclid's Window come to mind, and "An Imaginary Tale: The story of square root minus one" will help explain complex numbers in more detail than you ever cared. Reading about the history of mathematics and the writings of some of the greats, like Rene Descartes, Lehonard Euler, Gottfried Wilhelm Leibniz, Issac Newton (the amazing thing is all these greats lived within a century of each other)