I'll try to reply to the frustrations of the author of the OP. I'll give some hi...

graycat · on Nov 13, 2017

Part II

Super simple view: In math, there are sets with elements. That's the logical foundation of essentially all of current pure/applied math. The details are in Zermalo-Fraenkel axiomatic set theory assuming the axiom of choice. E.g., can construct from sets a set that looks like the real numbers we knew about in the 9th grade. Soon define ordered pairs and, then, functions. After that, a huge fraction of everything is functions. The proofs -- as actually written but without the crucial intuitive ideas that permitted finding the proof -- are all essentially just symbol substitution as in basic logic and Whitehead and Russell. Warning: Any math you write for publication should be easily translated back to just sets and symbol substitution; that and nothing else is the criterion. If you also want the proof to be readable by humans, there is more. E.g., in the proof might mention one by one each theorem assumption and where it gets used.

(iv) Learn linear algebra. Really the subject grew out of Gauss elimination for solving systems of linear equations -- with some additional attention to numerical stability (e.g., partial pivoting, double precision inner product accumulation, and iterative improvement) -- that's nearly always still the way to do it. It's fun to program a good Gauss elimination routine, e.g., just in C.

There see clearly that any such system of equations has none, one, or infinitely many solutions. Later will discover that the set of all the solutions is an affine subspace, that is, a vector space plus some one vector; that is, a plane that does not pass through the origin. And will discover that the left side with the unknowns is a linear function -- big time stuff.

So, that start was a small thing for a huge future.

Next up, we consider n-tuples of real (or complex, here and always in linear algebra) numbers. Then we see how to make the n-tuples a vector space. They are the most important of the vector spaces.

But should also see the abstract (with no mention of n-tuples) definition of a vector space because (a) the n-tuples are the most important example, (b) even when working with just n-tuples often need the more abstract definition (especially for subspaces, which are often the real interest, e.g., hyperplanes in curve fitting and multivariate statistics), and (c) there are other important vector spaces that are not just n-tuples (in signal processing, sets of random variables, wave functions in quantum mechanics, solutions to some differential equations, and much more).

So, learn about linear independence and bases (essentially coordinate systems).

Learn about inner products, distance, angles, and orthogonality. See generalizations of cosines, e.g., the Schwarz inequality, and the Pythagorean theorem.

Learn about eigenvalues and eigen vectors -- those eigen vectors are often the most important ones in applications, e.g., your favorite coordinate axes.

Then for the crown jewel, learn about the polar decomposition and, thus, singular values, principal components (e.g., data compression), the core of the normal equations in statistics, etc.

There is a remark in G. Simmons that the two pillars of mathematical analysis are linearity and continuity. The superposition in physics is essentially linearity. In applied math, linearity is the main tool, the key to the land of milk and honey. Well, a good course in linear algebra is a good start on linearity. In particular, those linear equations solved with Gauss elimination are linear as in linear transformations in, right, linear algebra.

Then, sure, that version of linearity takes one through much of all of applied math, e.g., Fourier analysis, the fast Fourier transform, X-ray diffraction, Banach space, Hilbert space, the classic Dunford and Schwartz, Linear Operators, etc.

So, a Banach space is just a vector space where the scalars are the real or complex numbers, there is a norm, that is, a definition of distance, and the space is complete in that norm. Complete is what the rational numbers are not but the real numbers are. Or, complete means that a sequence that appears to converge, that is, converges in the weaker sense of Cauchy, actually does have something to converge to and does. E.g., in the rationals, the approximations to more and more decimal places of pi have nothing to converge to but in the reals do.

In theorem proving, nice to have completeness. But, sure, computing knows next to nothing about completeness because we compute essentially only with rational numbers. So, can do a lot of work without completeness. Indeed, in applications, often we are just approximating, and the rationals can get as close as we please to pi!

Banach spaces are not trivial or useless: E.g., based on the Hahn-Banach theorem, there is the grand applied math dessert buffet

David G. Luenberger, Optimization by Vector Space Methods, John Wiley and Sons.

A Hilbert space is a Banach space where the norm comes from an inner product.

E.g., the set of all real valued random variables X such that E[X^2] is finite form a Hilbert space. That the space is complete for those random variables is nearly mind blowing; actually the proof is short.

(v) There remains Baby Rudin, Principles of Mathematical Analysis.

There see calculus done with essentially full care, as theorems and proofs. So, again, get lessons in how to write theorems and proofs on the way to being a good mathematician.

The main content of the book is just showing that a real valued continuous function defined on a closed interval of the reals has a Riemann integral.

The key is that that interval is compact. So, learn about compactness, which is of quite general usefulness.

Then with compactness and continuity, have uniform continuity. Now the doors to grandeur start to open: That the Riemann integral works is a short proof. And, later in the book, get the three epsilon proof that the uniform limit of continuous functions is a continuous function (was a question on one of my Ph.D. qualifying exams -- from baby Rudin, I got it!).

Compactness is so powerful that it is nearly the same as just finiteness -- there's a famous, old paper on that.

Well for a positive integer n and the set of real number R, in R^n a set is compact if and only if it is closed and bounded. Now we are cleaning up the Riemann integral and a lot of associated stuff.

At the back, Rudin gives a nice, short definition of a set of the real numbers that has measure zero (without really getting deep into measure theory) and, then, shows the Riemann integral exists if and only if the function is continuous everywhere except on a set of measure zero. Nice. Now an exercise is to find a function that is differentiable but whose derivative is not Riemann integrable. Might look in Gelbaum and Olmstead, Counterexamples in Analysis.

Also the later editions of baby Rudin cover the exterior algebra of differential forms. That material is of interest in differential geometry, some applications, and general relativity.

That's an overview of what baby Rudin is about.

Go through that book carefully and will come out with (a) some good knowledge of the "principles" of the analysis part of pure math and (b) much better skills at doing math derivations, definitions, theorems, and proofs. If you want to write and publish new proofs for, say, AI, baby Rudin is one of your best mentors, maybe your Fairy Godmother?

For being an applied mathematician, sure, you already guessed, and you were correct, that essentially always in practice the Riemann integral exists; so, why sweat the details? Okay, then just use baby Rudin to learn about compactness, continuity, uniform continuity, measure zero, and more on how to write proofs. And, really, focusing just on compactness, continuity, etc., can pull that off in a few, nice weekends, maybe just one. Then look at that result near the back on the uniform limit of continuous functions is continuous to see how to do such work.

Sure, Rudin discusses compactness, etc. on metric spaces. Well, easily enough, the set of real numbers R is also a metric space! And for positive integer n, so is R^n.

So, why say metric space instead of, say, just R^n? Well, first, the theory is cleaner because a metric space has so many fewer assumptions than R^n so that can see more clearly just what assumptions make the results true. Second, maybe some fine day all you will have is just a metric space and, then, can still use the results -- don't hold your breath while waiting for a significant application, either pure or applied, with a non-trivial metric space that isn't also much more! Or, proving the stuff in a metric space is no more difficult, more general, and maybe, and actually, more useful.

Or, maybe math had the results in R^n and then invented a metric space just to have a place to have just enough to make the results true! So, a metric space was invented to have the least assumptions needed for proving the results; so, what came first were the results, and the metric space definition came later. Maybe!

graycat · on Nov 13, 2017

Part III

(vi) Continuing on, there is the subject of measure theory. That was from H. Lebesgue, a student of E. Borel, right, in France near 1900.

They were correct: They improved on the Riemann integral. Don't worry: Whenever the Riemann integral exists, the integral from Lebesgue's measure theory gives the same numerical answer.

So, why a new (way of defining the) integral? Two biggie reasons:

(a) For a lot of theorem proving about integrals, e.g., differentiation under the integral sign, clean treatment of what physics needed from the Dirac delta function (right, there can be no such function, but measure theory has a good answer), definition of an integral of several variables, interchange of order of integration in iterated integrals, tying off some old, loose ends in Fourier theory, the deep connections between integration and linear operators, and more, Lebesgue's work is crucial and terrific stuff.

(b) Lebesgue's integral is much more general than the Riemann integral, and that generality is crucial, especially as the foundation for probability theory.

Now, for what Lebesgue did:

First, he developed measure theory. That's essentially just a grown up theory of area like you have known about since grade school. E.g., given a set, in the real numbers, R^n, something more complicated, or something fully abstract, the measure of that set is essentially just its area. With the generalization, sure, can have some sets with measure infinity, negative measure, complex valued measure, etc. But on the real line, with the usual measure, Lebesgue measure, the measure of an interval is just its length, and you already know about that.

But Lebesgue measure is darned general: It's tricky to show that there is a set of the reals that is not Lebesgue measurable, and the usual proof uses the axiom of choice.

So, for measure theory, there is a measure space with three things: There is a space, just some non-empty set, say, M; there is a collection of measurable sets, subsets of M called, say, S; and there is a measure m. Then the collection of measurable sets S satisfies the simple, essentially obvious axioms we would want for area and, thus, is a sigma algebra of sets. Then for each set A in S, its measure is the real (or complex) number m(A). We ask that m have the properties we want for a good theory of area. It's just a grown up version of area. The theory is not trivial; it was tricky to get all the details just right so that have a good theory of area, that is, so that area works like we want it to.

E.g., for the space, can have the set of real numbers R. For the measurable sets, have the intervals and, then, all the other sets needed to have a sigma algebra (a short proof shows that this definition is well defined). Then the Lebesgue measure of an interval is just its length, and the measure of all the other measurable sets are what they, then, have to be (need some theorems and proofs here).

So, for probability theory, a probability space is just a measure space (each point in that space is one experimental trial); a probability, P, is just a positive measure with maximum value 1; an event A is just one of the measurable sets; and the probability of A is just its measure P(A). What we want for probability is already so close to a theory of area that, really, we have little choice but just to follow what Lebesgue did. That's what A. Kolmogorov observed in his 1933 paper.

Second, with the foundation of measure theory, Lebesgue defined the Lebesgue integral.

So, what is being integrated is (usually) a function taking real or complex values. The domain of the function is a measure space.

Then for the integral, say, in the case of a real valued function, we partition on the Y axis, that is, in the range of the function instead of in its domain. So, we don't have to do Riemann-like partitions of the domain of the function and, thus, the domain can be much more general.

As the first step, we only integrate functions that are >= 0, and we do that by approximating, right, again with essentially rectangles, only from below the function, not both above and below as for Riemann. Here the domain of the function can be the whole space, e.g., the whole real line. We don't care about either continuity or compactness.

For a function that is both positive and negative, we multiply the negative part by -1 and integrate the two parts separately. If at least one of the two results is not infinity, then we subtract, and that's the integral.

A random variable is just such a function, and its expectation is just its integral.

Summary

Don't feel like the Lone Ranger; not everyone knows this stuff. E.g., from all I've been able to see from quantum mechanics, the wave functions are differentiable. Then I'm told that the wave functions, wondrous, are also continuous. From baby Rudin, of course they are continuous; every differentiable function is continuous! Then I'm told that the wave functions form a Hilbert space. Well, I can see that they can be points in a Hilbert space, but they can't form a Hilbert space because the continuous functions won't be complete.

In elementary probability, it is common, e.g., for finding the expectation of a Gaussian random variable, to integrate over the whole real line. Tilt: Baby Rudin defines the Riemann integral only over compact sets. So, can use an improper integral, and then they want to differentiate under the integral sign. Tilt again -- the needed theorems are not so easy to find. So, really, this common stuff, in probability, physics, etc. of integrating over the whole real line or over all of R^n is using the Lebesgue theory where have clean theorems for such things.

That's your overview and your work for the weekend. You are now permitted two beers and half a pizza but only if you have someone you like a lot for the rest of the pizza and some more beer. Can substitute Chianti for the beer. But no more math permitted; maybe a movie, but not math!