For a more theoretical/less applied approach that's still relevant to computer science/programming, check out Benjamin Pierce's Basic Category Theory for Computer Scientists. It's a very good introduction to the parts of Category Theory that are useful for [theoretical] CS, and delves into semantics of programming languages, types, and concurrency a fair amount.
I can't find something there that would help me implement better systems. What I do see is an attempt to shoehorn a mathematical model onto certain programming paradigms in order to be able to say that these paradigms are instances of the model. In short, a lot of talk about abstractions (or "abstract nonsense") but very little of engineering significance. Please let me know if I'm missing something by listing concrete examples of how category theory can help me with engineering problems.
Disclaimer: I'm not a mathematician.
You don't personally need to know about the Yoneda lemma to benefit from a better designed programming language whose type system was informed by algebras of types, allowing it to leverage exhaustive pattern matching.
With that said, learning about techniques used in functional programming that have their origin in category theory (maps, folds, algebraic data types, monads, monoids, etc) can allow you to think about how to solve problems (even in non-functional languages) in ways that can be more "natural" to the problem, in that the abstractions used resemble the problem description more closely. That ability to decompose difficult problems in new ways can definitely make you a better engineer.
I think of CT as the study of "composability" in its purest form. Mathematicians are finding that this language is really good at translating domain-specific knowledge between disparate fields, not unlike the babelfish from the Hitchhiker's Guide.
FWIW, some researchers are apparently finding interesting applications in the development of biological models. John Baez has talked about applications in this direction via control theory .
Mathematics is most commonly described in the language of set theory, where objects are constructed from basic building blocks to create bigger blocks. This is much in the same way as algorithms are described in something concrete, say a programming language.
At some point, certain high-level constructs started to appear, where the same high-level behaviour was exhibited in otherwise completely disconnected fields. Explaining what is going on then started to become messy, for example take following quote from Hatcher's Algebraic Topology concerning covering spaces: "This is often called the Galois correspondence because of its surprising similarity to another basic correspondencein the purely algebraic subject of Galois theory". The problem is that in order to understand this, the reader has to understand two fields which are not at all connected and furthermore, the relationship looks nothing alike in the language of set theory.
Category theory was invented as a formal language which could describe such high-level behaviour. In this language this language we would could now say that there is an "contravariant equivalence of categories", which helps if you've seen it before and doesn't help at all if you haven't.
So category theory is massively useful in understanding the similarities between fields, and can be used to transfer understanding from one field to another. It it also useful for understanding more abstract fields where the intuition must be gained by analogy with something more concrete. It doesn't do much in itself, and I don't think it makes much sense to study the mathematical aspects of category theory without any good examples to apply it to.
What does that have to do with engineering? I'm not expert here, and haven't yet read the linked book, but one could hope that certain similarities could be described in terms of this new language. Then concepts which now only live on as the expert experience of working practitioners, might be written down and hence taught directly rather than indirectly. One could hope that certain problems have common solutions which can be re-used across fields.
That's the essence of why category theory works nicely with pure functional programming, but beyond that simply fails to be useful. In particular, there has been virtually no progress in useful categorical models of parallel computation.
As a software engineer, category theory is sometimes useful in API design as well; although more often than not, your API is constrained by the existing ecosystem too much for category theory to give you much benefit (and, frankly, simple enough that category theory is overkill).
So for instance, one of the big applications to CS is in type systems, which describe what kinds of expressions/values can be put together, and in what ways, to create valid programs.
(But if you are into that kind of proving, than there are a languages a bit like Haskell, like Agda for example, that revel in those proofs.)
So far: introduction/preorders, monoidal preorders/wiring diagrams, categories/application to database schemas.
For those reading on a ebook, the book sources are available on its arxiv page https://arxiv.org/abs/1803.05316 so you can build a custom pdf for your device.
A mathematician's idea of "applied" and a normal person's idea of the same bear little resemblance.
Given the centrality of linear algebra to applied mathematics (even in places where you wouldn't first expect, like the theory of regular expressions and finite automata), this seems like a perfectly sensible coinage.
(Even though I must admit I get my back up at the omission of categorical proof theory, it's perfectly natural as a sales pitch -- type theorists don't need to be convinced of the utility of category theory!)
Where can I read more about this?
But I would also like to know what other resources Neel has to recommend.
I've tried to get into Category theory a few times but couldn't keep my motivation high enough once things got too technical. Anyone know how category theory could be useful for someone in ML or robotics?
I'm interested in learning Category/Set Theory to better understand the underlying math of how they work.
I have very little experience with ML/Robotics, but CRDTs might be useful for distributed learning, or coordination between independent robots.
CRDTs are relatively new so there may not be many classes yet that teach them. If you want to learn about them, I recommend you first watch this video: https://www.youtube.com/watch?v=OOlnp2bZVRs&list=WL&index=56... and then read this paper: https://pages.lip6.fr/Marc.Shapiro/papers/RR-7687.pdf . The video explains a lot of the math in a very easy to digest way and will make the paper much easier to understand.
So if anything, it might be MORE applicable for composing ML networks.
But so far, the best ML libraries can be found in impure / imperative languages like Python.
I think given a language with AD you already don't need to have a separate language for graph building and my understanding is that's one of the goals for the Julia team and I've seen other nice looking packages for AD in other languages
I'm also very excited about being able to design neural nets using ideas from functional programming. I love the Keras API but I don't think it's going to be the obvious API a couple of years down the line.
Now the serious answer. Category theory is about abstracting away the details of mathematical domains so that fundamentally parallel proofs and constructions can be seen to be essentially the same. It also allows you to create constructions that allow problems in one field to be transformed into problems in an apparently unrelated one where you might find them easier to prove.
The result is that in a new field, you recognize the category and suddenly have a whole bunch of results, and some important constructions to look at. Such as the tensor product.
And in existing fields you can do things like use homotopy theory to turn topology problems into algebra problems, where they are often easier.
"A florb is brazzle of carnatious snoggles. A florb is itself a snoggle and so can be a fnizzle of another florb."
This is actually the definition of what is normally called (in English) a SET, under the following mapping of vocabulary:
florb = set
brazzle = collection
carnatious = distinct
snoggle = object
fnizzle = member
Category theory is about extracting the fundamental structure of mathematical definitions so that these kinds of equivalences can be studied with mathematical rigor.
To add a bit of info, there is a system named Attempto Controlled English which uses a subset of Natural Language to express axioms. It is available here https://github.com/chrisdone/ace and here https://github.com/Attempto/APE
How would one associate patterns of axioms with appropriate relations in Category Theory ?
To me, the interesting thing about category theory is that it draws out the significance of relationships in a system. We often spend a lot of time thinking about the things, themselves, in a system.
- What are their properties?
- What do they mean?
- What is their identity?
In category theory, almost all the significance of individual things is erased. Instead, you often group things up into sets and other structures, and then call the whole set an object. Then you think about the existence of morphisms from one kind of object to another. A morphism is kind of like a function, in the sense that something goes in, and something else comes out. Except that we're not concerned about the properties of what's going in and coming out, so the function body doesn't really matter. A morphism is something more general than any specific function. It's more like a function declaration--specifying that something does exist without saying anything concrete about what it does.
So, you're operating on a level of abstraction that removes basically all unnecessary details about things and relationships, other than existence and some basic rules for how those relationships can be composed to form new ones. As it turns out, there's a whole lot you can say and prove about categories once you know a bit about their structure.
In my opinion, the real power is that any given system can often be "projected" into categories in a multitude of different ways, and depending on the structure of those categories, you can say apply proofs from category theory to say interesting things about that system.
Going further beyond that, category theory concerns itself with how categories relate to each other, by considering categories themselves as objects and relationships between them as morphisms.
Why does this matter to programmers?
Well, we often want to design systems that are predictable, no matter what the actual identity of the data that traverses them. This is abstraction, and we do it on many levels, all the time. We also want to chain together our abstractions. This combination of abstraction and composition is something category thoery is well suited to analyze.
I'm probably off here in some important ways, but this is my layperson's takeaway from spending some time trying to understand what all the fuss was about.
It kind of makes a few things apparent.
Tiles would be the morphisms and objects would be the number spots on the tile ends.
So the first insight: morphisms is the actaul object of study. The “objects” of a category is just a consequence of how the morphisms may, or may not, compose.
My first hurdle when learning things was being distracted by trying to understand what objects where, some times they would be types, sometimes sets, and it was never really clear what their cardinality would be. As if the theory hid important details that you had to keeps track of implicitly. But it doesn’t, objects really are inconsequential, they are just there to be a place for morphisms to meet, nothing more. Just like the number of dots of the domino tiles. We could have used color instead, and the tiles would compose in exactly the same way.
Second intuition, there’s no computation or anything else going on. Just as you say about the functions. The only thing of concern are the composability of those things called morphisms, be they functions or domino tiles.
In the lecture series linked a few places in this thread, Bartosz makes a fantastic point of this when deriving some properties of surjective and bijective functions using only composability.
Among the more readable category theory resources I'm currently reading.
Monads, functors, arrows, etc are just patterns that show up a lot in functional programming that also happen to have analogues in category theory. This isn't too surprising, because category theory basically started from mathematicians trying to formalize common patterns in disparate areas of math. Categories pop up all over the place in math, and math of course is the main abstraction that we formally model just about anything in reality.
Given how ubiquitous they are, I think it's worth it to have a working, informal knowledge of category theory. It doesn't take serious time or study to do that.
I'm not sure if taking a course in CT is the most efficient way to learn how to apply such concepts in computer programming, thought. Maybe someone with more experience can give it's opinion.
Its concepts are so abstract that many different things can map to them. So if you can prove that something is a group, or a monoid, or whatever, then by the rules of category theory, you can now prove certain characteristics of that thing. This could wind up being a vital step in a longer proof.
When this happens, it usually feels like you moved your proof forward without actually gaining any new understanding (because the concepts are so abstract). Therefore, this is often referred to as "general abstract nonsense," but in a tongue-in-cheek manner, because you actually did derive concrete value.
Yes, really. I wasn't making that up. Automatic differentiation is a page or two of code given enough categorical background.
There's also the regular misinterpretation that mathematical objects like, e.g. monoids, are well understood through CT. That's pretty false. There's the "intermediate level" of abstract algebra where you can study these objects and the world they live in much more directly and immediately. There's some real benefit from learning some parts of abstract algebra. Trying to learn CT in order to learn abstract algebra is complete overkill, though.
In both of these cases, my advice boils down to "just learn the things you're interested in directly as opposed to 'via' CT". That's because CT is really what happens when you boil and boil and boil away these different fields and are just left with some very universal ideas. Abstract nonsense.
If you're a mathematician and these concepts are your daily bread and butter... hey, you might still not want to learn CT because it's better to just work with what you're studying directly.
Okay, objections and disclaimers over. The thing you can learn from CT which is interesting and valuable is the very powerful, universal perspective of "if you want to understand something, look at how it relates to other things". Category theory is absolutely focused on this idea and most of its insight is that this idea, taken in its extreme, is extraordinarily far reaching.
Again, you can probably learn this more practically by studying where it shows up in something you actually care about. It won't be a theorem there, but instead just sort of "in the fabric" of the field, a major technique used in proofs. CT is nice because it unified those "folk theorems" from all over mathematics.
More relevantly, in programming the CT perspective lies close to the FP perspective where "transformations" of data and "pipelines" are given center stage as opposed to "objects" and "procedures". It's also embedded in ideas of domain specific languages and both concrete and abstract interpretations, thereof.
Most likely, you want to learn the actual techniques of interest and then maybe later just learn a bit of CT to get a sense for "how they all connect". I think there are opportunities for CT to influence your programming if you were to become familiar with it, but they're still kind of fuzzy today.
Finally, if you want to learn CT I recommend you learn abstract algebra first via Aluffi's "Algebra: Chapter 0". It's secretly a CT book, but is not trying to be so abstract. You can also learn a lot from Schanuel and Lawvere's "Conceptual Mathematics" which, I feel, is one of the better books for establishing the "category theoretic POV".
Here they present three PDFs with a number of assignments of varying difficulty: https://ocw.mit.edu/courses/mathematics/18-s097-applied-cate...
If you find that being able to answer a high enough number of assignments you know that the course is for you. If not, the lectures may still be of interest, but it is much less likely the less of the assignments you think are worthwhile for you.
PS: Question 4 in problem set 3 is quite (in)famous :-) Just the headline:
> Question 4. A monad is a monoid in a category of endofunctors (harder/optional)
So, if you want to be one of the few people who actually understand that sentence when it is quoted as a joke (usually in programming forum whenever monads come up), this is your course.
I don't think the particular approach taken by the instructors is a good one, so I replied "this is a course" (not necessarily the course to take).
A lot of the applications are weird and a bit contrived. A more traditional approach would suit many people better.
Category theory is a relatively new branch of mathematics that has transformed much of pure math research. The technical advance is that category theory provides a framework in which to organize formal systems and by which to translate between them, allowing one to transfer knowledge from one field to another. But this same organizational framework also has many compelling examples outside of pure math. In this course, we will give seven sketches on real-world applications of category theory.
It is more focused on programming, but just enough to be relatable, it’s still mainly about CT
(Actually enjoyed the whole set, and the follow up)
Or are you asking for examples of useful application such that category was absolutely necessary?
And just Haskell programming in general involve composition Effects are monads and program composition is composition over kleisli arrows .
- Hey, wait! You don't tell me why we need to learn this concept
Yes, most of my lecturers use Math to teach Math. It's not useful to me, or i can't learn that way.
A wish is just a wish though.
That's how life is. Some (most) times you don't know what tools you will need in the future.
“Smart” boards — expensive, and high drawing latency makes writing on them uncomfortable. Also, text is hard to read from a distance. (They may be better for accessibility.)
Dry-erase boards — boards and markers are expensive. Markers generate a lot of waste.
Chalk — cheap, naturally occurring (and safe to dispose of) material. Easy to read at long distances.
Also, don't put your fingernails on the chalkboard...?
If you have ever used one, you'd know they keep using them because they prefer it.
They are also easier to read from a distance compared to a dry-erase board.
The most amazing thing though, is that person or persons unknown goes round and cleans the boards at the end of every day. Even in the holidays.