
Reverse Derivative Categories - morningseagulls
https://arxiv.org/abs/1910.07065
======
morningseagulls
It's rare to see a CS paper, let alone a CS paper with category theory, with 7
authors!

It's fairly mathematical, so I'm going to quote the abstract for some context
as to why you might care about this:

 _The reverse derivative[0] is a fundamental operation in machine learning and
automatic differentiation. This paper gives a direct axiomatization of a
category with a reverse derivative operation, in a similar style to that given
by Cartesian differential categories for a forward derivative. Intriguingly, a
category with a reverse derivative also has a forward derivative, but the
converse is not true. In fact, we show explicitly what a forward derivative is
missing: a reverse derivative is equivalent to a forward derivative with a
dagger structure[1] on its subcategory of linear maps. Furthermore, we show
that these linear maps form an additively enriched category with dagger
biproducts._

[0] This is usually called the reverse mode of differentiation in the
automatic differentiation literature:
[https://en.wikipedia.org/wiki/Automatic_differentiation#The_...](https://en.wikipedia.org/wiki/Automatic_differentiation#The_chain_rule,_forward_and_reverse_accumulation)

[1]
[https://ncatlab.org/nlab/show/dagger+category](https://ncatlab.org/nlab/show/dagger+category)

