Hacker News new | past | comments | ask | show | jobs | submit login
Reverse Derivative Categories (arxiv.org)
2 points by morningseagulls on Oct 19, 2019 | hide | past | favorite | 1 comment



It's rare to see a CS paper, let alone a CS paper with category theory, with 7 authors!

It's fairly mathematical, so I'm going to quote the abstract for some context as to why you might care about this:

The reverse derivative[0] is a fundamental operation in machine learning and automatic differentiation. This paper gives a direct axiomatization of a category with a reverse derivative operation, in a similar style to that given by Cartesian differential categories for a forward derivative. Intriguingly, a category with a reverse derivative also has a forward derivative, but the converse is not true. In fact, we show explicitly what a forward derivative is missing: a reverse derivative is equivalent to a forward derivative with a dagger structure[1] on its subcategory of linear maps. Furthermore, we show that these linear maps form an additively enriched category with dagger biproducts.

[0] This is usually called the reverse mode of differentiation in the automatic differentiation literature: https://en.wikipedia.org/wiki/Automatic_differentiation#The_...

[1] https://ncatlab.org/nlab/show/dagger+category




Join us for AI Startup School this June 16-17 in San Francisco!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: