The downside of libcaca is that it makes several presumptions that aren't necessarily true outwith the world of IBM PC Colour Graphics Adapter compatibility.
For example, it has a tendency to abuse boldface black on black as a way to get a dark grey colour, making it not work so well on terminals where boldface really is a switch to a boldface font rather than brightening the foreground colour. On such terminals, black on black text is invisible, boldfaced or no.
Sadly, it actually works internally in terms of a 4x16x16x16 ARGB colour system, and could work quite well with 88-colour or 256-colour terminals, and very well with terminals that have 256x256x256 RGB, were it not for the fact that most of the information in the internal colour representation is thrown away by the ncurses and CGA-like layers that are built on top of it.
https://github.com/cacalabs/libcaca