On a related note, tonne (i.e metric tonne) is widely used, though not generally in the scientific arena. This is partly for historical reasons, but I think there's an element of it being a little awkward to apply prefixes to the kilogram, as one must multiply the "kilo". Nonetheless, we should exploit the full power of SI prefixes and use "megagram" instead. With two short syllables rather than one long syllable it takes about the same amount of time to say and is far less ambiguous.
This works well elsewhere; Fat Man wasn't 21 kilotonnes of TNT, it was 21 gigagrams.
There are three major reasons why Tarsnap pricing is defined in terms of picodollars per byte rather than dollars per gigabyte:
Tarsnap's author is a geek. Applying SI prefixes to non-SI units is a geeky thing to do.
If prices were listed in dollars per GB instead of picodollars per byte, it would be harder to avoid the what-is-a-GB confusion (a GB is 10^9 bytes, but some people don't understand SI prefixes). Picodollars are perfectly clear — nobody is going to think that a picodollar is 2^(-40) dollars.
Specifying prices in picodollars reinforces the point that if you have very small backups, you can pay very small amounts. Unlike some people, I don't believe in rounding up to $0.01 — the Tarsnap accounting code keeps track of everything in attodollars and when it internally converts storage prices from picodollars per month to attodollars per day it rounds the prices down.