It's fine until you do almost any arithmetic using the index. Unsigned arithmetic is rarely what you want, and it quietly converts all of your signed types to unsigned giving you large positive values instead of small negative ones. Maybe this isn't a common use case for most people, but it came up all the time when I was doing signal processing algorithms.
Using ssize_t is pretty reasonable, and you're not going to overflow ssize_t for loop indices except in some very pathological cases on a 32 bit architecture.
If the arithmetic argument doesn't convince you, here's another one: write a correct loop (using size_t) which traverses your array in reverse order. Here is a broken example which would be fine with ssize_t: