If your application is a well-behaving application in a well-behaving environment, then it will most likely run in a reasonable amount of time. Furthermore, if you leak a few mutexes, it's not really a big deal for most applications.
I think one of the implied points of the article is that you should not rely on finalizers for anything time sensitive or critical. For instance, if you MUST close a file handle or database connection, you should never rely on a .NET finalizer to do it because it's not predictable when or if it will run. You should explicitly close it (most likely by implementing and calling Dispose().)
It depends whether or not mutexes are "heavy" resources. If making them and not using them is free, then it's OK to treat them like memory and wait for garbage collection to do its thing.
