Amazon has been known to ban accounts that have excessive return rates[1]. One part of making this an explicit, branded service available to Prime members is reassurance that excessive returns of clothing items won't count against you, nor cost you return shipping (it's only free in general for certain return reasons).
Perhaps I don't hit those rates, but I've bought lots of clothes from Amazon of late, not under the wardrobe program, and have paid 0 return costs and not visibly had my account limited in any way.
Given that they aren't charging me for returns, I saw no reason to buy it under the wardrobe plan.
I also return a lot of non-clothing items. In those cases I do pay return shipping of course. In the last year I'd guess my return rate is quite high, maybe 20%. But prior to that maybe 1%.
Congratulations to Amazon, I guess, for making paying prime customers proud to get worse service then they were already getting for free.