To be fair, the poor shouldn't be racking up credit card bills. That's just adding fuel to the fire. A world existed before credit cards became commonplace.
You're talking about something entirely different. Let me take another crack at explaining. You're a grocery store. Some (large) fraction of your patrons use credit cards. Rather than charge just those patrons a 1-3% surcharge to cover Visa's fees, you simply raise your prices across the board by 1-3% so that you can make enough margin to survive.
A poor person, who can't (or shouldn't) use credit cards comes in to buy food. They now pay the full-price (already marked up 1-3%) -- essentially paying that as a "tax" that wealthier patrons, with good cash-back cards, get back later as a refund.
