> If the IRS doesn't think I've paid my taxes, I can go to prison.
No, if the IRS doesn't think you've paid your taxes, they can attempt to collect the unpaid taxes.
OTOH, if the Department of Justice can prove in a criminal prosecution that you've violated the criminal provisions of the tax law, you can go to prison. If the IRS thinks you've done that (which is different from just not paying your taxes), they can refer you to the DoJ for prosecution.
This change to IRS guidance doesn't change the criminal provisions of the tax law that would apply were the government to prosecute you, therefore, the change is not a change to criminal law, retroactive or otherwise.
So you may end up with penalties and interest, but you have a high probability of avoiding prison absent doing something stupid like trading emails with your broker on how you plan to evade taxes.
It's only a criminal matter if the IRS thinks you broke the law deliberately. If it's the result of crappy record keeping or something you didn't understand they'll usually tack on some penalties and interest and call it a day.
I know a guy who didn't even file for five or six years. Eventually he got a lawyer and made a deal which had him paying some fraction of his original tax liability.
They'll put you in jail if you make them, but they don't really want to put you in jail. They just want your money.
Oh, this guy? After getting his sweet deal he didn't actually pay them. I think he may end up in jail.
Doesn't that make this particular case a criminal one?