The only way I could really see tax rates being a top-tier freedom issue is as a restraint on government: the Grover Norquist theory that low taxes starves the government of the ability to do things, so in turn reduces spending and government involvement in everyone's lives. But I haven't seen that over decades. We lowered taxes in the 1980s, and that didn't stop government spending from more than doubling in that decade. We lowered taxes again in the 1990s and 2000s, so the government couldn't afford to invade Iraq or Afghanistan, and we invaded both anyway, borrowing money we didn't have to spend on the invasion (and we're still spending money we don't have on both), and then we spent more money we didn't have on bailouts in late 2008. Basically it seems over at least 70-80 years or so, tax revenue hasn't really been a meaningful constraint on spending.
I take tax-rates much more seriously. Taxes are a direct line to slavery--obviously a 100% tax rate would be complete slavery. My current 1/3 tax rate makes me 1/3 slave--maybe a fair trade for being 2/3 free, but a trade nonetheless. Unfortunately, as you succinctly pointed out, tax rates of late have had little relation to spending, but in the long-term input = output, so as a young individual I see taxes and spending as two sides of the same coin.
To the extent our government has shown restraint, I think that is due to people making their concerns known and visible, as well as some good old fashioned American luck. Politicians will tend to acquire as much power as the people let them.