In a two-party system (a necessary de facto in a single-member-district plurality system), failure to vote for a candidate is mathematically identical with a vote for the opposing party.
That's nonsense. By that logic both sides can claim "They didn't vote FOR us so essentially the voted for the other side", which means that by not voting you voted for both.
