Not to mention that if you pay taxes to a government that has a military presence anywhere you're much more directly supporting killing, if you want to look at it that way.
You really should look at it that way. The armed forces are directly representing your democratically expressed will. We should feel responsible, because we are. If the government is not interpreting our will accurately, we need to change the representatives.
The war where we blow up wedding parties full of children. I doubt there is popular support for that. We are responsible for every action, however popular the overall war.
But taxes aren't optional. You can't just choose opt out of them because you don't agree with what the money's being spent on. You might as well say you're complicit just by having being born in a country with a military.