Jailing the offender also prevents them from killing/raping/etc because now they are in jail. Really a false comparison here. I think all the countries of the world overwhelmingly have voted in favour that this cost is worth it. The countries that did not, made it very hard to execute prisoners anyways and very rarely use it.
That works out to about $0.14 per year per citizen, or $0.37 per household (assuming all my math is more or less correct)
I agree with your analysis that it's not a lot of money. I will ponder this.