The font sizes/styles don't bother me. What is a pain are the voting arrows - they are so tiny that i sometimes tap the wrong one, and there's no way to undo one's vote.
About 30% of the time I fat finger and click on the story link instead of the comments on my iPad. I don't know if that would be fixed with a bigger font or a bit more padding between the story and the comments link.
If you decide to do this, please consider using a larger font size on the iPhone, too. For my browsing habits, that would be greatly appreciated, though perhaps others find the font size on the iPhone to be OK.