The issue is that PDFs are typeset for print. That means things are designed to look best on the paper size such as using serif fonts and avoiding widows, orphans, and rivers. Multiple page sizes would be multiple times the cost. e-books lose all that typesetting and it doesn't always result in the best experience, but HTML/CSS is catching up in that regard.
What I was thinking of is that output to a PDF could still be primarily targeted for print, but have the software generate at least 2 pdf documents, one for letter/A4 size paper, and another that is sized closer to 4x6 with minimal margins. Then use deduplication so that it doesn't make the file size much bigger.
