I have always been concerned about that as well. Although aren't you somewhat safe depending on if you only open it with the corresponding pdf software? I thought the "only" vulnerability was the software it's used with and if you double click it or not.
Yes, though theoretically a single PDF could contain multiple exploits for multiple different PDF reader programs, I believe. Not sure if that's ever actually been observed, but it seems plausible in theory.
Most people use Adobe Acrobat, though, I think, so a PDF exploiting an Acrobat zero-day can be safely assumed to be pretty effective.
