The concern many have with military involvement in encryption and other security standards is that while they want for themselves exactly what we want for ourselves, they have an internal conflict of interests due to the fact that they want a bit of the exact opposite for other actors¹ which, if those concerns win out and they have the influence to force through or block changes, this could lead security issues² that potentially affect you or I.
[1] They don't want the enemy (whoever that happens to be at the time) to be able to keep secrets as securely as they want to keep their own secrets.
[2] An accidental backdoor or side-channel vulnerability not fixed nor mitigations made known publicly, because it is useful once discovered and they have mitigations to protect their use, for instance.
That's why it's a good thing it's a FLOSS solution. Despite a track record of bad code and security, GPG has always been good-willed, and has progressed over the years. For sure having more eyes/resources on the code can only help. Also the recent Sequoia PGP project can prove to be an inspiration for further improvements.
[1] They don't want the enemy (whoever that happens to be at the time) to be able to keep secrets as securely as they want to keep their own secrets.
[2] An accidental backdoor or side-channel vulnerability not fixed nor mitigations made known publicly, because it is useful once discovered and they have mitigations to protect their use, for instance.