I've dealt with a lot of bugs in FreeBSD, and most of them were found by non-security-experts. The people finding said bugs didn't necessarily know enough to be able to exploit the bugs, or even enough to be certain that there was a bug; but there were a lot of "this code looks weird" reports turning out to expose vulnerabilities.
I don't think you need to be a security or cryptography expert to find vulnerabilities the NSA might try to exploit.
While you're right that fewer issues means fewer potential exploits, all you're really doing is getting rid of weaknesses that should never have been there in the first place, but you're doing nothing to address the systematic undermining of encryption standards and hardware and software implementations of them.
To use a metaphor, it's great that you're making sure your windows are shut but the locksmith may have sold you a deliberately weak front door lock.
When speaking of encryption and "systematic undermining of encryption standards" you have to be aware of what that means.
What people fear is the deliberate weakening of the algorithm such that it can be broken (implementations can be fixed, the real nightmare being subverted algorithms), where broken doesn't mean what many people think it means.
Say you've got a 256 bits key, to brute force it you'd normally need to try 2^256 combinations of bits, right? Well, if you found a flaw that permitted you to brute force it in 2^200 attempts, then that's a massive improvement and you can consider the algorithm broken, but guess what - that's still exponential complexity with the issue being solvable by simply making the key bigger. And there are people working on these standards that are not on NSA's payroll and outside the US jurisdiction, people that aren't idiots, so bigger flaws than this aren't feasible.
This is why, even if they've introduced subtle flaws in current standards, that doesn't mean they have the capability of breaking the encryption - e.g. it is possible that they are able to break RSA-1024 keys, but RSA-2048 is an entirely different problem. And RSA-4096 keys will likely stay unbreakable, unless a huge breakthrough happens.
People give them more credit than they deserve: yes, they have cash and authority and can coerce companies and individuals and they can also plan for the long term, etc, etc, but let's be realistic about their abilities.
That probably works in FreeBSD, but I can't imagine being welcomed by Linus if you report that "this code looks weird." Although it might be worth risking the anger, just so one would have a personal rant to hang on the wall, similar to how people save checks from Knuth.
He ought to welcome it, since I get the impression his approach to debugging relies partly on spotting code that looks weird. Of course whether he would is another matter.
I don't think you need to be a security or cryptography expert to find vulnerabilities the NSA might try to exploit.