It's all a matter of trust in the end. Even with Open Hardware, the production line, or the transport of the hardware to your doorstep could be compromised. The same way that even Open Source software can be compromised if no one is paying attention or you are being specifically targeted.
That's all true. But if intelligence agencies are constrained to physically breaking in to your premises to bug you, that gets hard to scale up to pervasive mass surveillance.
