If I care about binary integrity, I wouldn't involve an app store. https://signal.org/android/apk/ is a perfectly cromulent way to get your hands on the APK file directly from the source.
pull the apk from your phone with apktool. Compile Signal reproducibly with their instructions. Use the diff.py tool they provide and check for the message that confirms the APKs match.
What does E2EE potentially give you? A promise, which does not involve trusting the service provider, that messages can only read by the recipient.
What does making the app closed source take from you? The freedom of requiring trusting the service provider = facebook