It would be awesome if researchers documented not only the vulnerabilities they find, but also all the software they analyzed without any findings, and how they analyzed it. It would save the community from re-doing the same analysis and maybe suggest what else other researchers might have missed. (This is not really different from the real need to publish negative results in science.)
Actively developed open source projects are a moving target, thus it'd be good to indicate which versions were reviewed or tested.
If a vulnerability were later found in a version that was checked then it would be good for the OpenSSF to review how it was missed. Hopefully that'll let them be better in the future.
I also like the idea of them documenting their methodology. Open source the process of finding vulnerabilities in open source projects!
Root cause analysis ("How did this thing happen in the first place, does it exist anywhere else, and how can we prevent it from happening again?") should be a core part of our work in Alpha-Omega, and we should include this information in our publications.
To the larger point -- the idea of "open sourcing the process for finding vulnerabilities in open source" captures a lot of the work already being done in OpenSSF, but there's a lot more we can do. (If you're interested in helping us with this, we'd love to have you!)
>It would save the community from re-doing the same analysis
I vaguely disagree. Secure software is not a well defined state so security analysis is not a process with an end condition, so theoretically, you can never stop looking.
But documenting failed attempts and methods is still worthy because duplicate work may still teach a lessons.
Hi! I co-lead the Alpha-Omega project, and I'd be happy to answer any questions. We won't have all the answers, though - it's still very early days in this project, and we're going to learn a lot in the coming weeks and months.
How can folks get involved today? First, you could participate in one of the OpenSSF working groups. The "securing critical projects" group is working on understanding how to identify critical projects. The "best practices" group is working on writing "leave-behind" material that we can share with projects when we report vulnerabilities via Omega -- e.g. here's how to enable code scanning, branch protection, 2FA when publishing to npm or PyPI, etc. The "identifying security threats" working group (the one I lead) talks about Alpha-Omega, but we'll soon be splitting that out to a separate meeting. OpenSSF working groups are open to anyone.
We're going to hire a few people to join the core project team, and while the list hasn't been finalized, I expect it will include a few engineers, security analyst/researchers, and a project manager. We're working on finalizing this and should have jobs posted shortly.
For Alpha, we're still working on the model we'll use to engage with larger open source projects -- this will probably be contract work, focusing on a "menu" of options (e.g. security code audits, threat modeling / design review, security bug triage, proposing security fixes, improving security processes, etc.), but nothing has been finalized yet. We want to build a relationship with project maintainers, understand where (and if) we can help, and then do our part. We know that all projects are unique, so we want to meet them where they are.
For Omega, we're building a large-scale analysis platform, looking for new, critical vulnerabilities across at least 10,000 projects. I expect the hardest part of this will be to reduce the noise common to most security analysis tools - this is what the engineers we hire will be focused on -- continually tuning and improving the analysis platform.
We have an information session scheduled for February 16th, and we'll do our best to answer everyone's questions. There's a registration page linked from https://openssf.org/community/alpha-omega/. You can also join the OpenSSF slack and explore - there is plenty going on.
Reading between the lines, is it correct that at this point you are mainly interested in corporate participants rather than individuals who want to work on this a few hours per week?
We need corporate/similar participants to help pay the bills. We have enough funding to get off the ground, but we want to be able to scale up quite a bit. Corporate participants might have other benefits, like giving us access to specialists in certain areas.
But we also need/want individual contributors or volunteers. The best way to help right now is probably through the OpenSSF working groups, but we're actively discussing ways that Alpha-Omega could leverage volunteers and individuals more directly. If you have ideas, please get in touch -- I hope to have a better answer here in a few weeks.
