I understand and welcomed the initiative when it was first discussed. Meanwhile I've implemented it on a couple of domains and hat it running for all of 2018. Last month I removed it all again (the sites still have a responsible disclosure link but not at a standardized URI).
It was a massive waste of time for me to engage with a group of "unknowns" -without prior relationship- who now had a channel to fast-track into my inbox (and got my attention).
The volume of mails was anywhere from 5-20 emails per week which is a lot if you need to go through the points in every mail but you don't have any idea if the entity behind the poor-security-report is even skilled. So you start in good-faith that the report they gave you (e.g. you tell yourself that your own time analysing it is justified "they're probably competent", "they know what they're doing otherwise they'd have a different job", "maybe somebody will get lucky this time", or depending how many bad reports you already read you might be down to "even a blind chicken finds a kernel of corn every now and then")
it seems I ended up having endless discussions with people who automated the whole thing: they crawl the web for /.well-known/security.txt URI and if the find it, automatically start-up metasploit or burp-suite and then send you the canned report while asking you to fix these "serious problems".
Yet if you quiz any of these "researchers" deeper about individual items in their canned reports you get nothing but blank stares, incompetence and attempts to weasel out: "but burpsuite says that it is an error and you should correct it", ...
Initially I went through every email patiently. I tried to engage these guys on why they strongly felt it were vulns or 0days (LOL). I knew what they reported was canned and they never questioned the context or the errors themselves. None of them were people that I would hire or trust to give me good advise. Advise wasn't always just bad. Often they have a whole consulting company sitting behind them who not only "fix your problems" but also migrate your whole stack to Drupal or Wordpress or some such nonsense.
If I look at the other end of the spectrum (infosec-memes and "thought leaders" on twitter) I get people complaining that some customers just don't know how bad their security is and they even dare to ignore their reports or worse (question them on their authority in whether this is a bug or not). The whole thing is the deaf leading the blind here. I get what security.txt was trying to improve because there is/was a real issue for people to find a point of contact. But I do not think security.txt is in any way useful. And it's a total waste of time and money for small companies and bigger companies alike. (if you're bigger you get more attention = more reports but that doesn't improve quality - it only adds more work on your end because now you have N-people (instead of 1 or 2) discussing these constant "non-problems".
Reminds me of all the “dubious security vulnerability” stories on the Old New Thing blog, e.g. https://blogs.msdn.microsoft.com/oldnewthing/20111215-00/?p=... where MSFT needs to investigate obviously bogus “security problems” just to be sure they didn’t miss anything important.
This is a rather common tactic, they spam out automated security reports, and request a payment for their "services" from those who reply and engage further with them.
On a related note, I often feel uncomfortable reading about a responsible disclosure, and find coordinated disclosure to be a much more balanced term in the context of disclosing security vulnerabilities.
I haven't explicitly tried to enforce encryption, but probably the drive-by style reports would require extra steps that their automation might not handle. So probably a good first filter. But then I'm still no wiser since the ability to use pgp isn't a qualifier regarding knowledge of the engineer or quality of their report.
It seems that the underlying problem is that those that do good work in this space don't scan the web to find new customers/leads to pitch their service in shambolic ways. And the skiddies who want to make a quick buck will outnumber the good who might accidentally have ended up on your site (because they like your product etc).
the noise/quality ratio in the whole approach is just too big for this to work well in practice. I'm still waiting for the recruitment industry to catch up with the practice and use the security.txt as a sink for people who want to be added to a list of experts that will be contacted when "the company is ready to do a full security assessment post-MVP". I realize this would be fraudulent and I'm not advocating for it - just saying that fake-job offers aren't uncommon either so this will just be a question of time.
What you're describing is a lot like the bug bounty program I ran for a previous employer. It was mostly low-effort scans and "reports" templated from something a big company had made public once. No understanding of if not using HSTS was actually a vulnerability, just the expectation of burp -> report -> $$$.
There were a handful of genuinely good contributors, but probably under 10% of reports.
If you don't mind sharing, what kind of ballpark amount of traffic were you getting for those domains that generated that many emails per week on the subject?
just north of 500K visitors/day (that's across all sites but they were anyway operated by the same org and in 1 specific industry niche).
I conjecture that with these drive-by style reports you won't see a huge fluctuation in reports and traffic might not have a huge effect. I got no proof for this but if my hunch is right then there is a limited number of bad-actors that operate in this market. And if they all operate by copying their model from one another then the traffic should be similar regardless of how your popular domains are. Would be cool to have some data on this.
Thanks for this post. Gives more than enough reasons not to use security.txt.
Having it in a place which requires human interaction to form a contract makes more sense. Cant automate that.
It was a massive waste of time for me to engage with a group of "unknowns" -without prior relationship- who now had a channel to fast-track into my inbox (and got my attention).
The volume of mails was anywhere from 5-20 emails per week which is a lot if you need to go through the points in every mail but you don't have any idea if the entity behind the poor-security-report is even skilled. So you start in good-faith that the report they gave you (e.g. you tell yourself that your own time analysing it is justified "they're probably competent", "they know what they're doing otherwise they'd have a different job", "maybe somebody will get lucky this time", or depending how many bad reports you already read you might be down to "even a blind chicken finds a kernel of corn every now and then")
it seems I ended up having endless discussions with people who automated the whole thing: they crawl the web for /.well-known/security.txt URI and if the find it, automatically start-up metasploit or burp-suite and then send you the canned report while asking you to fix these "serious problems". Yet if you quiz any of these "researchers" deeper about individual items in their canned reports you get nothing but blank stares, incompetence and attempts to weasel out: "but burpsuite says that it is an error and you should correct it", ...
Initially I went through every email patiently. I tried to engage these guys on why they strongly felt it were vulns or 0days (LOL). I knew what they reported was canned and they never questioned the context or the errors themselves. None of them were people that I would hire or trust to give me good advise. Advise wasn't always just bad. Often they have a whole consulting company sitting behind them who not only "fix your problems" but also migrate your whole stack to Drupal or Wordpress or some such nonsense.
If I look at the other end of the spectrum (infosec-memes and "thought leaders" on twitter) I get people complaining that some customers just don't know how bad their security is and they even dare to ignore their reports or worse (question them on their authority in whether this is a bug or not). The whole thing is the deaf leading the blind here. I get what security.txt was trying to improve because there is/was a real issue for people to find a point of contact. But I do not think security.txt is in any way useful. And it's a total waste of time and money for small companies and bigger companies alike. (if you're bigger you get more attention = more reports but that doesn't improve quality - it only adds more work on your end because now you have N-people (instead of 1 or 2) discussing these constant "non-problems".