It feels really crummy to be accused and convicted of an "offense" by an algorithm, especially without any recourse.
I once had my account with a major cloud provider terminated for "violating our terms of service". After contacting support, they then claimed that someone had gained access to my credentials.
What evidence did they have? None. I just updated a VM's metadata too frequently (about once a minute). This tripped an ML model, which caused them to automatically terminate my account and send an automated email saying that I had been a bad boy.
This took down a key part of my business for about 5 hours (while I navigated my way through layers of customer support and ultimately temporarily moved this functionality to another cloud provider). Customers were not happy.
It took about 2 weeks and multiple support tickets for the full story to come out. I got them to refund a few months of charges (amounted to several hundred dollars at the time) and restore my account. There was never any recognition that they made a mistake.
I get that companies need to resort to automated means to handle fraud or abuse. But they should also own up to it, add some humility in their automated outreach to customers ("our automated system has detected possible X" instead of "you are guilty"), provide clear escalation paths to talk to a human, and provide a way to "shield" your account (identity verification, upfront deposit of $X, etc) that forces them to contact you before any enforcement action.
In my case, I upgraded to a paid support plan ($100+ per month) in the hopes that their system will be a little less trigger happy with my account in the future. I don't use support at all, it's purely a lame form of insurance that may or may not actually protect against anything.
When these algorithms get it wrong, it completely sucks. And since no tech company has any semblance of customer support, you're completely hosed.
With respect to Facebook: I posted a shop vac last April for $50. I got a message that I was banned from using marketplace for "violating community guidelines."
However, if you believe this happened in error, you could request a review.
So I did! And was denied. I did this process a few more times and each time was denied. Once I requested a review for the third (or fourth?) time, I received a message that said "Unfortunately, your account cannot be reinstated due to violating community guidelines. The review is final."
I have no idea what happened.
So now, I can't use Facebook Marketplace because of some stupid error in their algorithm that can't be ever appealed. Which is a bummer, because I've legitimately found some good electronic finds on there (and have been able to offload things I don't have use for).
Meanwhile, their algorithms for advertising and marketing useless stuff to us are just perfect. A passage from Yuval Noah Harari's book, "Homo Deus: A Brief History of Tomorrow" highlights this:
> A recent study commissioned by Google’s nemesis – Facebook – has indicated that already today the Facebook algorithm is a better judge of human personalities and dispositions than even people’s friends, parents and spouses. The study was conducted on 86,220 volunteers who have a Facebook account and who completed a hundred-item personality questionnaire.
> The Facebook algorithm predicted the volunteers’ answers based on monitoring their Facebook Likes – which webpages, images and clips they tagged with the Like button. The more Likes, the more accurate the predictions. The algorithm’s predictions were compared with those of work colleagues, friends, family members and spouses.
> Amazingly, the algorithm needed a set of only ten Likes in order to outperform the predictions of work colleagues. It needed seventy Likes to outperform friends, 150 Likes to outperform family members and 300 Likes to outperform spouses. In other words, if you happen to have clicked 300 Likes on your Facebook account, the Facebook algorithm can predict your opinions and desires better than your husband or wife!
We need Habeas Corpus for tech. Companies should be obliged to tell you what your violation was, and you should have the opportunity to challenge the judgment in which you are able to present arguments and evidence.
Additionally, I think there should be a right to download your data after being banned, whether or not the ban was fair.
I've experienced many similar incidents – I've been running multiplayer game servers on these "discount" server providers for a while now. They receive umpteen requests per day about spamming, hosting pirating/other illegal content, botnets, etc with only an associated IP address.
Vultr has a nasty habit of forwarding these directly to the current holder of the IP address with a dire warning that your account will be shut down within 24h if no action is taken, regardless of the timestamp of the complaint. Abusers just create & destroy servers frequently frequently to acquire fresh IP addresses to host malicious content on. It became a morning routine of copy+pasta responses to these emails to keep the servers online.
This is like arresting someone for the crimes committed by a previous apartment tenant. An IP address can be spoofed and it isn't equal to 1 person for all time. It will take a class-action tort judgement against the flimsy, lazy correlation people to get the message that their actions are causing harm.
I don't think it was me flagging it, I missed the post saying it was cleared up, I think I just stopped at the post with the abuse report link which is right above it.
How is this not slander? Making false claims to disrupt someone's business seems like the perfect case of slander and should be treated as a legal matter in my opinion.
It might be, but at least in the US slander is rarely prosecuted criminally. That leaves civil action, but that's limited to how much damage the OP suffered as a result. Given that it's a small personal blog with little traffic/monetization, any legal costs will outweigh any winnings, if any.
It's more likely to be the tortious of interference ( https://en.wikipedia.org/wiki/Tortious_interference ) but even then, the kicker for almost all non-criminal cases is that you have to have monetary damages or there's really nothing much you can do.
Now if net neutrality had teeth, maybe you could angle that way.
You are describing the actual malice standard which only applies to public figures. Also, an accusation of that type would be considered "defamation per se" which would mean that there is not a need to prove damages. According to my layman understanding.
am I also fine if I negligently write and keep using an algorithm to detect and report people for being molesters when I know full well that it's actually shitty and I regularly receive reports that it's resulting in false positives?
That's facebook's situation. Even though they've got hundreds of billions and could easily afford to, they don't want to pay humans to verify these things and they know that innocent people are being hurt by their terrible lazy code, they just don't care.
I suppose you'd have to prove that they knew the algorithm was bad. They'll just say "we're trying our best, but it's hard". And the truth is that it probably is hard. The question is whether knowing that your algorithm is imperfect is equivalent to intent to harm.
Sure there is, they're a bigcorp and did what they did. Don't let companies get away with crimes you and I wouldn't just because they're too disorganized.
It's a personal blog (which didn't even go down), so the damages are zero or negligible.
And it's probably not defamatory at all, because the bar for what's "reckless disregard" for truthfulness is quite high. "We run a commercial service that identifies malicious websites and 0.01% of them are unintentionally false positives" is not gross negligence.
edit: (To defend myself because I'm getting piled on in votes)—I agree that this shitty behavior, but I just don't agree that there is or should be a legal remedy for it. You all have way too heated hot takes, HN. There's no life-changing injuries here for which five-figure attorneys would be a proportionate investment—this is, no offense, just some blog which did not go down, and everyone will have forgotten all about this by Monday. This is just not the kind of conflict the legal system is meant to resolve!
Let's not downplay it, the damages are reputational - not (necessarily) financial. They were falsely accused of engaging in criminal activity in the hopes of getting their services provider to terminate their relationship with the author.
This trend of hiding behind algorithms and percentage figures needs to stop, yesterday.
I don't see any tangible reputational damages here. No one seems to have read this except Digital Ocean (?)
- "They were falsely accused of engaging in criminal activity"
And Netcraft sincerely believed that this was a criminal-activity website. That is not libel! Untrue statements without malice (or a very severe amount of negligence) are not defamatory in the US. You are allowed to accuse people of things—that is protected speech.
DigitialOcean is a hotbed for hosting spam/abusive domains, spam/abusive email hosting (used to host temp emails for creating fake accounts on other platforms), and hosting abusive VPNs/proxies.
The good thing is, this is very easy to detect when things are hosted on DO.
The bad news is, companies are harsh on DigitalOcean and will have some level of false positives.
I have a postfix server I've hosted on DO for over 15 years. It is only on the UCEPROTECT block lists that block all of DO, everything else is clean. I use it mostly for receive-only and use Amazon SES to send anything where deliverability to customers is important.
What's a solid low cost ($5/month) VPS provider that is below the radar for this kind of broad brush blocking based on guilt by association? Thinking of making a move, but I kind of hate to give up the 15 years of good reputation on my IP address and starting over.
Is this DO specific? Aren't there other hosting providers with competitive rates, or is there something about DO that makes it especially good for this sort of thing?
If all you want are virtual servers (==droplets), DO is one of the more expensive options around. Of course substantially cheaper than AWS/Azure/GCP, but much more expensive than Hetzner/OVH/Netcup or anything on lowendboxes.com).
What DO has is some of the best content marketing around (great tutorials on server stuff) and scale. The first attacts beginners (which includes most of the spammers and hackers), the second puts them on people's radar.
The fact that there's an entire consulting industry around helping people figure out very basic pricing for very basic tasks on AWS should be a wake-up call that their documentation and calculators are awful.
Then again I'm sure there's no incentive to improve it, since if for whatever reason you were laid off from AWS... you could have a lucrative consulting gig the next day helping people with the blindingly obvious consumer pain points you refused to resolve.
I'm pretty sure one of the major reasons AWS is useful to large companies is that it removes prices and billing as far as possible from the place where you actually create resources. This enables people to just do stuff without the usual red tape involved in buying something.
If you want to see pieces as you do stuff you can go to the Google Cloud
While it’s less applicable to the VPS options, the open ended pricing of AWS and GCP gives people starting out or with very little monetary slack a lot of heartburn. And you do still need to have an account that covers the whole kahuna.
Reasonable or not, the company you keep is treated as a thing.
As far as I can tell, Google Blogger is still a pretty reasonable site for a blog and some static pages. And it’s free. Of course you may not like that option for various other reasons.
False equivalence. There are many free/low price options for 'personal sites and blogs' that aren't sleazy hosting companies if cost is your key differentiator. Now, if it's "I will only accept an option where I can paint my bikeshed the exact color I want, with any brush I choose to use", then yeah, you've painted yourself into a bit of a corner options-wise and may end up as collateral damage. TANSTAAFL.
The reality is many people don't know the "reputation" of DO, see it as a cheap hosting site with good tutorials, and start using it.
They then get caught in the crossfire, but the correct complaint would be to DO not to the people who are taking the relatively reasonable and easy way out of "ban this part of the Internet, it sucks."
Sure, but the OP made out like the only options for personal stuff were cheap, shitty hosting providers, or expensive ones. And that's not true. We can't make people do research, but don't equate "lack of due diligence" with "lack of options".
It’s pretty easy to guess what features (either manually made or AI based) the phishing detector saw:
1. “Facebook” and “login” in the URL
2. URL redirect
3. “Facebook login”, “password login, “forget password” etc in text body
4. The quoted email from Spotify sounding close (in vector space) to phishing text.
5. A link to Facebook settings, followed by a series of steps; these instructions say to log in to a non-Facebook url using your Facebook email
All of these together was probably enough to hit some threshold.
From there the issue was just misaligned personal incentives, all along the chain from engineers at Facebook to Netcraft and Digital Ocean, that leads to false positives being an acceptable outcome.
On my personal Facebook feed recently, I have seen a few spam posts where the content is hosted on Digital Ocean servers. I actually reported one of these to DO myself and the account was terminated.
I wonder if there’s been an uptick in DO-hosted spam, and now some heuristic is being a little too eager with taking down DO content?
Author of the post here. Although Netcraft have said it was a false positive, I’ve yet to hear back from Digital Ocean so as it stands, my droplet is still going to be nuked in 24 hours as far as I know.
I know the linked the post is from 2018, but afaik you can create a password separately now for Spotify, and then log in via your FB email + pass combination if you used FB login in the past. I remember doing this a few weeks ago.
Facebook is primarily at fault, although I would also blame DigitalOcean for having unreasonable minimum enforcement time limits.
DigitalOcean should reach out to Netcraft and decide on a more reasonable minimum enforcement time that is based on Netcraft's actual support throughput capabilities.
This arbitrary choice of a 24 hour time limit hurts their customers. I plan to switch my hosting provider away from DigitalOcean due to this incident.
So let me see if I'm understanding this: Facebook pays a company called Netcraft for brand reputation management, and they scan the internet for misuse of the Facebook brand, assume it's "phishing", and send reports to cloud hosts to take them down. Sounds like the mafia.
I had my entire domain listed as a phishing domain on Google for having a real, legitimate Google OAuth sign-in flow on one of my (unlisted) subdomains. They didn't even disable the app in question, just delisted my entire domain.
cloudflare plasted red "DANGER! BAD WEBSITE!!!" banners across my company's website because the morons at netcraft told them it was a "phishing website"
thanks cloudflare for altering my website to tell my customers that I'm trying to phish them and to avoid us because some third party told them that was so
what was the cause of netcraft to sending this to cloudflare?
they found a 20 year old exe that had a link to our website as a string inside the binary
not a binary we had produced, just some random innocent third party program that mentioned us in their about box
if I had could have proven that I'd lost sales/reputation I'd have gone after netcraft and cloudflare for defamation
what use is a DDoS shield if they'll automatically shut down your site if someone sends them an anonymous form saying "they're hosting bad things!!!!"
Way too much of the internet is on cloudflare. They are and will continue to be the arbiters of speech on the internet. God forbid I browse the web with a vpn!
Sometimes I even get into infinite captcha loops without a vpn, even just on my phone. I LITERALLY cannot browse certain sites like wayfair because of cloudflare. I have done nothing wrong.
To be fair I’m never sure who is imposing the captcha s about things that are alien to my society (traffic lights hanging above lanes, yellow taxis, crossings without flashing amber lights), either cloudflare or google, but I’m certain it’s to continue the adtech economy.
Cloudflare received a phishing report regarding:
mycompany.com
Below is the report we received:
Reporter: Anonymous
Reported URLs:
http://www.mycompany.com/deadlink
Logs or Evidence of Abuse: Hello,
You are currently hosting a site which is associated with an ongoing
malware attack. The malware either communicates with, or is spread
directly by following malicious links:
hxxp://www[.]mycompany[.]com/deadlink [1.2.3.4]
Visit the report below to see details on one of the attachments
associated with this URL or IP address:
https://www.virustotal.com/en/file/12345678/analysis/
More information about the detected issue is provided at
https://incident.netcraft.com/1234/
Would it be possible to have this URL taken down as soon as possible?
Many thanks,
Netcraft
Phone: +44(0)1225 447500
Netcraft Issue Number: 123456
I made the unfortunate mistake of having the backend for my game on Hetzner and on launch day the abrupt spike in traffic made their abuse detection go haywire. No amount of explaining that everything was under control would satisfy their support team, and they null-routed my server.
This is my primary concern. I want to use Hetzner so badly for important workloads, but just can't stomach the risk (even if it's a small one) of being null-routed.
What is the host that guarantees not to do this? Azure took out our entire prod subscription once due to a shared IP that they allocated to us being used by another customer in an attack.
It's possible that I've simply not hit the corresponding circuit breakers but no issues with AWS for similar launches and spikes in traffic. It's expensive though, and my experiment with Hetzner was a misguided effort to reduce costs.
I had an AWS account get shut down because Netcraft straight up lied to them that there was some ongoing and active criminal activity and the server needed to be shut down immediately. Obviously no criminal activity, or even simulated criminal activity. It was hosting a web frontend for a common security tool, and they just happened to see the banner while scanning the internet.
I confronted Netcraft about it and they just told me that they didn't actually detect anything like criminal activity, but they need to say that or else Amazon won't take their reports seriously. Pretty fucked IMO.
I reckon their business model is for the architecture astronaut who set up Wingman[0] to leave the company and then everyone is too afraid to shut down it’s 1000 node k8s cluster.
AWS will not do this. I've heard of Azure doing this more than once. GCP I have no idea but if something does happen on GCP good luck talking to a human, better luck them not just telling you that you did something wrong.
That is actually best practice for cloud-hosted services. Traffic spikes can in general cause auto-threat-detection to panic. The only difference for the big service providers is their thresholds are higher.
(The maddening thing is that some of the circuit breakers are undocumented).
I get this feeling that Hetzner has too many customers, they don't want any new ones (onboarding is intentionally annoying) and that they are trying to lose customers so that they don't have to deal with them
Or when you report people advocating violence to others.
I haven't done it often, but 100% of the time I've hit "report" on something advocating violence towards a specific individual, the automated response has come back and told me the comment had been reviewed and was within the rules. The most memorable time was a death threat to a politician -- not that I agreed or disagreed with that politician, it was another country where I don't have skin in the game; I was just an onlooker really shocked by violent rhetoric.
IIRC the comment was "KEYS without the E" ( = KYS = Kill Your Self). Ok not obvious to everyone, but I explained clearly in the report, and that with like 3 seconds of Googling the recipient will figure it out, if not already obvious to them.
Honestly a bit surprised on that one, usually things like that get handled.
Then again, modern moderation is provided by the most bargain basement person or AI capable (and as it moves to AI it will be amusing in a horrifying way how people get around it).
No idea where you are or where you're referring to, but there's a lot of cultural variation out there in the legal code; some jurisdictions around the world are much more "free speech absolutist" on what rises to the label of death threat, where some others are very strict. In the context of a public-forum-esque discussion website it's possible language that isn't specific and actionable isn't a death threat by the letter of the law. To spitball, a report of misbehavior that's legally actionable in a place where it's "just" a t.o.s. violation might get sent to the wrong support team, and then kicked back as possible report spam, by a busy reviewer who's judged for their next pay raise by how well they kept the company they work for out of hot water with the local DA. Maybe you'd see better results under a generic harassment report or similar? Such is the world we('ve always) live(d) in. The only thing that saves us from the bureaucracy is its inefficiency.
I am in the US and the politician being threatened was in Germany.
I'm not talking at all about prosecution or legality of speech and its varying definitions under the law (of which I believe German law is more restrictive than here). I'm talking about Facebook interpreting their own rules, which I am pretty sure do not allow threats of violence.
Also, I am in favor of free speech and open political discourse. I am not in favor of saying a politician in a democracy should be murdered. There may be edge cases in the law where that is not considered credible, however, in terms of my own personal ethics it is very unambiguous when that line is crossed, and I think that's true for others.
I had a friend murdered by her boyfriend and because he was black, when the story broke her personal facebook was flooded with racist comments towards her for a while, even years after the incident, one most recently was how she's been "one year clean and sober from mud diving" and hundreds of us have reported those comments for racism and every time Facebook came back saying they don't violate the rules.
But one time I replied to an old lady who was saying bigoted and hateful things towards a trans person with "Thankfully like you, your views will die soon too." and I got 30 day ban for hateful comments.
> But one time I replied to an old lady who was saying bigoted and hateful things towards a trans person with "Thankfully like you, your views will die soon too." and I got 30 day ban for hateful comments.
As you should do. There's no excuse for wishing people dead because you disagree with their beliefs.
FB’s advertising algorithm just loves a good hate post. It brings people together like no other. Ignore the fact they are arguing and threatening to kill each other, look at those views! /s
I recently sought out the results of all the scam accounts I had reported (even though FB says they'll let you know the outcome, they don't) and every single one was resolved as "doesn't break out community guidelines".
As a result I've stopped submitting any reports, and question all FB metrics about active users & account numbers.
Good advice. UK in this case, but if I ever see one again I probably will. But as I'm sure you know, "Report" and "Block" are a single action on FB, so unless you took screenshots before reporting...
> and question all FB metrics about active users & account numbers.
any company/person using stats to self promote should always be questioned. of course numbers are going to be rounded and massaged in the direction most rewarding, but the extent of the rounding makes a difference. lots of weasel room available in these types of stats.
Advertisers are FBs customer. You don't mess with the people who pay your bills. You didn't think you were the customer here, deserving of any consideration?
I once had my account with a major cloud provider terminated for "violating our terms of service". After contacting support, they then claimed that someone had gained access to my credentials.
What evidence did they have? None. I just updated a VM's metadata too frequently (about once a minute). This tripped an ML model, which caused them to automatically terminate my account and send an automated email saying that I had been a bad boy.
This took down a key part of my business for about 5 hours (while I navigated my way through layers of customer support and ultimately temporarily moved this functionality to another cloud provider). Customers were not happy.
It took about 2 weeks and multiple support tickets for the full story to come out. I got them to refund a few months of charges (amounted to several hundred dollars at the time) and restore my account. There was never any recognition that they made a mistake.
I get that companies need to resort to automated means to handle fraud or abuse. But they should also own up to it, add some humility in their automated outreach to customers ("our automated system has detected possible X" instead of "you are guilty"), provide clear escalation paths to talk to a human, and provide a way to "shield" your account (identity verification, upfront deposit of $X, etc) that forces them to contact you before any enforcement action.
In my case, I upgraded to a paid support plan ($100+ per month) in the hopes that their system will be a little less trigger happy with my account in the future. I don't use support at all, it's purely a lame form of insurance that may or may not actually protect against anything.