> One surprising side effect of having a .xyz domain is that the mere inclusion of .xyz inside of a text message will result in a silent delivery failure for many providers.
This is wild to me. Tested it out myself and I couldn't send an SMS with a spot.xyz link to/from Google Voice <-> T-Mobile. And no "failed delivery" notice either, just a silent failure. And yet I still get so many texts that are obviously spam or phishing attempts.
Blocking of messages/emails and blanket email server/domain/extension blacklisting is the same as a postal service not delivering mail to or from a particular entity/street/town.
Doing so silently and without a valid and case-specific reason should not be legally allowed.
Edit: Added "street/town" to analogy, and "case-specific" before reason
They should really update the "Mission and strategy" chapter on their Wikipedia page [1]. In particular the part about "Protecting Consumers & Public Safety" seems horribly outdated!
I will have to look up how this works in the EU and here in The Netherlands. Something to do for the weekend.
I really would like to see what was the legal reason behind that, I know the US has issues with gov't agencies using their opinions as law but I thought mail was constitutionally protected?
The FCC just makes decisions, we're talking regulations here, not laws. They justified it with "preventing spam" and enabling competition with unregulated OTT apps.
As far as the sanctity of the U.S. Mail, it only applies to sealed envelopes/packages, and Congress can ban items from the mail (e.g. lottery pamphlets, spurious tokens, gasoline, etc.)
Doesn't congress make a law saying breaking x regulation carries y penalty and at the same time gives the agency power to decide what constitutes breaking x? I know for a fact that BATFE does this rather frequently.
Also, thanks for the clarification about the secrecy of US mail. I apreciate it
We've run into this issue with replies to texts that the user sent first.
Telecom spam filtering seems to be a ridiculously primitive and wide net. I can't imagine a valid use case for dropping a text sent to a number when that number just sent you a text a few seconds before.
I don't understand why SMS spam has such a big issue with false positives compared to email spam when emails are practically free to send but SMS is much more costly.
(Yes, I know there are a lot of false positives on email too ... but we run into false positive SMS spam issues a lot even though it feels like it should be a much simpler problem to solve).
Perhaps their blocking systems are stateless, i.e they don't bother doing a lookup of communication histories because it's expensive when you're dealing with millions of texts an hour. They just run each one through a bunch of rules and drop matches.
I believe that, completely. But keyword silently blocking is an objectively bad approach. Tell the sender it failed if you're so keen to do so. Or tag it with a big POTENTIAL SPAM at the beginning of the message and send it. Or literally any of the dozens of smarter ways of content filtering than (if .xyz in y).
Bad actors are why we can't have nice things. Think of how simple, clean and efficient computing would be if we didn't have to keep criminals and idiots at bay.
Very interesting. I definitely get phishing SMS messages from time to time, but I didn't realize these were some of the very few which actually made it through. Any idea how these bad actors are able to send out these massive batches of spam SMS? My naiive guess would be bulk purchasing disposable SIMs but I imagine it's more sophisticated?
It's whack-a-mole where game is skewed wildly towards the moles.
Basically there are tons of VOIP companies, with varying levels of give-a-shit and spam detection capabilities.
Generally they are incentivized to let people self-serve on their platform - spin up quickly and start running traffic, or blasting spam, whatever. Especially if you're a small company, you're probably more likely to look the other way for a bit if someone is spending money on your platform, until regulators call, and you can be like "ok we looked into it and shut them down".
Also you don't want to be overly aggressive, because what if a great customer comes onto your platform, loves the ease of setup, and starts running legitimate traffic, then you shut them down because they were triggered by whatever crappy spam heuristics your small company came up with, and the customer is gone to another platform where they don't have to deal with that.
Then the company/group running the traffic moves onto other VOIP providers until they get a bad enough name or push the envelope so much that no one will take them.
Then they just create a new "company" that no one recognizes the name of, and start again.
Honestly I think an open sourcing of spam detection heuristics and algorithms would be a massive help, but companies that are good at this obviously see it as a competitive advantage, just like the email space - for example if Twilio is great at keeping spam off their platform (no idea if they are, but they would have the most resources to do so), then all numbers registered with Twilio are less likely to get flagged/blocked downstream - all Twilio customers benefit.
Twilio can say "any number you buy with us will be considered clean by downstream parties, no need to worry about getting flagged/blocked, then having to change the numbers you use for your business to communicate with customers, which could be saved in their phone already, etc."
The patterns spam takes vary wildly, often being specific to telecom laws and practices in specific jurisdictions, so it really is a tough problem. If an algorithm flags spam, you often want to then reach out to the customer and try to understand if there's a legitimate reason for the traffic patterns, etc. So there's a layer of customer relations beyond the algorithms that's also tough to scale.
There's already an opt-out legal framework in place for marketing calls. Mass sending SMS spam to opt-outs is illegal. Prosecute the crime. It makes zero sense to try to guess from content.
If I had a spam texts folder that showed me everything I was being blocked from, I'd both appreciate it and not feel this massive breach of trust that things being sent to me are being completely ignored by a third party system.
The system that does this is absolutely primed for censorship, and we have no way to know it's not being used.
1) Neither the SMS protocol nor any phone I've ever seen has any mechanism to file messages in "folders".
2) Processing SMS messages and delivering them to subscribers has a cost. Doing so for high-volume junk messages would place a significant burden on carriers.
3) Most carriers used to charge subscribers for receiving SMS messages. Some still do! Charging subscribers to receive spam SMS messages would be, quite rightly, called out as inappropriate.
> 1) Neither the SMS protocol nor any phone I've ever seen has any mechanism to file messages in "folders".
My phone (ROG Phone 3 w/ Android 11) automatically flags spammy texts into a "Spam & Blocked" folder, I assumed this was a stock Android feature - is it not?
1 and 2: true (to a degree, phones sort messages by sender which is a folder), but if a SMS already reached the provider they have the data. No need to send spam to the client. Instead display the SMS on some webinterface the customer can access. Or email it.
I'm not opposed to spam filtration as a user default, but doing so silently without any indication of what is being filtered or ability to verify it is working is not acceptable for such a vital messaging system.
I've personally noticed a lot of phone text spam being FROM email addresses recently. I think they are just abusing some feature in MMS, though, not SMS. It's weird seeing a list of phone numbers (usually SMS two-factor), some contacts' names that I have entered in, then a ton of random email addresses on my texting app (standard Android Messages app).
It's actually worse than that. It isn't blocked because of the sender or recipient, but because of the content. That would be like the postal service reading your mail and deciding that because of an address in the text of a letter, it shouldn't be delivered.
Amusing analogy. The postal service's unwillingness or inability to do just that has severely damaged their utility. If the USPS had a junk-filtering option I'd sign up tonight. Perversely, the postal system seems to embrace junk mail (e.g. if you sign up for address forwarding the USPS sends you a fat envelope full of junk mail as a "confirmation")
I wouldn't. I mean sure, it would be nice not to get so much junk mail. But I personally don't think it is worth the risk of important mail like bills, tax info, new credit cards, etc. as accidentally getting flagged as spam. It also increases how much you have to trust the postal service.
I like to chime in with this one when possible (due to a deep resentment for credit card offer spam). In case you didn’t know, you can opt out of the credit industry’s vast marketing machinery.
The problem is that just receiving the message is in-and-of itself bad for the end user. It's not the volume you think (assuming the other poster is accurate about relative volumes) - it's far, far more. Imagine getting 1000 SMS/day that all have a "spam" warning attached, or worse, no warning at all. You'd just stop getting any value from SMS at all, and ignore it.
I mean, going back to the postal service - even the weekly pile of "here, throw this away for me." dead trees we receive (in the US) is mildly irritating. Imagine THAT x 1000!
I'm grateful for the silent block in this case. I mean, my social security number is being canceled, I'm about to be arrested by the IRS, the FBI found a suspicious package with my personal information in it and my car warranty (didn't know I had one) is up for renewal. And that's just this morning. What more can I stand? One of these days I'll press 1 out of desperation...
Also I hate govt/big-corp censorship as much as the next person, but none of this seems remotely political or ideological. And consider the alternative.
That's not the issue - the issue is not alerting the sender that the message has failed.
It's not a big deal if the receiver never receives the message - we can find a different way of reaching out or fix the content problem or whatever. But we never find out. As far as the sender is concerned, the message succeeded.
This is a problem and the very bad spam heuristics employed by even the most competent actors (gmail, for instance) mean that anyone can be impacted by this.
You missed the mentions of scale in the post you replied to (and elsewhere in the thread). Imaging needing to hit that or delete tens of times, maybe hundreds, maybe more, for every non-spam message you receive. You'd soon get sick of it. You'd soon accidentally delete, or otherwise miss, an important message in amongst the plethora of junk.
As a consumer, I can see both sides of this. On the one hand, I like energetic spam blocking without fear of legal liability, even if there are occasionally a few false positives. On the other, I do not want ISPs/telecoms to be the arbiters of traffic (net neutrality).
The net-neutral solution is for ISPs/telecoms to not spam-block, but rather have spam-blocking be an optional, additional, layer that the consumer can choose at will, or not have at all. But the problem with that solution is that it requires the consumer to do extra work to obtain spam protection, and the consumer would not be protected by default. It also means extra work by all parties delivering spam messages. Unless spam ceases or things otherwise change, I think the clunky solution we currently have is fine for the most part.
Then make it set to "on" by default, and if more than 50% of customers switch it off then change the default.
I also think that this should be a requirement for social media. You should be able to opt out of separate filters for "spam", "misinformation", "breast-feeding", and whatever other reasons a social network has for banning legally protected speech.
In effect, sure, but in implementation these aren’t comparable. Postal services usually come with monopolies and mandates that ISPs, telecoms and email servers usually don’t.
USPS has a monopoly on first-class mail in the US and a Congressional mandate to deliver to every address.
It was wild to me too. I have an .xyz domain, which seemed appropriate for a non-commercial math site. I'd try to send links of math experiments to friends and colleagues via SMS, so they could tell me if they worked right on their phones or not. Can't tell you how much confusion and frustration it caused that the links were simply not being delivered, though all the conversation around the links went through just fine. No error was reported on either end. A year or so ago, I did a lot of searching trying to find some explanation of this bizarre behavior, but found literally nothing. It's nice to know I'm not crazy, at least. Is there a published list of what domains are not allowed to go through?
Quite likely only investors in Google / Alphabet stock know that site and have it bookmarked because that's where Alphabet publishes its quarterly earnings. I also guess for the same reason, it only gets significant traffic once a quarter during earnings season.
I ran into this recently even on Facebook Messenger. A friend of mine was hunting for a short domain name and I had a list of some three character .net and .org domains I recently had found that were available.
Cut and pasted the list and the message wouldn't send.
Narrowed it down to one. Typed just the bare domain. Wouldn't go through. (It was something incredibly benign like n17.org)
Couldn't find a history on that domain name for why it would have been filtered.
At least messenger responded with 'couldn't send message' but still no clue as to why... and it took me sending each domain name individually until I found the one that was failing the entire message.
Is it reprehensible only when it impacts you or is it still reprehensible when it's blocking hundreds of spam messages a day you might otherwise be receiving?
Surely there are better ways to reduce spam than blocking entire TLDs? I also think it's the silent, unfixable nature that annoys most people. Email spam goes into your spam box, where you can still access it. You can mark email as not being spam. No such luck here
Well, from their perspective. Not from any reasonable perspective; I have a few obviously-spam emails in my gmail spam folder right now, but I've had plenty of problems with gmail refusing to deliver completely legitimate email to me.
Who cares? The only things that make it into my spam folder are obvious spam. Meanwhile, messages from people I know personally aren't even delivered at all. There is no way to characterize this as reasonable or even acceptable. Google is using metrics that are not related to whether an email should be delivered.
They need to tune whatever they're doing down to the point that legitimate personal communication at least shows up in the spam folder. If a lot more spam shows up in the spam folder too, so what? A spam folder that contains mostly spam and also some misclassified personal messages is significantly better than a spam folder that contains nothing but spam because it automatically dropped your misclassified personal messages.
Yeah, there's a lot of spam out there. My employer's spam filtering software used to send out weekly statistics telling us what percentage of incoming e-mails (across the entire company) were spam. The spam percentage was remarkably constant from week to week: about 90% of all incoming e-mail was spam!
I deliberately set GMail to the lowest level of spam filtering they'll allow, and still only receive a couple of spam messages per month at most. They need to adjust their levels.
I run my own mail server and never silently drop any messages. I also have a catcha-all inbox and my primary email address is available publicly in many places (my website, git repos, bug trackers, ...). The amount of spam is really not that bad.
...whoa, yeah same here. tried "test spot.xyz" then "test spot.com" T-mobile <-> T-Mobile. "test spot.xyz" did not send. Even weirder, I got a confirmmation that it was delivered.
It looks like T-Mobile looks for ".xyz" within the SMS and will silent drop the SMS (though it will claim it is delivered). ".xxyz" works, "..xyz" or ".xyzz" does not. "xyz" works, so does ".xy".
MMS has delivery reports too (I implimented support for it myself for mainline Linux Phones). It even has read reports, but no carrier seems to honor using it (which is why I didn't bother to impliment it).
I'm not sure if Android/iOS gives the user an option for it (which may be the source of confusion).
I'm not sure if SMS supports read reciepts, but I didn't think so. The MMS standard allows for read receipts ("MAY" not "SHALL"), I was unable to get it working, and I suspect it's due to no carrier support.
I was unable to get read receipts working at all, and I suspect it's because the carrier doesn't impliment it.
Delivery receipts, I've edited my comment. I've never been able to get read receipts to work. If I enable it, sometimes I will receive an actual text message that "123-456-7890 has read your message", instead of just marking the message as read.
Many years ago when incoming messages used to cost, each delivery report I recieved after sending a message out costed me exactly one incoming message, in India in 2000s. Many phones still offer Request Delivery Reports.
A lot of systems block anything by default that isn't standard. For example, if you happen to own a domain to serve as your email that doesn't end in .com, .edu, .gov, then many systems will instantly invalidate you saying you don't have a valid email when in reality you do. A lot of companies or programmers don't seem to realize that its 2021 where we have hundreds of domain extensions to choose from.
I had a .ninja domain for a while, and I had to contact a certain DNS provide to add support for that TLD. They were very responsive, but I still had to ask.
Pff. That is nothing. Try to run your own mailserver and deliver a mail to a t-online.de address (T Mobile Germany).
They basically only accept pre-approved providers. If your have your own domain and infrastructure you have to petition them to whitelist you. Totally insane.
Related thing from the past.. Gmail once had a bug(?) where if you sent any email containing a URL with the domain starting "0x", it would go straight to spam. I imagine it was a rule hard coded to block the use of hexadecimal long IP URLs, but it also picked regular domains starting 0x. It was fixed a few years ago.
I own a domain starting with 0x and I spent a lot of time talking to people I knew at Google to get that one fixed because my mail would not be delivered.
If you’re in Canada, and send an SMS containing the string “special message” to or from a Telus customer (or one of their sub-brands), it will be silently dropped. Telus is one of the big3 telecoms here.
I have no knowledge of the ROI involved here, but would love to understand this: Twilio is 0.75c to send a text.
Is it possible for a spammer to generate >$75 per 10,000 people spammed? I've no idea were the SMS spams I've got link to (not about to find out) but they are so obviously spam.
We use SMS for communicating with users and would be happy to more a lot more per text to escape the 'positive ROI for spammers' territory.
Probably decent ROI which is why it keeps on happening!
They just need one person in each 10k spammed on average, to click the phishing url asking them to pay a fake bill and then charge them $328 instead of the $3.28 displayed o the page.
I received (and reported to their scam Dept) a phishing SMS yesterday pretending to be from Australia Post asking for $3.28 to release a delivery package I'm waiting for, which is most people in Australia nowadays with the current slowdown in mail delivery speed.
I am only guessing that the $3.28 phishing purchase would have attempted a $328 charge on my card... but that would be wildly profitable if the input costs per successful fraud were under $100...
How did you determine that these messages are through Twilio and not one of the dozens of cheap unscrupulous knockoffs? This is very easily identified, and you are are slinging serious accusations.
Because spam really is that rampant. There aren't that many communication systems with a small search domain of user ids where anyone can send and receive messages from anyone by default.
The latest spam method right now is to get malware on to android phones and have the actual phones do the spamming. So if there is a cost, it gets applied to random people and not the spammers.
Also if there was a cost per SMS on phones these days it would be the death of SMS because no other system charges.
I am pretty sure this is not intentional. Somewhere some classifier in Google has overfitted onto .xyz. They will probably fix that some day so this will not be true forever either.
Whoa. I use an xyz domain daily. This thread is eye-opening. Here's the reply from a SpamAssassin validator.
My domain is almost marked as spam solely on TLD grounds. What's the point of a TLD if it isn't a first-party domain on the internet?
SpamAssassin Score: -0.599
Message is NOT marked as spam
Points breakdown:
-5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at https://www.dnswl.org/,
high trust
[***.***.***.*** listed in list.dnswl.org]
0.0 URIBL_BLOCKED ADMINISTRATOR NOTICE: The query to URIBL was
blocked. See
http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block
for more information.
[URIs: ***.xyz]
-0.0 RCVD_IN_MSPIKE_H2 RBL: Average reputation (+2)
[***.***.***.*** listed in wl.mailspike.net]
0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record
2.0 PDS_OTHER_BAD_TLD Untrustworthy TLDs
[URI: ***.xyz (xyz)]
0.0 HTML_MESSAGE BODY: HTML included in message
0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily
valid
-0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from
author's domain
-0.1 DKIM_VALID Message has at least one valid DKIM or DK signature
2.0 FROM_SUSPICIOUS_NTLD_FP From abused NTLD
0.5 FROM_SUSPICIOUS_NTLD From abused NTLD
0.0 TVD_SPACE_RATIO No description available.
I host my own email, and I got nothing but huge amounts of spam from .xyz domains, so I manually increased the SpamAssassin score for just .xyz to +4.0, as KAM was only adding 0.75 for it. It's the only TLD I've had to do that for.
Unfortunately for the people with legitimate uses, for email admins it's just a really easy (and arguably necessary) shortcut to block a ton of spam.
On my personal mail server I have an SA rule that gives .xyz a high score (like a +5 or so) because a while back I was getting a huge amount of spam coming in from senders/URLs using domains of it. I haven't revisited that rule in a long time, and suspect it's not unlikely that others have done similar things.
I was pretty excited when ICANN opened up a bunch of new domain extensions, but it does sometimes feel like "all these extensions are great if you don't plan on using them".
It was pretty cool that I managed to buy a bunch of domains like <my last name>.<new-tld>, but to be honest I really don't see myself using my .blackfriday domain for anything. For that matter, I think that (somewhat ironically) `my-last-name.email` would not be taken very seriously for a primary email address.
I use a `.app` domain for my personal email, which has its issues, but if I owned a business, there is no way on earth that I would be using anything but .com.
> I really don't see myself using my .blackfriday domain for anything.
Well, there's about a 1 in 7 chance that it would be the perfect domain to host your obituary. I'm sure you could make a smart watch app which detects when/whether to make the site publicly visible.
I had .email for a while. Many shopping sites wouldn't actually let you use the address, presumably given some filter assuming that "email" was a fake address. Because my name ends with "ss", I switched over to .es, which conveniently is a country TLD (Spain). That's worked very well, though occasionally I'll get spam in Spanish, which cracks me up.
I have an .email and am slowly replacing my gmail address with it. 99% of services seem fine; occasionally there is some old laggard with broken validation schemes ("more than 3 letters for a tld? YOU MUST BE A HACKER!!!11!") but then it's likely you don't want to do business with such people.
I wonder how well these new TLDs work for custom email if you use them with Google Workspaces / Google Apps for Domains or another reputable email hosting service. I've been using a custom domain (though a .net) for decades now and since I moved to Google Apps years ago I haven't had an issue being seen as spam.
I use the free email provided with the domain hosting I got with a domain on one of these gTLDs. The only real issue I get are places that think the gTLD isn't a valid email domain, ensuring I always have to fall back to a more traditional email provider for some places.
Otherwise, deliverability-wise I haven't really experienced any issues. My mail is regularly delivered to the big email providers.
I haven't had many issues using .xyz with gsuite (with the exception of a couple of sites that didn't accept it as a valid email), but most of the people I email are @gmail.com.
You're obviously American. .com is not the only acceptable domain. Each country's tld is perfectly fine for anything in said country. In fact, it's really weird when companies here in Sweden use a .com domain instead of a .se
It's kind of a pain to use .com.au because you have to register an ABN and have a company / yourself be directly related to that name. So I can't just come up with any cool domain name I want and create a .com.au for myself without legally registering that name too.
I secured <my last name>.name many years ago. Must have been early 2000's (.name is around for a long time). It wasn't exactly cheap and it is not one of the super cheap domains now. The registry seemed to have a relatively strict policy regarding who can own a .name (not sure if still true, haven't checked again since like 2002). So, the best preconditions to keep their space clean, it seemed...
Yet I gave up on it for the same reasons mentioned in the article: It has a terrible reputation and seems to randomly be blocked here and there.
crypto space is making use of the new ICANN approved TLDs pretty rapidly
their customers are on discord, twitter, telegram and wechat so email delivery is not a factor
the entire sites and revenue drivers are entirely client side (with the "servers" being the smart contract methods stored on the nearest blockchain nodes, this has only one initial upload cost but functions similarly to lambda functions except the users pay for the computations), when the domain is down or blocked, the user can interact directly with the nearest node hosting the website's associated smart contracts, if they are interested enough
this is working really well for a lot of organizations, and it has been this way for several years now
makes lean SaaS services even leaner, and allows them to grow even faster - as long as their customer base is already a crypto native. I haven't seen any organization succeed if they have to sell their customer on some crypto browser extension.
not sure why you're being downvoted -- one of the coolest parts of the web3 world is that you don't need a server, you just need static web hosting. This means that besides blockchain network / router speed (most transactions get broadcast through a few friendly or public nodes for each network, and each network has different transaction speeds) interacting with a web3 application is often extremely fast. The best web3 apps use cloudflare or some other equivalent and deliver static html/js/css from the edge https://twitter.com/simplyianm/status/1437506136568041472?s=...
And, yes, a lot of the new web3 projects use alternative tlds because they're cheap and catchy. They also tend to use food-related nouns as project/coin names because branding is hard and a lot of them haven't been used by companies in the past.
very few people understand the primary market in the crypto space, as they only know the speculation side and don't like that.
many of the people with the capacity to understand the primary market or revenue generating side are too busy hating that "crypto" gained an additional context that is more widely used than their enthusiast obscure cryptography interest and they use every comment section to let everyone know that when they aren't busy weeping under their Alan Turing shrines before they have flashbacks of boarding the bus at 7am to work for an ad conglomerate, a life they ironically respect more.
Before I get into email business(I run my own email forwarding service[0]), I don't understand why provider block those domains.
Then I immediately got it. The amount of spam emails from .xyz .click .faith .top is huge. And with every email comes from them, we have to run spam scanner, which isn't cheap. So we have to score those TLDs more sensitive.
> We should have known better from the beginning as we previously founded Outreach.io, the leading sales engagement platform, making us no strangers to email deliverability. In the early days of Outreach, we had utilized some short .xyz domains to use for shortened links in emails sent on behalf of our customers.
Translation: We used .xyz for spamming, of course .xyz is associated with spam.
Got a nice .xyz domain mainly for mail with SPF,DKIM correctly set up and tested against multiple validators.
No big issues so far except for the HR department of a potential new gig which can painlessly mail me@mydomain.xyz about job interviews BUT never get my replies back.
I don't who to blame more in this mess:
- Me for playing smartass instead of using a @gmail.com because they impose the rules so everybody comply to them (maybe my reluctance to encourage this broken system explain my recklessness)
- The IT department of this organization that probably didn't what to deal with modern standard and/or reasonable spam filtering and set up a blunt rule for new TLD (I mean come on it was a REPLY to a mail ADRESSED to this specific mailbox)
- The broken system that keep on inventing arbitrary new rules that everyone must implement to keep getting accepted by "the big players".
(For instance I already had to change hosting two years ago because apparently you are also responsible for bad neighbors)
Guess i'll just have to be brave and migrate to a more classical TLD and set up redirects to ease transition. But it's pretty annoying to start over with crap like that because some dudes in "the big players" teams decided to ban a whole TLD just because it's "easier".
> (maybe my reluctance to encourage this broken system explain my recklessness)
This is a great example of a Collective Action problem. Everyone would be better off if we could break the gmail domination of email policy, but as an individual you will have zero effect on gmail's dominance and only suffer the pain of not being a part of the system.
And I would love to see EFF a little more involved in that matter. If things continues to goes downhill this far how many years left before big players decide to outright reject mail that doesn't come from a curated whitelist of their own?
The responsible answer should be IRL legal actions against real spamers because they'll always adapt to new arbitrary protocol rules faster than legitimate users, it's their jobs!
Even from an environmental standpoint I get tired of user-shaming articles about why you should delete your email for the planet. Maybe as engineers our duty is somewhat to propose a new version of the mail protocol that doesn't allow this much crap to fly around in the first place. Current solutions seems to revolve around the concept of "everybody should duck and cover if anything is suspicious" thus blocking some legitimate message that no sane human would reject should they be in charge instead of a basic AI.
PS: I'm not suggesting by any mean that you should punish any human being with manual moderation.
PS:PS: Maybe a NGO whitelist system is a solution, I'm just fearfull about which entity will end up with such power. But actually domain filtering is already kind of an implicit unpredictable non shared whitelist build on top of ICAAN register... So here we are already...
There's good reasons to use a custom TLD aside from being a smartass, or disdain for Google policies. It's portable and if you have a very common name, unless you were one of those early Gmail adopters buying invites off eBay, you won't have a reasonable and short alias.
I have had similar experiences with corporate firewalls blocking my .app domain.
I got in a painfully stupid argument with a middle-age IT admin “we don’t want to our employees installing apps”
It’s not an app, you don’t install it, it’s a “WebApp”, it’s just a freaking fancy website who’s domain ends in .app - lol, this was like three years ago and just thinking about it is getting me heated
> I got in a painfully stupid argument with a middle-age IT admin “we don’t want to our employees installing apps”
If they think that any domain that ends in .app is for installing apps, their mind is gonna be blown about some of the sites on .net and .org domains...
I have a .haus domain for personal use. I can send and receive email just fine, but I do run into a lot of apps that do some sort of misguided "validation" on the email address and reject .haus as an invalid domain. One retailer lets me use the .haus email address as a login, but once I log in and try to make a payment it requires me to enter a different "valid" email address to send the receipt to. It's very irritating.
I've been using a .info domain for email for, I don't know, 15-20 years. Maybe 3-4 times in those decades I've run into a service that won't let me sign up with my "invalid email". And once, I was locked out of my smart garage door opener app because a new version decided my already-registered email was now invalid for logins. Customer support kept telling me to just reset my password, but even the password reset form decided my email was invalid. A few months later, another new version of the app decided my email was valid again.
I have similar issues with my two main emails, which end with `.app` and `.sexy`. Both of these work fine, but validation will fail a lot of time (particularly for `.sexy`, but even for `.app`), forcing me to defer back to an unwieldy .com that I own.
I used to have a weird .red domain and I always found it awkward to tell people. So I ended up going with firstname@lastname.me and literally no one has got it wrong or looked at me funny.
I had a .co domain, and it was a pain to spell it out to people. "It's like .com, but without m" and people usually got confused, or thought it was a typo and "fixed" it as .com.
I have a .dev domain now and everything seems to be running smoothly, plus it's +20% cheaper.
The reality of Internet filtering and firewalls, and a rule generalisable to any attempt at control and autonomy, is that the effect-to-effort ratio matters. The principle of a small effort with a large result is behind the architecture of every switch, gate, door, valve, or dam.
New generic TLDs have the disadvantage of being recently unleashed. There are no venerable sites on XYZ, or its siblings. Much of what's registered there, and that word was "much" and not "all", is absolutely unworthy crap. And for those who are faced with defending either their own or their customers, clients, users, employees, or other stakeholder's security and time, wholesale blocking of the entire TLD solves a lot of problems with very little downside cost.
The obvious response is "but there's a lot of crap on legacy TLDs as well". Yes, there is, but there are also valued, venerable, and essential domains, and blocking all of them is not a viable option. (Though the prospect of whitelisting is becoming increasingly attractive.)
I've known people who are, on the one hand, Internet freedom advocates of decades-long standing --- before most people reading this were born. Who wholesale block access by all China ASNs to their webservers --- because all they see from such networks is malicious traffic. Again: effect-to-effort ratio here is high.
No, it's not "fair". Yes, there's collateral damage. But you're absolutely fighting not merely human nature but all of control theory in trying to combat this.
Register on XYZ and you'll be increasingly fighting a common practice of default-deny, whitelist-by-request. For every user you're trying to reach.
And you should ask yourself if it's really worth it.
XYZ, meantime, are mining and arbiraging short-term cashflow for long-term reputation at the specific expense of its legitimate customers. Those with the least bit of sense will abandon the registrar, leading to an ever-accelerating reputational death spiral.
The XYZ TLD is a hotbed for spam due to it's very low fee's for purchase / renewal. The registrar was, at one point, selling massive blocks of xyz domains to foreign squatters and spam artists for quick cash. No wonder it's become blacklisted by email/cell providers.
Can anyone try `abc.xyz`? and see if that fails to send? It would be very typical for our corporate overlords to be omitted from our spam censorship filters.
>The text including the .xyz link is notably absent. Until I realized what was happening, I would sometimes have some very strange text exchanges with people whenever I would mention my company or my email address. Once we switched to spotvirtual.com, this issue went away.
>Ironically, Google Voice also has the same behavior with abc.xyz.
There (was?) even a semi-parody site called Domains For the Rest of Us[0] that generates .COM domains that you can use for side projects (or startups?).
The new gTLDs are a godsend since all the domain hacks have been largely exhausted. E.G: `del.icio.us`.
I like the new avalanche of gTLDs since it reduces domain squatting, domain hacks, and stops people snapping up short .COMs as if they were some digital gold to be mined.
Not to mention the hassle of having a really obscure ccTLD like .SO and having to battle to get that domain back if it was seized by pirates, yarr
I read a series of blog posts about a guy who would essentially work backwards from a domain name to start a business. E.g. he would buy things like 'weehawkenjobboard.com' then SEO a job listing service for people/businesses in Weehawken NJ. I thought it was a pretty clever strategy.
Personally, I'd go for some non-printable characters. But, maybe I'm just nostalgic for when starting a directory name with ALT+255 rendered the directory inaccessible to Windows 9x...
Not to be secretive, but I do have an emoji domain name (.ws allows them). You can use it for email, but it's a bit of a pain to use the address in practice. It's easier to send links directly to people, but it is kind of funny to be able to use emojis to email my kids.
Kind of related: I have a 'firstname@lastname.email' email address. I had booked movers online as I am moving apartments. I thought it was confirmed because I got an automated email back confirming the booking. I gave them a call about a week later to double check everything was OK and it turned out they never booked me in because they thought it was a fake address ("Wow, this is really weird, I've never seen an email address like this").
Luckily, they could still book me in but at a different time slot...
> we would occasionally get feedback from users and prospects that the .xyz domain felt unprofessional
I had a .xyz domain. I thought it was easier, the domain was short to type.
I was completely wrong. I asked a few non-technical friends. They said they would never use my site because of the .xyz, it felt like a spam site. I redid the site on .net with a longer domain name - much better results.
I teach college and I can tell you that most people don't type urls. Many don't even know how. I will tell them to go to an address like kahoot.it to play a review game and most of them just type kahoot and search. if they do actually type in a url they will type kahoot.com instead of what I told them to type which takes them to the site for creating kahoots not directly to playing them. (you can get there from .com but it isn't the quickest way)
I have my personal site on an xyz domain because it's the only thing I could justify spending on. I don't intend to earn from it, it's just a static site, and it's significantly cheaper than anything else. I'll probably stick with it.
Does anybody know if there's a consolidated list of domains and their various blacklist/deliverability issues compiled someplace? I for one would love to know how broad this problem is across the various TLDs for network filtering/email/sms/messaging etc. Seems like it would be a pain to maintain even as a snapshot but I would definitely be interested.
Disable auto-image loading, and it will cut down the ability for companies to do this.
Unfortunately, this often times leads to direct phone calls along the lines of, "Hey taftster, did you get my email? It shows that you haven't opened it yet."
I get unenrolled from electronic statements from Capital One and a local credit union if I go 12 months without “opening” an e-mail from them. I do open and read their e-mails but since I don’t have image loading enabled, they don’t know that so they “helpfully” start sending me paper bills again, and stop sending me the e-mails to say that the bills are ready. It’s incredibly annoying.
This is a total shame because .xyz is extremely catchy and, in my mind, could be the new .com in a few years. All the other TLDs are hard to remember -- in my experience, people will ask "was it my.website or mywebsite.com?" but if you tell someone "it's mywebsite.xyz" they always get it right.
This has not been my experience at at; people assume saying .xyz is a joke and then act surprised when the URL doesn't work and you say "no really, it ends in .xyz".
We thought we were being smart when we bought a .io domain. Can't tell you how many times we told people the site was foo.io, and they would say, ok got it. "foo.io.com".
It's sometimes difficult to believe how much misguided logic is put into input validation. Addresses which must have a street and a number, middle names not allowed, valid postal codes not recognized or auto-filling the wrong town, arbitrary maximum length for street names, and I could go on. We programmers (or we product managers?) invest way too much time in nonsense.
I really hate this. I've seen separate input fields for street number and street name. Meanwhile you have vendors with street addresses like "Vodafone House, The Connection, Newbury RG14 2FN"
.net and .org were the original .xyz. Back in the late 90s and early 2000s they were seen as less reputable for businesses. I think they still carry some of that tarnish.
.org domains actually have some credibility as there's this misconception that you need to be a non-profit to register one, like how you have to be a accredited university to register a .edu domain or a government entity to register a .gov domain. .net domains however have a bad reputation regardless for some reason.
mail-tester.com is great and I use it frequently to check (and recheck) the mail servers that I run.
However, your (correct) evaluation of their weird DKIM/DMARC/MX values notwithstanding, I currently have 10/10 totally perfect score from mail-tester.com and gmail marks my email to my wife as spam.
As in, a 15 year history of my email address having multi conversations per day to her email address and some of my emails (which are responses to her emails) get marked as spam by gmail.
These are also good reasons to avoid using .so domains. You can also expect mail delivery issues and blanket corporate firewall blocks on .so. The rising prominence of https://notion.so is changing the cultural situation somewhat, but very slowly.
A potential issue with ccTLDs in general is they aren't subject to ICANN policies at all. Countries can do whatever they want with their TLD, ICANN's only involvement is keeping their root zone entries up to date.
This means you're subject to the politics of whatever country's TLD you're using. If the country's lawmakers suddenly decide that their TLD should only be for use by local entities, or that owners of popular domains should pay more, or that certain types of content is banned, you have no recourse.
Notion said they were switching to .com "as soon as our engineering team has the bandwidth", but it's been a year so they might've changed their minds on disrupting the branding
This isn't much different than when Google de-indexed millions of .co.cc domains. They determined that there were so many spammers on those domains that it was better to just remove them all and stop worrying about it. It did get a very few legit sites in the process, but not enough to care.
I get that the people here want more control over their devices, but to be fair, anyone posting here is at the extreme end of the tech spectrum when compared to your average phone user. Those phone users want someone else to help them. It's why I have spam assassin crancked super tight on the mail server that my parents use. They would rather miss a few legit emails and texts than get flooded with spam.
we would occasionally get feedback from users and prospects that the .xyz domain felt unprofessional and that they would prefer to use an app with a different URL. This was surprising feedback, as we did not believe that, beyond the initial discoverability of our product, the domain itself would create this type of impact.
Not surprising at all to me, who has used the Internet for over two decades --- to be honest, all these new and unusual TLDs, whenever they show up in search results, are almost entirely sites filled with SEO spam and similarly useless content. It's nearly an instinct to ignore them at this point.
(As for the company, it's too bad virtualspot.com and virtual-spot.com were already taken; spotvirtual.com looks weird, but at least doesn't have the negative connotations of an even weirder TLD.)
As a guy who's blog and email consists of .xyz domains, I can only say two or three times in ad many years has it ever been a problem (that I'm aware of, at least), and then it was a website not letting me create an account using it for email.
Same here. My DKIM headers are signed by a well known and respected email provider. I assume if you self host the results are gonna be wildly different.
Sometimes I send reminders from my xyz domain to my corporate email accounts (which tend to have a rather aggressive filtering) and everything seems to work fine.
Yep - used to work at a bank that very aggressively blocked gTLD because they had a (very stupid in this case) security-first mindset. Despite having multiple first-class URL filter products that can detect reputation and site category without needing to bother an analyst or cause a disruption.
SOCs, web filter, email filter teams and vendors all need to catch up to the 2010-era idea that carpet-blocking TLDs is not the first tool to reach for when securing a network, especially when you have a good URL filter in place.
Unpopular opinion (maybe): given the current situation, we should probably consider phasing out TLDs somehow. It's becoming more and more clear than no TLD outside those established before the '90s are actually viable for anything outside of "my small personal blog". It would also avoid people having to remember if the TLD is .net or .com, for instance (even though in my experience .net is slowly disappearing too).
I have my name .xyz and I’ve mostly given up using it for email, because I am sick of:
* “Do you mean ‘biz’” on web forms
* other forms just refusing to validate unless I disable the client-side validation
* other systems ostensibly accepting it and just never sending me anything, because it fails to validate silently in their backend
* having to put whatever I am trying to get done on hold for a few minutes when I need to read it to a human, because they’ve “never heard that one before”
.biz was a new TLD, so yeah. It was part of the 2000 round of new TLDs. I don't think I've seen a .museum, but all of the others from that round give me negative vibes, although I guess slightly less than most of the newer new tlds.
That said, I've got a 'clever' .pictures I use to share images and a totally appropriate .fun that has no need to have positive domain associations.
I have a .xyz domain for my personal stuff. The biggest issue I've had with it is that steam refused to acknowledge that it was a valid email domain. So they just wouldn't let me switch off gmail to me@mydomain.xyz because it didnt get past their filters. That was the biggest roadbump I had for switching off gmail.
We did a bunch of testing of crawling, indexing, and checking of rankings of the 15 top tlds. The .xyz actually was crawled and indexed by google within a few hours, many others took days to get crawled.
Google prefers to crawl and index .xyz sites over others domain endings. But they won’t rank them well in the index.
I run email servers and I get such a massive amount of spam on "vanity" TLD's that I just block them outright. I don't automatically block them all but any that start sending serious levels of spam get blocked. Which is most of them and that block covers the whole TLD. It's just too much work to try anything else.
Now this is just for incoming email. I still allow web browsing and links to these domains through various systems and outgoing mail to those domains works.
The incoming mail though, I just can't allow it. It's just pure spam at ridiculous levels.
tough story for a company and I know there's a ton of shady TLDs out there now but this will change rapidly I think - it used to be a .com world but as we all know .io etc has changed rapidly in last 5 years. Lately due to lack of .coms I get the feeling a lot of the other TLDs like .shop, .whatever are being used more and more for random sites for startups, projects etc, so I'm sure as they become more accepted in tech systems like SMS (weird about the filtering) and servers etc.
The reason .xyz domains are banned is because of the amount of spam sent from that domain. In my case 100% was spam. So blacklisting it was an easy fix.
If I was owner of the .xyz TLD domain, I would be very concerned to kick out spammers because it kills the value of the .xyz TLD.
It is rather disappointing. I run my personal blog and email on .xyz because it's great for graphics puns. Hotmail and gmail will accept my messages, but corporate email servers often seem to blackhole me.
I don’t mind the message being marked as `spam`, and I don’t mind looking up the spam list, but I feel scared when I can’t see it at all (has useful message been blocked by mistake?)
I've always felt conflicted about this. I generally support moving everything to HTTPS, and requiring it for new TLDs isn't a terrible idea because there's no chance of breaking anything legacy.[1]
On the other hand, Google owns the TLD, controls the HSTS preload list, controls the most popular browser. The idea that an entire TLD could be added to the HSTS preload list was a completely unilateral decision by Google. It makes me uneasy.
[1] ...unless you were using the domain internally assuming it would never be added to the root zone, which bit some people when they did this with .dev
> Our goal is to help makers find an awesome home for their project and not to help you flip.
We reserve the right to refuse, or cancel membership to anyone without explanation.
Nice, so only you're allowed to flip your parked domains.
> One surprising side effect of having a .xyz domain is that the mere inclusion of .xyz inside of a text message will result in a silent delivery failure for many providers.
Why are people afraid to use the real term for this?
It's called censorship.
Your provider is silently censoring your text messages. In peacetime. You can't expect it to improve when that's no longer the case.
Censorship has strong connotations of authoritarian regimes with political motivations. When they're doing it in peacetime with (I believe) a genuine goal to benefit the user by removing spam messages or by making abbreviated URLs like 'spot.xyz' clickable (even if that parser is broken, written by someone who only expected .com/.net/.org), it's just called parsing.
In much the same way, propaganda is just advertising with negative connotations, and a cult is just a religion with negative connotations. Calling all advertising propaganda or all religious people cultists is not likely to win people to your cause.
Censorship is part of an agenda. This is not censorship, and you calling that waters the phrase down. If I have a poorly constructed email filter, is that censorship? I wouldn't say so, and this example is more in line with that than with any active censorship.
So, in short, no one is afraid to call something censorship, I think they are just waiting for the right time. When it is applicable.
No, No, Yes. "Censorship" comes from the word "censor" - an intermediary who controls speech. Programmers need to stop assuming that every situation is scale- and convenience-invariant.
If the email provider says "Would you like us to add a set of rules people tend to add?", is that censorship?
There's a very fuzzy line somewhere, on one side of which a provider is helping users get what they want, and on the other is blocking content they don't want users to receive. I'm exploring where that line is.
While you have a right to send emails to me, I have a right to sign up for a service that automatically blocks emails I don't want to receive. The line is crossed when that service starts blocking emails I would like to receive. I'd say there is a pretty competitive market of email providers, and the rules are reasonably transparent about what's being blocked. Thus, it seems that "censorship" is a rather strong accusation here.
I would say the line has to do with how informed/empowered the user is about the initial content, ongoing changes, and their ability to make their own modifications.
The original comment was about text messages, of which there is certainly not a competitive market (the Ma Bell T-1000 has reassembled itself into only 3 remaining pieces), users were surprised at the behavior, and there doesn't seem to be a straightforward way to opt out of stupid rules like blocking whole TLDs. So it's a far way from being able to say that such blocking represents the will of the user.
What if users don't want to be informed, or make their own modifications? What if they just want a click a button and not receive junk mail, albeit also not receiving the occasional non-junk email because it had an unusual address.
I'd guess there are far more users like that, which is precisely why there are no major email providers offering the kind of service you talk about.
As always comes up in these conversations, while you have a right to speak, users have a right not to listen, and to use tools to help accomplish that.
You say you are "exploring where that line is", but then continue pushing a single sided view by conjuring some perfect user archtype who simultaneously has very strong opinions but also can't be bothered to express them. I know users are unreasonable, but completely discounting their agency does not make for reasonable analysis.
I've merely put forth a straightforward definition of "censorship" - one where there is a third party censor who controls the content of speech.
To translate your scenario into an earlier time - if most people in a society don't want to hear thoughts that conflict with the teachings of the church, and they appoint someone to an office to approve play manuscripts before they're performed, is that censorship?
If merely labeling centralized user-uninvolved content filtering with the technical term of "censorship" makes you uncomfortable, then perhaps you need to revisit your own assumptions.
Have you ever thought you might be wrong? Go and talk to the nearest non-technical person you can find and ask them whether they’d consider it censorship. It seems you’re in for a surprise.
I don't see the relevance of how the average person might respond. People associate the term "censorship" with badness, and thus react with cognitive dissonance when an idea that they like is characterized by an appropriate technical term that has negative connotations. I suspect this is exactly what's happening here - I've basically made some pretty dry analysis based on a straightforward definition, while you've reacted with a larger pro-third-party-filtering (ie censorship) narrative that goes way beyond anything I said.
> censor: to prevent part or the whole of a book, film, work of art, document, or other kind of communication from being seen or made available to the public, because it is considered to be offensive or harmful, or because it contains information that someone wishes to keep secret, often for political reasons.
Miriam Webster definition:
> to examine in order to suppress or delete anything considered objectionable
If I automatically delete content merely because I think you won’t be interested in it, I don’t see how that counts as censorship under the standard dictionary definition.
That said, you are of course entitled to your own opinion of what the word means. But please don’t be surprised if I think you sound a bit melodramatic suggesting that Google is carrying out censorship by automatically deleting emails offering me cheap viagra, especially when I can still see these emails by clicking on a link or reconfiguring the rules.
Your second definition doesn't even invoke a third party (it claims even more than mine), and your first definition only differs in that it references some motives. But motives are a bit of a red herring in the day of probability-based filtering.
I've never claimed "Google is carrying out censorship by automatically deleting emails offering me cheap viagra". You're the one that keeps invoking such hyperbolic strawmen - the original topic was the surprise blocking of text messages.
Honestly, I have no idea what you _are_ trying to say at this point. Something something censorship is all I'm getting. And long words like cognitive dissonance.
> Programmers need to stop assuming that every situation is scale- and convenience-invariant.
I’ve seen it termed the “All Ns are equally likely” fallacy. I.e. when programmers write code, they know that they should write different code for when N is 0, for when N is 1, but as soon as it goes higher, most programmers tend to write code which is optimized for arbitrary values of N, even though in actual practice N might almost always be, say, at most 10. This often leads to inefficient and overcomplicated code, where a simpler algorithm might be faster most of the time while still able to correctly, if more slowly, deal with non-typical values of N.
Given the context, it's absolutely a spam thing. When the .XYZ gTLD launched, registrars were incentivized to discount it aggressively, sometimes as low as $1 for the first year. Spammers loved this.
It's spam protection. Which I guess is a form of censorship, especially in a medium like SMS that has no built-in way to mark something as spam and still deliver it.
I'd argue its not censorship. If I don't pick up your call because I don't recognize the number, or my phone is on silent, is that censorship? Lacking an agenda or a message I'd hesitate to call anything censorship.
Like, without a censor actually redacting things or controlling the conversation, can it really be called censorship?
It's not censorship if one of the peers on the conversation do it. It's certainly censorship if a monopolistic or oligopolistic platform does it. And there's a lot of middle ground where things get hard,
>>It's certainly censorship if a monopolistic or oligopolistic platform does it
this seems like a good point, I'll have to think on it. For this particular situation I'm having a hard time seeing the argument on the basis that .xyz domains are cheap and get used for lots of attacks as stated in the article, so is it censorship or defense?
I think the question at the heart of my disagreement would be "what speech is being censored?", I don't see a compelling answer so I have a hard time seeing it as censorship at all.
It would get deep into the grey area if Google users had any capacity of enabling the communications with those sites. As of today, Google is the one in control of the communications, and dictate who can reach anybody over most of the internet. They just don't have a policy of empowering their users to decide who they want to talk to.
What speech is being censored is harder to discover. They are blocking people from communication without any feedback, and it would take a large effort to reach them and discover who they are. Certainly most of what is blocked is spam, but that's true for whatever block you implement today, unless you spend an unreasonable amount of resources targeting it into non-spam.
OK, I think we are probably pretty close in agreement. I still don't think censorship is the appropriate word here but also want to be clear that is what I am disagreeing about, not any of the larger issues.
Spam protection is very hard, so xyz was sacrificed because most of it was spam. Filtering out viagra pills junk mail from your inbox is also censorship.
Censorship would be intentionally preventing people from seeing something that they want to see. Here, the main intent is to prevent spam and nobody wants to see spam.
Even as a free speech advocate, it's hard to see a problem with this.
You'll find I agree that the concept of silently spam filtering on TLD of a link is something that pisses me off.
You'll also find that calling this "censorship" as if it has to do with government action, or that it has to do with the content of the specific site, is ludicrous.
Censorship is far from not limited to government action. I see this misconception a lot, and I believe it comes from Americans misinterpreting their first amendment.
If someone omits or deletes people’s texts for racist reasons, that is still censorship, even though it is not done for the opinions of the people who are being censored.
This is wild to me. Tested it out myself and I couldn't send an SMS with a spot.xyz link to/from Google Voice <-> T-Mobile. And no "failed delivery" notice either, just a silent failure. And yet I still get so many texts that are obviously spam or phishing attempts.