Using these tools to mitigate spam is as likely a scenario as this, imagine that every spam mail receives a masterfully crafted response showing utter fascination and interest in SEO, or helping out a Nigerian prince.
Every phone call to an unregistered number is answered by an artificial, frail, and forgetful lady that is trying her best to register gift cards.
When reporting an e-mail as spam it will not only block the address but waste the spammers time, rendering the actions unprofitable.
At the same time, it's not necessarily pleasant to consider the prospect of an internet where 99.9% of traffic is generated by AI-powered spambots engaged in adversarial games with AI-powered anti-spambots.
The open internet will be a dark forest, beautiful but mostly populated by metal-brains, meatb-brains will retreat to their human only spaces, local Facebook or Instagram where you know everyone in person or WhatsApp chat groups.
Btw, the chants from the book are real (and part of the interludes if you get the audiobook - which I recommend since the initially boring but ultimately important parts it aren't skippable without effort).
> In Neal Stephenson’s new novel Anathem, the Decanarian Erasmus’ daily chore is to ring the Clock bells in his “Math” in a special sequence each day as he chants out the sequence. Those sequences and chants are all based on mathematical formulae that composer and coder David Stutz has put together into an album called Iolet.
I like the thousander chant but I only have regular laptop speakers. What do you recommend to really "get" the song? What would I hear beyond those super low basses?
A good pair of headphones would be my next choice.
It's not much "under" it - but rather the fill your senses with what it would be like to be in the Mynster as each nave did their chats in the great octagonal hall. And while its a passage about the Hundreders...
> We sauntered across the meadow to the Mynster. Even so, we got there in plenty of time, and ended up in the front row, closest to the screen. Voco continued ringing for some minutes after we arrived. Then the eight ringers filed down from their balcony and found places farther back. A choir of Hundreders came out into the chancel and began a monophonic chant.
Or the description of a Thousander chat
> ... As I walked toward it my perceptions cleared suddenly and I shook my head in amazement at my own silliness in having imagined it was an amphibian or a truck. It was plainly a human voice. Singing. Or rather droning, for he had been stuck on the same note the whole time I’d been awake.
> The note changed slightly. Okay, so it wasn’t a drone. It was a chant. A very, very slow one.
> I could have stood there watching and listening for hours. I got the idea—which might have been just my imagination—that {spoilers} was singing a cosmographical chant: a requiem for the stars that were being swallowed up in the dawn. Certainly it was music of cosmographical slowness. Some of the notes went on for longer than I could hold my breath. He must have some trick of breathing and singing at the same time.
Also in Reamde and later Fall. I forget exactly, but i think in Reamde they flood the internet with shit, and then some years later in Fall everyone has curated echo chambers of content, such that people signed up to different streams are basically living in different realities. Which i guess is just the logical conclusion of what we have now.
There are still solutions in that case. Email addresses only allocated to verified humans, which only accept email from the same addresses or specifically allowed outside email addresses.
Alternatively, email addresses that are necessarily paid for that have maximum sending limits or extra costs above those limits, so those addresses are not profitable for spam bots to use.
I could imagine some kind of “super priority email” that costs a dollar or something, that gets prioritized and is very unlikely to be marked spam.
More ideas: Maybe the amount paid per email sent changes depending on the percent of previous emails marked “accept emails from this sender”. Maybe some percentage of the e-mail fee is given to the recipient of the email.
Note that if a highly spam resistant email account is adopted by everyone the amount of spam sent may fall significantly. Gmail doesn’t count because it still isn’t as good as a system where sending millions of spam emails is simply too expensive.
Cyberpunk 2077 features a world after this has happened. Not because it happened, its just after. If there is a reference to an older internet it features its disuse due to this.
Banks also waste energy deterring people from robbing them. Internet or not, we'll have to stomach a good amount of bad stuff in order to get to the good stuff.
Cyberpunk 2077 features this reality. Its not discussed much, but basically only local small scale networking is done (like citywide scale) because the broader internet protocol has either crumbled or is polluted by adversarial bots. So its pretty taboo to go there or simply unused.
I'm sure someone else knows the lore better. There is a lot of supporting literature, just not a key component of any stories.
I think the lore is that the wider internet was intentionally taken down by a virus made by a master hacker in the past. Any attempts to connect with it gets you killed by either the virus or "feral ais", but some of the mega corps pay people to explore it every once in a while looking for stuff.
And a plausible way for them — on either team — to take over completely. Both treating us not even as pets but as grass. In this analogy the good AI are gardeners, the bad AI are cow farmers.
>imagine that every spam mail receives a masterfully crafted response showing utter fascination and interest in SEO
That would necessitate reliably detecting the emails as spam in the first place though. False positives in particular could be devastating. Imagine a chat bot coming up or going along with business proposals in your name for example.
You could do it with humans - every time you click "mark as spam" it doesn't just trash the email, it begins a long and drawn-out chatGPT conversation with the spammer, stringing them along.
Look up 'pig butchering' when you get a chance. It is an elaborate process of convincing your mark to fall in love with your persona and asking them to send it money. It takes months to years to execute but marks end up losing their properties, retirement accounts, life savings, everything to it.
There's lots of legitimate email traffic that would find itself stuck in here. I could see business questions being answered and those answers actioned on. Or legitimate sales prospecting resulting in actual orders being placed. If you choose to let a tool do your communication for you by impersonating you to the extent that another person would reasonably expect that they're talking to you, then I'm not sure you can just say "lol, that just was my spam bot" as a way of getting out of it.
> Using these tools to mitigate spam is as likely a scenario as this, imagine that every spam mail receives a masterfully crafted response showing utter fascination and interest in SEO, or helping out a Nigerian prince. Every phone call to an unregistered number is answered by an artificial, frail, and forgetful lady that is trying her best to register gift cards.
Right. And who the fuck would pay for GPU time for that ?
Apparently phone scammers hit Americans for $40B in 2022. At this point the economic incentives are significant enough that a federal budget line item may be in order.
Sure, turn loose these tools to answer the actual spammers/UCE. But:
Speaking as an ISP, if somebody turns loose what is clearly an AUTOMATED tool shitting up the contents of my abuse@ispname.com inbox with reports from some software script, I can guarantee you it goes to /dev/null
At some point we will just block their MX at the SMTP transfer point and call it a day.
98% of that already is abusive DMCA rights holders who are ignoring our federally designated DMCA-agent address for copyright violation complaints. With their automated 3rd party things complaining about people torrenting Yellowstone or whatever.
Actual reports that are clearly written by a human saying "hey it looks like this /32 of an IP address is compromised as some sort of botnet" will get a thousand times more attention. Or the very rare cases where we have a network-engineering emergency escalation and somebody calls me on the phone.
Anything generated by chatGPT or similar will be clearly obvious enough that it matches a similar pattern and comes from an automated script.
I wonder if that inspires new platforms or types of tech that verify someone was typing it in vs pasting. But then do bots get better at typing it into the input boxes? Ugh.
Sure, but most spam will go the other way. When you're in a forum and you express an opinion that doesn't fit what the bot swarm is designed to police, they will gang up on you and argue against you from various angles, and you'll think your opinion is just super unpopular. If you are a famous person they'll do reputational attacks, after amassing karma points and followers across the swarm. Then they'll move on to combating the remaining human-powered outlets like nytimes.com, becoming the next Vox or Vice, but totally AI-powered. Their bots will spread their articles over the other ones. And finally, having driven humans underground in the dark forest of social media, they'll basically dominate the Internet.
The real sea change will come, though, when bot swarms control capital. For example, amassed for doing spam tasks. That capital can be deployed in a variety of ways, but the point is that corporations and networks will prefer bots because they generate more social capital, in coordinated ways, and also amass more financial capital than humans.
If you think this is far fetched, IT ALREADY HAPPENED on wall street and hedge funds. Bots have replaced people in trading and control the capital. Humans who trade among the bots get fleeced and don't even know it.
If as a (hypothetical) Nigerian prince spammer I get a "masterfully crafted response" from a mark, it would be obvious right away that I am talking to a bot. The kind of people who respond to such bait would hardly be able to write anything like that.
A "masterfully crafted" response to a nigerian prince spammer would probably sound a lot like a person who can barely write, possibly with what sounds like the beginning of dementia setting in.
A lot of spam is just a link to a page for someone to download malware or enter card details. It is already relatively easy to get these taken down if you care enough to, but a waste of time as the senders have moved on by then. The idea that spammers cannot make something that people tricked by their scam will use, but that insulates them from time wasters, is ridiculous.
So the two AIs will be talking to each other, trying to suss out if the other is fake (a sort of Turing test), trying to con the other to keep talking or to really buy in?
> Princess Bubblegum is concerned over her mortality, and reveals her designated successor to Finn and Jake. However, things get out of control when the heir turns against Princess Bubblegum.
> […] The heir is Goliad, a pink Sphinx with a mound on her forehead […]
> […] Goliad reveals that her mound concealed a third eye, and proceeds to psychically control Finn and the obstacles in the course for a perfect completion. She explains that with her in control, everything would be perfect. […]
> […]
> […] Finn and Jake confront Goliad but are beaten by her psychic powers. Goliad tries to read Finn's mind, and Finn narrowly avoids revealing the plan by interrupting his memories with nonsense. The new creature; another sphinx with an eagle's head, white feathers, and golden hair; rescues Finn and battles Goliad. Goliad tries to convince her brother, named Stormo, that they should work together, but Stormo refuses by screeching. They engage in a psychic showdown, but with their powers matched the two creatures are eternally locked in a mental stalemate.
> I’m no AI or machine learning expert so I don’t know how it works. But I am also worried that spammers could use ChatGPT to get around Gmail and Outlook’s spam filters.
This will not only increase the spam-problem, but will most likely be used to scale and do targeted phishing attack as well. I wrote an extensive article[0] where I analyzed this. And to no surprise, GPT-3 can be used to generate dynamic phishing campaigns on the fly in multiple languages, classify email responses, improve email thread hijacking attacks etc.
People are overestimating the importance of the message body in spam classification. The stuff that appears in your spam label on gmail is what google considered marginal, almost ham. The vast, vast majority of what they think is spam is rejected with temporary failure codes at SMTP time and never gets delivered with any label. IP reputation and other related metadata features are the key features in spam classification, and repeatedly sending different messages is not a valid test of whether the body looks spammy or not.
Yeah, I’m starting to hear and understand that more myself. There was a extremely long twitter thread (i think from former Reddit CEO) that said the key to content moderation is moderating bad behaviour, not bad content.
I concur on this for the most part because I would say that my custom postfix + spamassassin + opendkim setup, on my self run MX, correctly classifies 75%+ of the spam or outright rejects it for SMTP transfer just based on:
a) invalid rdns of other mx
b) invalid spf
c) invalid DKIM / no DKIM signature
d) failed RBL list check - I subscribe to and feed it a few different common sense SMTP RBLs
Rejecting as spam things in the above category before it even looks at the content.
Adding a high score for invalid rdns, spf or dkim before something generally similar to spamassassin or a more advanced message subject line/body analyzing system begins classifying things help.
And then additional score is added of course for text spam content in message subject line and body.
Which RBLs do you use/suggest? My main anti spam solution is currently just Thunderbird's adaptive spam filter, but that doesn't help when I access it with a different app or via web.
One signal for spam is just the same email going to many different people who don't like receiving it, but "the same email" doesn't have to mean it has exactly the same text.
As I have said before, the future will have two kinds of AI everywhere.
Their AI to get you to buy something, do something, believe something, or in a warzone to kill you and Your AI to protect you from Their AI. Reality may even become so dangerous and illusory that humans lose a lot of their agency to Your AI.
All this can be done since GPT3 API has been available.
I see a lot of people thinking chatgpt is something new capable of such stuff but GPT3 is far less restrictive and has been able to do all this for almost an year now.
But you must pay to use these APIs. They did give me $20 free trial though. You could make a bunch of accounts and abuse the free trial I guess. It must be cost effective for the scammers.
You can change up the prompt to change the writing style so spam filters will have trouble catching this new world of spam.
AFAIK all it takes to bypass Gmails spam filters is to resend the same scammy email that was Flagged but using a different email address. I get the same kind of scammy emails, flag them as spam, and then a couple days later, Gmail lets the same email in though it is coming from a different 123134r12345124@blahblah.com address.
In my experience the new strategy is to just send an email with no body, no title, optionally with an image of a conventionally attractive lady, in the hope
people respond after which they're automatically whitelisted (as there's now a 'conversation').
I get a lot of those. Never download the image so I don’t know if there is an attractive lady or not. I do always mark them as phishing spam for gmail but it just keeps delivering more. I lost count of the free Yeti mugs I’ve missed out on.
Anecdotally, almost every email I see in my gmail inbox is advertisement of some sort. There's newsletters I never signed up for, special offers from companies I've never had dealings with, it never ends.
Some of it hasn't even been sent to me as an email, but shows up in the inbox as though it was an email.
Granted, there's fewer scam emails than in my non-gmail inbox, but man is there a lot of spam.
well, you don't need ChatGPT for that. I receive daily scam in my gmail with 99% the same content each time (and i mark it spam each time)... the worst part ? the server serving it is google (but the email attached to it is always a different obscure email)
I must be in the extreme minority upon reading comments on email spam, as I almost never receive a spam message in my main gmail inbox that is truly spam and not something I'm getting an email for because I lazily opted-in to a mailing list or was put on one due to association with a product. Actual, oldschool spam (Nigerian princes, claim your prize etc) literally never gets to my inbox. Maybe I'm just not a popular person. Or maybe I've kept my email private enough that it doesn't get scraped for such things? I don't understand the discrepancy between my experience and so many others here.
It makes sense in itself, but then why do you think the second email went through in this case, considering no factors other than content didn't change?
Interpretations I've seen of ChatGPT type tools, and playing around with it, I would sum up as "reducing the marginal cost of creating BS to $0". Great for content farms, spam, disinfo/propaganda campaigns.
Stuff where it doesn't have to be correct, have a high hit rate, or even be edited. Just need to produce plausible enough sounding, human-like content.
> Gmail’s spam filters are Google’s weakest tech. At least I don’t know of anything worse.
I get a fair amount obvious spam coming through the filter, but the issue with any sort of classification is the tradeoff between False Positives and False Negatives.
The occasional False Negative causes a lot less damage (2 seconds to delete or report as spam) verses the damage of a False Positive (not seeing an important email for two weeks or ever).
I can’t find it but Gmail actually had some sort of whitepaper or something once about Gmail-originated SPAM and how it became a huge problem which was partially the reason they started doing phone verification.
The parent’s comment is valid. Any modern email peer is doing domain based reputation which is possible thanks to SPF and DKIM, and if you don’t have those configured you’ll have a bad time. Then it’s the job of the domain owner or email operator (postmaster) to make sure you’re not blasting out SPAM and respond to abuse feedback. If you think about it, this is the only sane way for email to function without preauthentication.
The only major outlier to this is Outlook, which is still doing IP based reputation. And of course a long tail of small server operators that rely on legacy SPAM lists from decade ago and reject only legitimate emails and pass through plenty of Viagra ads.
In addition to further emphasis on "trusted senders" in the form of contact books, we'll also hopefully see a rise in identity-validated S/MIME. Though I get the feeling it'll hurt really bad before either gets deployed to a sufficient extent.
When reporting an e-mail as spam it will not only block the address but waste the spammers time, rendering the actions unprofitable.