I'm just waiting for someone to hook this sort of stuff to some kind of machine learning system, to learn and adapt and come up with new replies. And some 100 years hence we'll be explaining to kids that the singularity arose out of people's annoyance with spammers ;)
A friend of mine is actually running clustering right now, we want to try and classify the emails into categories automatically when a new one gets forwarded (so we can pick appropriate responses).
I think that this or something very similar was a plot point in Neal Stephenson's Anathem. Cannot find a good reference, but it was pretty entertaining.
I've always wondered why smartphone apps don't have better access to the telephone portion of the device. Why is it so difficult to do something as simple as recording calls? It would be neat if I could get apps to change my voice or encrypt the audio.
I don't know if this is why, but there are a bunch of laws and stuff about what you can and can't do on a telephone. For example, many states have limitations on who can legally record a phone conversation, with or without various forms of consent. Maybe phone manufacturers don't want to facilitate easily breaking those confusing laws.
They do mind, there just isn't a clear cut of a solution they can force someone to put into code. You can't reliably detect if the phone is being used by the driver or by a passenger.
Without being able to make that distinction, it's nearly impossible to force phone software makers to write rules preventing "illegal" phone use in situations where it is illegal.
Instead, they punish using verifiable methods such as direct observation (cop sees you doing it).
The situation may be similar for recording calls (I'm not familiar with exact laws in this area), but I'm assuming there are also privacy restrictions in calling which lead to the software restrictions.
http://www.ipadio.com/ lets you record and share your calls with others. It works okay. You basically conference call with a recording phone number.
I remember once using something called HearSayApp, but it disappeared off the face of the internet without a trace. It used HTML5 to place a call, somehow. And the audio was crystal clear.
It's probably got something to do with some kind of regulations. Voice changers are fun, but not when calling 911, for example. Maybe it's a requirement that the sound goes directly to the baseband or something?
I don't buy that argument. What liability would they have when I choose to do something?
They could get around it the same way Toyota limits their liability from me using the touch screen while I drive - pop up a message saying when and how the feature should be used.
The difference is that fiddling with a screen is a traffic ticket and potential accident where you are at fault for being inattentive.
For call recording, the phone maker is potentially facilitating a felony. It's really complex. Does the phone stop recording when you cross state lines? Does it allow covert recording?
If I'm from California recording a call in DC on vacation and accidentally enter Maryland and get arrested, I'm suing the company.
The phone thing is complex too. Where I live (suburb of Austin), it's legal to use your phone but about five miles down the road it's illegal.
Likewise, if you are in California, you can't send a text while driving, but you can enter an address into the GPS app (probably).
All Apple and Google would have to do is warn you that recording calls may be illegal.
Radio Shack sold phone recorders in every state and AFAIK, nobody ever sued them.
I have Google Voice and Google lets me record calls on that. I'm not sure what the big difference is for when the call is over Android.
Edit: There have been cell phones that had call recorders built in, just none that run Android or iOS (AFAIK). I know a lot of Motorola phones included the functionality.
If it is a single-issue negotiation (i.e. know your own bottom line and there is no BATNA https://en.wikipedia.org/wiki/Best_alternative_to_a_negotiat... other than this one choice), therefore you're not afraid to lose (i.e. there are lots of potential suitors), then bargain like a maniac. The only thing they have to hold you by is a time-limited offer, but by doing this they've revealed part of their own non-bottom line (i.e. fear of losing is part of their BATNA), so explore this.
Have recommended before and recommend again the book 'Getting to Yes', and Harvard's very open research and sharing in this area: http://www.pon.harvard.edu/
Getting too serious? Not at all. Harvard's role-plays that can be purchased very cheaply ($1-3 per copy) are great teamwork activities.
They tend to focus on numerical amounts denominated in dollars, but these financial numbers can be easily substituted for time, number of people, anything that's a number. Practicing new concepts with things you're immediately familiar with tends to lead to remembered solutions. Instead save the comparison and application to the postmortem.
The problem with Spam has always been the minuscule cost of doing it. If the cost can rise above the profit, spam could become unattractive. Imagine if spam attracted a sudden swarm of bots, just as a new email addy attracts spammers now.
And bots could do pretty well there. I recently read an article (cannot find it; but it may have been even linked from here) which explained the benefit of poor grammar in the spam (i.e., they want to put off more knowledgeable users as the cost of the follow on emails is significantly higher).
El cheapo bots have pretty basic English skills which can help them fit naturally into the spam target patterns.
The same logic that favours using an opening gambit that scares off all but the most unworldly of people works against using bots for the followup conversation. You don't want to finally find someone that genuinely believes that a random central African stranger might want to split $138,400,000 with them and has funds ready to send only for your bot to fail the Turing test by misunderstanding their question about how to pay the deposit.
Also, anyone smart and knowledgeable enough to be able to write bots capable of persuading exceptionally gullible people to part with their money is probably capable of moving to the next level and going after the slightly-less gullible, who are more numerous and richer.
It's also probably a bit flattering to the average scammer to assume the flaws in their patter are a deliberate filter pattern though. They're just as likely to open with a highly plausible (probably copied) offer of high value second hand goods on a listings website that probably attracts dozens or replies and then respond to each of those genuine expressions of interest from people willing and able to pay with badly-written response(s) that conflict with the detail of the original ad, not to mention totally forgetting that little old Scottish ladies don't have email addresses registered in the name of Nigerian men.
"It's also probably a bit flattering to the average scammer to assume the flaws in their patter are a deliberate filter pattern though."
It doesn't have to be deliberate. It probably evolved. If worse spelling and grammar work better than correct spelling and grammar, then even if no spammer ever consciously says "Ah, I need to make this crappier", the spams will evolve to be crappy.
Moreover, while our brains in this discussion are taking cognitive shortcuts and putting spams on a "correct <-> crappy" single dimensional continuum, the reality is probably even more complicated, and it's actually specific crappiness that works better than others. For instance, I could hypothesize that it's not just typos, but typos that the recipient plausibly believe are the result of a non-native speaker from the relevant country. (I say "plausibly believe" rather than whether the non-native English is truly representative of the relevant country, because that's what matters.) That's an obvious possibility; more subtle things are possible and probably even likely. And again, the spams will evolve into those more subtle possibilities and exploit them even if no individual participant is sitting there and trying to deliberately figure out the best spams to send... which is, obviously, almost certainly untrue as well. And whoever is sitting down to write The Perfect Spam is probably doing so with a lot more data and a lot less scruples than you or I would apply to the problem.
I've often thought one of our best anti-spam defenses is the sheer mailbox stuffing quantity of them. Even the best crafted spam about how I won the lottery I never entered looks a lot less plausible the thousandth time I receive it.
Occam's razor implies that if the people originating the scams mostly are based in central Africa (true) and display the specific crappy spelling patterns and linguistic quirks of many other moderately-educated central Africans using the internet for non-nefarious purposes (also true) one needn't assume that there's any particular process that leads people to write like their compatriots. Especially not when similar imperfections betray their attempts to masquerade as a Scottish widow selling their husband's canal boat or Australian trader that wants to buy your product if you'll send him the import tax.
I'm also little inclined towards scepticism there's much evolutionary optimization going on when the standard email scam format hasn't even adapted to get around now unbiquitous standard webmail spam filters, which are essentially orthogonal to the gullibility of the account owner.
Besides, wouldn't an evolutionary process that wasn't a conscious attempt to avoid generating false positives (or slavish copying by people that don't really analyse in much depth) tend to optimise for reusing the emails that generated more responses rather than less?
The big money in spam is not in selling Viagra, it's in selling spamming.
I'm acquinted with someone who does spamming for a living, mostly in the online casino segment. He does very well, and surprisingly polite company cares more about the money he has, than the way he makes it.
Replace "spamming" with "marketing" and that makes less sense. "The big money in marketing is not in selling Viagra, it's in selling marketing". Spamming is a form of marketing/sales, and Viagra is one of the products being sold.
I hope you and your family are healthy and happy. As I have detailed in my previous correspondence, my offering to you is a way to waste spammers' time by making them reply to emails. I am sure your esteemed self would have great use for such a service. Please wire $1500 to my account for attorney's fees.
Hello Stavros,
I'm not sure I understand. Could you go into more detail?
*In all seriousness though (lol): You could have different types of personalities, depending on the email address of the person who doesn't want the spam. Like someone pretending mnesty.com is their work address but is still treating it like a personal message. That way the spammer will disregard the fact that the reply email isn't the same as the original.
EDIT: Nevermind, I see you said you did that already... I think. ... Help?
It's too hard to write generalities, I'm afraid. You can only write about 10 different sentences before you start to repeat yourself (or at least I can't do it). Feel free to send me sentences for me to add!
This is amazing. Looping in my supervisor. [add a second email agent to thread. The two email agents will fork the email chain for the spammer and ask similar questions, hopefully wasting more of their time.
I'm so sorry, but I think I lost my whole inbox. Could you please send me a recap of our whole conversation?
[sent with the original seed email as body]
True, it is tricky. But it'd be a fun exercise. I suppose it's an exercise in dark humor... hm...
"Hey, I'm switching to my work email. Refresh me on the details again."
"I'll have to get back with you. Can you repost a summary?"
"I'm sorry, but your emails are hard to read. Could you format them alittle nicer. Thanks!"
"Gee, I'd love to help. It's hard to think of how, though."
What you really need to take this to the next level is to be able to crowd source some content. Maybe check if you have a response that you think will work and that you haven't used yet and if not ask a pool of users for suggestions. You could use the fork suggestion above to do some testing among the proposed responses.
Have you considered word replacement lists? Like for "This looks really cool", "This looks really[cool, great, awesome, other synonym]". Might string people along longer.
Lenny is great. My 11-year-old and I play the Lenny game sometimes -- one of us is telemarketer, the other plays Lenny. Lenny is a brilliant piece of software. I just worry that eventually most call centers will be onto Lenny. Already some telemarketers recognize him. Luckily, as AI and voice recognition improve, more such systems will come online for the amusement of youtube audiences and to thwart telemarketers and scammers.
I'm guessing the emails this generates will become flagged as spam themselves. Do spammers use spam filters too or would that prevent them from getting replies?
Spam filters could do very little, because it's almost impossible to tell whether a conversation was run by a bot - and it's outright impossible to do so for a single response.
However, you raise an interesting point: blacklists could reasonably be put into place. I'll make a merge request to use Spintax.
Very nice, and a great test bed for AI code. Lots of low hanging fruit, like follow-ups to specific branches of conversation, extracting certain information from the spammers replies to make the answers look more real, etc.
Someone should organize a competition to see who can create the longest running conversation (or the funniest one).
Yeah, that's an interesting aspect of it for me. I'll have to clean up the code a bit, because it's very "weekend project" quality, but it's an interesting way forward.
If you are using the free plan, note that you aren't really getting DDoS protection. They offer "I'm under attack" mode which can be bypassed easily and it does not protect against l3 and l4 attacks.
I love this. I've actually presented on the concept before - more so about ensuring that people targeting your user base have to waste a lot of time without realizing it by doing things like shadow bans weeks before actually "detecting and cancelling their account" but this is a perfect example of the concept.
Have you thought of adding in a check in if they haven't responded after a week?
Perhaps, "Hi there, I thought we were really making progress, but I haven't heard from you for a week. Is there still a chance I could get in on this great opportunity?"
Yes, but right now it's very event-driven (each email triggers a new one), so it would be more complicated for little benefit (given that spammers wouldn't reply anyway, they only stop replying when they think you're wasting their time, and that wouldn't change with an additional email).
1) If lots of people start using this, spammers might learn to avoid wasting time with Mnesty's CEO. Having a few more fake companies would help.
2) The bot could probably be made a lot smarter with some natural language processing. But it seems to fill its purpose pretty well already. I think this just shows how desperate for leads spammers are.
I would also suggest giving the spammers more intense work to do. Like a pdf form for new suppliers that they have to download, fill out, sign, scan back in, and return. Or maybe submit to a form page with a horribly difficult captcha.
Not just clear the fields but change the field names using server side scripting to keep track of the changes that way auto-fillers such as RoboForm can't be used.
edit Also re-order the form locations, placeholders so it's nearly impossible to script automation for the page!
What do you mean by unlisted? I forwarded it, then received a mail from smannesty containing direct link to the thread.
EDIT : oh, I also removed the private key from url param before posting it here.
EDIT 2 : typo in my initial message: by "followed", I meant "forwarded" (in french, we say "faire suivre", which translates word by word to "make follow")
Another important one: I just had to delete a conversation because my email was still in there. You should probably make the email stripping more aggressive, and make it look throughout the whole email.
Edit: just noticed I get a chance to edit the conversation. That's pretty cool, although of course it would be even better if I didn't have to :-D
I was going to say yes, but then I realized it needs a bunch of MX/etc settings on your end, so let me streamline the process a bit more (and possibly add something to the README). Thanks for the offer!
> Hello,
I have wired the funds you asked for. Could we please proceed to the next stage?
Daniel Wong
CEO, MNesty, LLC
Is that a canned response? This happens later on in the thread (does it recognise that the topic of the conversation is "funds" rather than say "product")
It has various categories of replies, which right now you can specify when you forward a message (it sends you a management URL back). Later on, when I have enough data, I'll do some simple ML to have it pick a most likely category as a default.
Each category comes with its own set of responses, but you can see where the category changed because the bot will start talking about a product and then switch to talking about funds.
It'd be more effective if it didn't re-use responses later in the chain (unless it'd totally run out of alternatives). Although, arguably, it's funnier when the spammers/scammers do continue to find new ways to reply to the same couple of questions.
Ahah, this is fun :) But please someone, let google know this exists. I just had a forward refused by gmail because "it contains viruses", and I realize I've forwarded 5 spams before that. I would not want google to think I'm "actually" sending spams :D
> Spammers will usually realize they’re talking to a bot (or at least to someone who isn’t interested or not going to give them any money) after around 3-5 messages, but some have sent up to 15 messages before giving up in frustration.
It turns out that some DNS servers cached the old nameservers and are now failing to resolve those. It's unfortunate, but all we can do is wait for the cache to expire :(
Funny thing is that most of the people who downloaded it came from Nigeria and seemed to think that it was a tool that allowed them to scam people. It was quite funny when they put their own contributions and then one of them was talking to a real scammer trying to scam him.
Had to delete the thread cause it didn't make much sense though.
This could works in short run as long as this type of service stay as niche and not too popular. Spammers always, always have a creative work around it. I imagined a counter measure from them by build a bot detector that will spam back the spambot and one hell of continuos email loop doesn't do any good to our internet traffic.
I've tried forwarding a spam email from Apple Mail, but I got a response that it didn't work out. This is the raw email message I sent - http://pastebin.com/qNwT04my
"I guess adding a delay would improve the bot's credibility by a lot!"
And effectiveness in stealing spammers' time. If the bot immediately auto-responds to each subsequent (presumably human-initiated at this point) email, then the spammer may more easily follow the thread.
On the other hand, if the bot waits 30-90 minutes, the spammer likely has gone on to other things. There's an additional (albeit minor) cognitive load here if they want to engage with a potential sucker.
It actually has a random delay of 1-8 hours, which shows in the timestamp. It only sends the first email right away, because it presumably took you some time to forward it to Spamnesty.
The box these are running on is already pretty full (it's where I keep all my side-projects, and it's got around eight running on it now), so I'm trying to keep it lean. Features definitely get cut on a cost/effectiveness ratio.
Lol. I used to bait spammers as a hobby, and after a few exchanges with one, I'd tell them how lucky I was that I had been identified as the real owner of $324 million in a lost government account. That I had to have been the luckiest person in the world, because not only that, just last week, I won the national Nigerian lottery, and the week before that, a distant great-aunt from Nigeria had named me sole heir in her will!
This is not a bad idea. Perhaps if the major email providers had bots that did this sort of thing, spammers would be overwhelmed with responses, with the tiny proportion of real fish lost in the mix.
There appears to be some problem with DNS after I moved to Cloudflare. Keep refreshing, it will work, but sometimes people get NXDOMAIN/SERVFAIL when resolving.
I have never understood what were the goals of these messages. People saying they have a fund of millions. Is everything arranged for me to send money to them? Simpĺe as that?
I half-fell for one of these once. I was a teaching assistant with my contact info posted on our department page, and a woman in Africa wrote seeking a tutor for her son who would allegedly be doing a summer program at my university to prepare for enrollment in the fall. It all seemed reasonable until she wanted to wire funds to my account. I told her to have her son pay me in person, and she never wrote again! This was a pretty good scam because there were no millions involved, and she'd done her research to match the pitch to the mark.
In addition to the other possibilities mentioned, another common scam is to pay you too much, and ask you to send back the difference. Then the original payment gets reversed or turns out to be fraudulent, and they keep what you sent.
A lot of those "make $$$ working from home!" things are scams like this. They'll offer you, say, $1,000 for a small amount of work. On payday, they'll send you a $1,500 check, along with some excuse for why it's too high, and ask you to send them the extra $500. A couple of weeks later their check bounces, and you're screwed.
Another technique is to just ask you for help transferring money. They might send you $4,000, tell you to keep $500 to compensate for your time, and send $3,500 to their friend/partner/subsidiary outside the country. Then the $4,000 evaporates. This one is especially clever since the original scheme was probably some form of money laundering or other financial crime, so the victim will usually be reluctant to go to the authorities afterwards.
Don't know where GP is, but in USA you can forge a check with bank and account numbers. Of course the check must still be cashed, but perhaps a string of deposits and new checks on different accounts would make this take long enough to get actual cash...
The stereotypical 419 scam, however, has some "plausible" reason why the main transfer has to wait on a smaller transfer in the other direction.
Wait, you want me to type in my username and password into your computer, and then you can take any amount of money from my account, and all everyone needs to do is trust each other?
Basically, they spammer is hoping to milk you for "fees" that they will say are necessary in order to free up/transfer the money that they have waiting for you.
One of the goals is to find someone who'll fall for such a scheme. If it was more plausible, they'd waste time on someone who'd end up not pulling the trigger when they realized it was too likely to be a scam.
Once spammers start using bots to carry conversation, this might change. If bots can handle part of the communication, it probably makes sense to start planning more subtle scams.
The problem with hashcash for email is that there's no way for the sender and receiver to negotiate the amount of work to be proved, so they must correctly guess how much to perform/require. The most sensible requirement to guess is zero, since all existing email is sent without any proof of work, and the value of interoperability is higher than blocking spam. After all, if you're willing to lose interoperability, you can block 100% of spam by just not using email at all!
Note that it may be possible to have servers negotiate an amount of work to perform, but having someone else (the server) perform the work defeats the point of the system. The client must be told to do the work, but it's a core feature of email that it's not realtime: messages can be composed offline, and queued up by servers, so by the time a "needs more work" response is received, there's no way for the server to send a message back to the client.
Could the work requirement be advertised as part of the destination's MX record or something? I know that in theory you're supposed to be able to send the message to your upstream server and let it worry about how to deliver, but in practice it seems like these days you can count on being able to look up the target's DNS info.
Another problem is finding a quantity of work which is high enough to stop spam, but low enough not to stop legitimate traffic. You want people to be able to send email from their RPiZero or original iPhone, but stop spammers who may have massive botnets, or at least decent PCs.
> Could the work requirement be advertised as part of the destination's MX record or something?
The frustrating part is that only the good guys would follow such a scheme. As long as there's a fallback available, no matter how deprecated, spammers will use it to avoid such payment schemes.
I suppose it could be used as a strong signal for spam filters, in addition to other security/authentication schemes which have been adopted.
> Another problem is finding a quantity of work which is high enough to stop spam, but low enough not to stop legitimate traffic.
Yes, this is another problem which negotiation could alleviate. These days there would also be the option of offloading the actual email sending to a service (either run on your own server, or "in the cloud"). Of course this just shifts the problem, but at least such services can provide arbitrary APIs, and hence can restrict their hashcash power to authenticated users, or require users to provide proof of work in a way which can be negotiated down, e.g. by building up a reputation over time.
Not sure how botsnets can be tackled, but raising the amount of effort even a small amount would hopefully make a lot of spam-based scams unprofitable. I'm sure the problem would just shift then, e.g. to using the botnets for more DDoS attacks or other more lucrative schemes.
I can't think of a way to make this work with email, but I do think a proof of work like hashcash would be useful for Web APIs instead of using API keys:
- Give a 402 reponse if hashcash isn't included or isn't enough, with a link to a price URL
- GETting a price URL may return different values depending on identifying information like IP address, user agent, etc.
- Prices can be adjusted based on server load, whether we recognise this agent as malicious or benevolent, etc.
- Different types of request can have different prices, e.g. GETting a specific resource could be cheaper than searching or performing some expensive computation.
There would also need to be a mechanism to avoid replays, which would make things slightly less RESTful. I haven't thought of anything more useful here than an increasing request ID.
I've done a crappy implementation for my blog to fight spammy comments. It works OK, probably more because it's a totally custom thing that isn't worth time for spammers to fight, rather than because it's actually effective.
The way I did it is:
- When the user focuses in a comment field, the page makes a request to the server asking for initial parameters.
- The server returns the number of leading zeroes required, the number of distinct hashes it needs, and a salt to use. (This is just a constant in my code right now, but could be varied based on client specifics.)
- The page then crunches on the work as the user types their comment. The submit button is disabled until it's complete.
- The proof of work is submitted to the server along with the comment. The server then checks to see if it's good and accepts or rejects. A properly working client should never be rejected (since it fetches the required parameters in advance) so the rejection doesn't have to be too fancy.
- Replays are prevented by storing the salts in a database, and deleting them once they're used for a comment.
I changed the standard hashcash technique a bit by requiring the client to submit multiple distinct hashes. Requiring only one hash works fine, but results in a lot of variance in how long it takes to compute the proof of work. You might tune it for an average of 30 seconds, but a decent percentage of clients will get it in 1 second, or will take 60 seconds. By, for example, requiring 7 fewer leading zeroes but requiring 128 distinct hashes, you get the same average but with a lot less variance. You can also display a semi-accurate progress indicator this way. The downside, of course, is that you have to send more data and the server has to do some extra work to verify.
My web site is a hacked-together thing that's been gradually accumulating since the late 90s, so it's kind of ugly. The hashcash code is not very modular, either.
If you'd like to see it anyway, I pulled out the relevant parts here:
The comment-inline.js file is directly embedded in the HTML for the comments area, and is the glue code between the actual UI and the hashcash computation code. The hashcash.js file is where all the client-side work happens, and it handles the actual hashing, making multiple attempts, checking to see if an attempt produced a good result, and such. Then commentsubmit.py handles the server side by returning hashcash parameters when requested, and checking the provided hashcash for validity when submitting.
I have a brief blog post about it here, which you can also use to see the system in action:
Instead of hashcash why haven't we solved the problem by just attaching real money to every email? The convention would be that you "accept" the payment only if it is spam. $1 gets you into the inbox. If it isn't spam then after 48 hours the token just expires and the sender keeps their dollar.
I assume the money transfer can be solved today using Bitcoin, and it wouldn't be a terribly complicated protocol.
This is exactly the same problem faced by hashcash, just wrapped up in more layers of complication. If emails require a $1 attachment to get into your inbox, you will receive no emails, since nobody is currently sending any emails with $1 attached. Just rewrite my above comment, but replace "hashcash" with "dollars".
> I assume the money transfer can be solved today using Bitcoin, and it wouldn't be a terribly complicated protocol.
You do realise that Bitcoin itself is a complicated protocol built on top of hashcash, right? Keypairs, blockchains, mining, etc. is just adding unnecessary complexity to this problem; not to mention your choice of the US dollar as the denomination, which requires an exchange rate, etc.
Hashcash is pure waste, this at least benefits the one whose attention is wasted by the spam.
> You do realise that Bitcoin itself is a complicated protocol built on top of hashcash, right?
Of course, but it already exists.
> If emails require a $1 attachment to get into your inbox, you will receive no emails, since nobody is currently sending any emails with $1 attached.
Yes, but assume there are people sending bulk email legitimately who have delivery issues. All you need is one major provider to accept it and then there is a benefit for senders. DKIM and SPF prove that the deployment problem can be overcome, even for relatively weak attacks on the spam problem.
Probably don't do that, I'm not sure I can handle that much spam. Mailgun would begin charging for that much email, for one (I'm on their free tier now).
Is this common? Do a lot of you get spammed with Indian companies / individuals claiming to do web development? I have received a few, and the funny thing is, NONE of the ones I received even had a Web Site?! How does that work?!!
This seems like a noble endeavor which will be useful (albeit under different names over time, I assume) right up until the spammers start using bots to hold their ends of conversations, to make their own operations more efficient.
Eh, machine learning is advancing enough that the sort of people that would fall for Nigerian prince emails could be conceivably convinced they're talking to a real person.
