Hacker News new | comments | show | ask | jobs | submit login
Yahoo breaks every mailing list in the world including the IETF's (ietf.org)
256 points by somerandomness 927 days ago | hide | past | web | 114 comments | favorite

We really want Discourse (http://www.discourse.org) to get to a place where we have extremely robust 100% open source mailing list support -- we now have reply via email, notify via email, and community contributed submit topics via email. Working on attachments via email this week.

It's interesting -- unlike forums, people really enjoy mailing lists. I don't think I've ever met anyone, ever, who said they liked forums. But mailing lists seem to inspire people.

I want to see a long term hybrid model where you can interact nearly completely via email, or a good, modern web UI that YOUR org owns (not google groups or yahoo groups). This should be supported.

Forget forums. Forget mailing lists. Bring back NNTP, so we can just point people at com.mycompany.engineering.infrastructure and not worry about setting up another lame inbox filter.

http://forum.dlang.org is an NNTP forum. You can also access it via newsgroups news://www.digitalmars.com/digitalmars/D or even a mailing list at http://lists.puremagic.com/cgi-bin/mailman/listinfo/digitalm...

It works out quite nicely for us.

I love the DLang forums. They are so damn quick!

Wow is their forum software open source/freely available?

I love how snappy they are.

to be honest.. i would like NTTP back as well.

it solves all these problems decently well. i hate mailing lists, because they're not that efficient to use, and that includes the google ui to stuff (and gmail for that matter).

I'm a big fan of NNTP, though I haven't used it in years.

Sadly, I think that culture and clients are likely to sabotage this. Usenet worked pretty well when it was a handful of marquee tech firms and edus using tin or gnus. Similarly, email list work pretty well when everyone's using pine or mutt, understands threading, and uses postfix response format.

The worst mess I ever saw was a list that mixed entertainment industry types with techies, using a wide range of email clients, and featured a lot of people who really liked the sound of their own voices (talked, and wrote, far too much). It was all but impossible to follow discussions, threads were utterly mangled, etc., etc.

So I'm not sure Usenet can continue to work, on a wide scale (not to mention most people don't have a client installed). Though one can hope.

I think a lot of people understand postfix response format. but top-post because they think having the newest content at the top, rather than having to scroll, is better.

Yes- the top/bottom post debate is probably older than netscape.

Actually I think where Discourse is going is the same place Microcosm is going.

The software is just an intermediary that connects people having discussions and collaborating.

The first version may well be a web forum.

The second version may well be an email interface / mailing list.

The third version may well be native mobile apps.

But all versions work all of the time and people should be able to connect around their shared interest regardless of how their interface preference might divide them.

In that regard, you have the usenet dream of people picking whatever client they love, and there's little reason NNTP can't just be one of the available interfaces to the underlying swirl of discussion and collaboration.

It would need some form of voting, distributed moderating support (including UI in clients), html posting/images and advertising to be a real alternative for most forums though. NNTP is much better than web forums but web allowed people to add features ('innovate' shudder) much faster than a 'set standard/get clients to update' cycle can.

I agree that NNTP is nice, it is however lacking some features that forums offer these days. Ideally there would be a push to a new version of the protocol that adds these features (and more such as using hashcash to combat spam).

You don't need anti-spam measures if your newsgroups are internal. :)

I don't think that particular genie can be put back in the bottle.

When I was younger, I loved hanging out on forums. There was perhaps a more robust community feeling, since forums can have multiple sections, and include off topic and chattier discussion without people complaining about being deluged with mail. Avatars and signatures let people express their personality and made it easy to tell the author of a post.

Then again, to some extent I'm just nostalgic. What then was personalization I might now see as a rather low post text (signal) to metadata (noise) ratio...

I spent a lot of time in high school on PHPBB systems. I find it less attractive now, using mostly IRC plus some forums, though I don't frequent them enough to be a "regular". I absolutely hate subreddits - I generally find specific enough ones to be of low quality compared to the equivalent forums.

I also absolutely love Futaba-style imageboards - it's a great blend of anonymity, simplicity, and community. It's a shame 4chan's reputation is what it is.

Most of the problems you'd want to avoid could probably be solved if you get rid of the images, and required people to sign up but still post anonymously.

Granted, then it's no longer strictly an anonymous imageboard but in a lot of cases, the images don't add to the discussion anyway.

Well, why do you need anonymity, then? Might as well display names by default.

I disagree with getting rid of the images, though. They are a useful tool, and most forums feature some sort of attachments. attachments that add nothing to the discussion are a culture problem, not a technical one.

I think it's useful to not necessarily have to link your posts to your account if you don't want to.. anonymity from other users and anonymity from the staff have never been the same thing anyway. A lot of the problems on sites like this (HN, reddit) stem from their focus on chasing after and maintaining reputation, which itself can encourage an echo chamber and a sense of elitism. Having an account would make it easier (not necessarily effective, but easier) for the staff to deal with systemic abuses, because you have something more persistent than an IP to deal with.

Anonymity often encourages trolls to come along and inject horrible comments as it is, allowing posting without creating an account would just lower the barrier to such posts - it's not fun for a moderator to have to deal with that (I've dealt with such for about 12 years now).

People are happy to leave horrible comments whether or not they're anonymous, have a pseudonym, or are using their Facebook account.

What's important is if there's an active community encouraging them to do something better. (This is different from just stopping them from doing bad things.)

You could require creating an account and posting through it, but still not show that account to other users. Although then trolls might just create new accounts. It's not an easy problem to solve.

> Well, why do you need anonymity, then?

One example off the top of my head: Because some of us would like to put it to productive use and discuss technical work-related stuff without a possibility to link it back to our employer?

I'm where you are. IRC and twitter together work well for me.

Forums are still one of the best sources for narrow topic specific discussion. I frequent several (advrider.com being the biggest) and the topic divisions keep off-topic discussion to a minimum.

advrider.com ++. Really is great for adventure motorcycle riding. If you just pay attention to popular TV you'd think you have to be some sort of film star to ride around the world on your motorbike. Then you read some of the Ride Reports there and go "Holy hell! Lots of people are doing it"

I love forums. I am not crazy about forum software or the inconsistencies between different forums, and often wish for a reliable interface I could use to manage my subscriptions, like an NNTP newsreader or used to do - there are a few mobile solutiosn because many forums are so awful on Mobile, but not many desktop based ones.

I always found mailing lists a pain in the rear to be honest, because the content is usually surrounded by all this auto-generated administrative cruft in addition to the administrative interface of the mail reader itself.

I love what you're doing with Discourse but I wonder if there sin't some selection bias going here.

>unlike forums, people really enjoy mailing lists. I don't think I've ever met anyone, ever, who said they liked forums

I bet if you asked people if there were forums they liked, you'd get a different set of responses.

Forums do seem to be generally less popular than they once were, though, probably because a lot of the functionality they used to provide has been folded into social media and blogs with social logins.. just plain forums on their own might seem a bit atavistic to someone who grew up in the age of facebook, youtube and whatever the new thing is i'm too old to even know about.

I think they've been largely replaced by reddit. Autonomous subreddits aggregated into a single feed instead of separate global subforums, upvotes instead of bumps (that also serve to self moderate), threaded conversation instead of quote trees, reddit is basically just forums adapted for scale.

And I would argue that that is why Reddit sucks so much (personal opinion - I can understand why someone might enjoy the community there, but I do not in the slightest).

Don't get me wrong, I enjoy the news aggregation feature, but the community aspects are terrible. The benefit of forums was primarily in their segregation of a small-but-active community. Reddit works in precisely the opposite manner, so you end up with boring, repetitive, least-common-denominator tripe. Any subreddit small enough to avoid that is too small to be worth the time - it seems that small communities (by which I mean actual communities, as opposed to random conglomerations of people with vaguely similar tastes) tend to have other sites and methods of communication they frequent, such as this one.

I think that the poor scaling of forums is their greatest strength, and Reddit is an example of what happens when forums do scale.

I like forums, there.

Also, FWIW, I hate mailing lists (although recognise they often hold high value information).

I love forums and hate mailing lists. There just isn't enough functionality in mailing lists, and I end up feeling spammed.

Also, Reddit and HN are basically forums.

Reddit is a forum, and I love it.

Quora is a forum, and I hate it.

I don't think I've ever met anyone, ever, who said they liked forums.

Hacker News is a forum.

And a lot of people's biggest complaint is that HN does not notify you of replies to your comments. I use HNNotify to inform me of replies but it is not perfect. The big problem with HNNotify is that it also emails you top level comments for every story you submit. A less frequent complaint is that you can not collapse threads. Neither are an issue with mailing lists.

I would guess this is deliberate to discourage flame wars.

did you get some age statistics on that? I really hate e-mail, mailing lists etc. I was thinking that it might be a conflict between the generations because I am a bit younger than the people I used to see on the mailing lists.

Do you use gmail?

The client (or provider) has bugger all to do with it. I'm old and cranky, and have hated mailing lists since I was a young 'un. Properly-threaded, searchable forums (preferably locally-replicable) are better in every imaginable way. Plato was able to do it sometime back in the early Devonian; Iris/Lotus (whether you like the Notes client or the default app UI or not) made it easy and distributable in the late Cretaceous. The parts exist to make it workable with FOSS and an open, less-astonishing UI, both connected and local. Neither familiarity not ubiquity is an indicator of quality or suitability, and email threads are just about the worst format that can possibly be forced into service in this sphere. That email was once, for all practical purposes, the only available choice doesn't make it the best choice.

I'm on several mailing lists, and the client I'm using shows the threaded discussion and search works fine. I get to use a real editor and not what Firefix/Safari/Chrome/IE provide in too little text areas (like now).

yes. But it's more like I'm not interested in stuff forever. if for a few months I use a lib, and wish to send a patch or something, then life move on, I change job, I replace the lib, etc. I have to subscribe, get spammed, find a way to unsubscribe filter the thing etc. Whereas on the web, I decide when I go reading and if it's not relevant to me, I don't go anymore and that's it. Nobody cares if I put my answer over or under the citation, and anyways, if anything has already been said before, they will tell me to read the ML history which lies... on the web, where it belongs in the first place.

It would be really nice if you could make Discourse more nice and usable without Javascript: http://i.imgur.com/Jml22Hg.png

I see no reason to use JS for a simple forum like webpage just like I would see no reason to use HTML in mail.

Given that Discourse is an Ember.js app at heart, I wouldn't bet on that anytime soon.

Another (late) voice that prefers forums over mailing lists.

One argument about what that I haven't seen so far in this discussion is that I can go to check the forum whenever I want (and not go for a couple of days if I decide to) whereas a mailing list is more like a push model where you receive the emails no matter what, even if you can't take care of them for a couple of days.

I have to say that I have very little experience with newsgroup though, so it might be a best of both world, but let's face it, it has becomming almost inexistant.

Don't know if I really am going to be the first guy, but I (vastly) prefer forums to mailing lists. I also don't like IRC.

I think I'm a weirdo, though. Or a web-head.

I have a question to all those of you who wrote/will write/think that they prefer forums to mailing lists - I've just picked you randomly to reply to as I don't want to spam the thread with the same question:

Which mail client do you use, and have you ever used a mail client that supports an easily accessible threaded view, like mutt? Or have you ever used a Usenet news reader, like slrn?

I really don't like web forums and very much prefer mailing lists and NNTP, because it gives me a unified interface for all the different lists and newsgroups with very low UI latencies and powerful functionality for dealing efficiently with huge discussions.

That's why my hypothesis is that many people who prefer forums do so because they have never seen/used a good/powerful mail or news reader, and only ever have used mailing lists through some webmail interface - which indeed probably is a lot worse than web forums, especially if it's high volume. So, I am asking you to provide some data to check this hypothesis against ;-)

Like one of the posters above mentioned, forums have multiple subsections. As an example let's say the forum is about animation, I can read Japanese animation section and ignore the Spanish animation section. With a mailing list it's all or nothing.

4chan is a forum, neogaf is a forum, gamedev.net is a forum. I think they all work much better as a forum than a mailing list.

I think mailing lists serve a different need/style/purpose than forums.

What dfc said, but other than that I think that is more an argument for usenet than for web forums? At least the "subscribe" step tends to be better integrated with newsreaders than with MUAs, so it's easier to subscribe to a whole bunch of groups. But on the other hand, subscription tends to be only a small part of the overall experience, so I am not so sure it really matters all that much.

How is that different than being subscribed to debian-spanish@l.d.o and not subscribing to debian-japanese@l.d.o? Or subscribing to debian-devel-spanish@l.d.o and and not subscribing to debian-devel-french or debian-devel-portugese?

For one it's easier to browse without committing. If there happens to be no new topics in Japanese Animation I might read Spanish Animation and if I want, reply to a topic. With N mailing lists I have to commit to the entire list rather than just individual topics in subforums. In other words I have to subscribe to the "Spanish Animation" list rather that just one topic inside.

Yes, I have tried such clients. I'd say NNTP > forums > mailing lists, because I don't want to spend time writing inbox filters or the like. Email is for a very different purpose than forums, and I like having separate accounts for both. (NNTP, meanwhile, is dead - I'd have to pay for access, and the communities just aren't there any more).

Yeah, the filtering setup indeed is more complicated than necessary, I guess a more streamlined process for that would be nice (the whole aubscription process, that is, like some URI scheme that would invoke a "subscribe to mailing list" function of your MUA, which would initiate the subscription, set up the filter, configure the list address as a subscribed mailinig list (for Mail-Followup-To handling)?).

On the other hand, once you have one filter setup, it usually isn't that difficult to copy and modify it for a new list!? And signup is a one-time thing, after all. But two-click subscribe certainly would be nice and shouldn't really be all that difficult to do.

(Edit - sorry, forgot the second part of my answer, so here it is:)

Usenet is not dead, though quite a bit less alive than it once was ;-) - but no, you don't have to pay, there are a few free news servers, if you can live without the binary groups, for example:

http://www.albasani.net/index.html.en http://www.eternal-september.org/

(Those are read/write, you can find many more public read-only servers)

I believe that eternal-september.org provides free access to newsgroups. As for the communities, it really depends on the newsgroup. Some of them are still relatively active.

I use Thunderbird as both my mail and news client. It has a threaded view and adequate filter/killfile capability (though not as sophisticated as some other newsreaders that use a score based system).

> unlike forums, people really enjoy mailing lists. I don't think I've ever met anyone, ever, who said they liked forums.

I think it entirely depends on what the purpose of communication channel is serving.

Mailing lists are transient passive participation. I can sign up to a list and never have to do another thing because I use email all the time. Occasionally a back and forth discussion might pop up, but I can easily choose to ignore it by simply glancing at the subject line.

Forums are persistent active participation. I have to specifically access the forum, possibly logging in in the process, to see what activity has happened. Many do enable some kind of email notification with a set frequency. Digest emails lose the benefit of the quick glance decision to attend or not, while all activity would be similar to the mailing list model. As forums can encourage more silo-ed conversations or short disposable responses, getting all activity is generally not ideal, however.

I love forums. Their layout and functionality make a lot of sense to me.

I like mailing lists quite a bit, but consider them a barely adequate substitute for an NNTP group. The biggest advantage is that brain-dead email clients can't break the threading, a close runner-up is that you don't have per group policies WRT reply-to munging, all groups work the same in your client of choice.

I really like discourse's vision , but I wish that the plugin ecosystem will get some more activity. I recently tried to create a simple plugin but stopped it due to my limited knowledge of ember.js .

Doing plugins right is hard.

My philosophy is also that 90% of what you would want to do with plugins should be built in, anyways.

I'm the opposite - I love forums, hate mailing lists. Mailing lists clutter my inbox, while I can browse forums to my heart's content without it intruding on my personal mail.

The joelonsoftware forum seemed to be well liked. It had the feature of unthreaded comments which made it harder to sustain pedantic bickering and off-topic tangents.

Off topic, but do you keep a list of sites using Discourse? I really like the software but have yet to run into it anywhere besides your demo site.

Some already using it in the wild include:

- bbs.boingboing.net - discuss.howtogeek.com - discourse.ubuntu.com - forum.greenheartgames.com - discuss.atom.io

sort of, http://www.discourse.org/buy has the official 3 early beta partners. Those are solid and mature since they've almost been around a year now!

In the last 3 months we went from 500 Discourse forums to 1500+. (We screen out the obvious test installs with IPs for names, etc.) We're about to take the brakes off as we get to V1.0, I have been reticient to really encourage people to launch new Discourse sites while we are in beta. But it's speeding up as we get closer. Open source FTW!

We don't have a public directory of all Discourse forums -- we do internally, since everyone pings us for versioning, but I have been hesitant about exposing it until we're clear the community is OK with a public directory.

I love my online friends, and I hang out with them in a forum. A mailing list serves a completely different purpose to my mind.

Having managed a number of largish email systems over the past few years, I've got to say that dealing with Yahoo -- DKIM and other anti-spam related issues -- is one of my larger ongoing headaches. The lack of response and transparency of Yahoo in general is a huge problem -- I managed to get resolution in one case via an executive email carpet bomb (this after repeated contacts with their support team and direct emails to the CTO/Postmaster's address).

Sadly, as much of a fading giant as Yahoo are, their email presence remains huge.

Postmaster address? According to http://rfcignorant.org/lookup/yahoo.com (and from personal experience), Yahoo do not have a postmaster address. If you want to report a problem, you need to find a deeply-buried form somewhere on their web site. Oh, and if you don't have a Yahoo account, you're out of luck, because you need to log in to use that form.

I'd emailed Raymie Stata directly. Never, ever, got a response. Contrast with Microsoft: I dialed the switchboard, tequsted the EVP in charge of Hotmail, he picked up on the first ring, and I was talking to the relevant manager 15 minutes later who worked with me for a few months resolving the issue.

I'm not a Microsoft fan, but their people were very professional in this case.

Raymie Stata hasn't worked for Yahoo in a long time. You can go here, https://help.yahoo.com/kb/postmaster when you have issues. Was very easy to find. postmaster.yahoo.com

He was CTO there at the time I was contacting him. This has gone on for more than a few years.

Yahoo may be bad, but Outlook/Hotmail are without a doubt worse. Exhibit A:


I disagree. I've had Yahoo reject my emails completely silently (didn't arrive in inbox OR 'spam' folder, even though Yahoo's MX said the message was accepted). Yahoo was entirely unresponsive about this issue. We didn't have the same problem with any other provider and it was resolved immediately upon switching to a third-party email delivery service.

Additionally, Yahoo has a huge amount of abuse and doesn't seem to have an abuse handle either; you have to fill out some form buried deep on their site. On the other hand, I've reported abuse incidents to Hotmail before and have gotten an actual reply from a human (a rarity when submitting abuse reports; most places act on them but don't bother responding).

Yeah, don't even get me started on Yahoo's spam emissions and lack of reporting.

In terms of being responsive to issues, I've actually had far better response from Microsoft than Yahoo, as I described above. Call the switchboard, ask for the SVP in charge of the division, get transferred directly to him, he picks up the phone on the first ring and talks for ~10 minutes to hear my issue (spam transiting Microsoft's network). Tells me I'll hear from the manager in charge, who calls me 15 minutes later and works over the course of several months (these things do take time) to resolve the issue and improve their systems.

As I wrote above: I'd written Raymie Stata repeatedly after getting a complete runaround from Yahoo's tech support (and diving into dead ends on their website), never hearing a peep from him. Not until I emailed pretty much the entire C-level suite and senior managers with a bit of data showing the nature of the problem (postfix delivery time stats) did I get a return response, from Yahoo's "concierge" service. That finally resolved the particular issue I was dealing with, but that's one appropriate response in years of dealing with the company.

The resolution with Yahoo was essentially the one described in your Serverfault link: get explicitly whitelisted. That's not uncommon with top-tier email service providers.

I played with DMARC about a year ago. I put it in monitoring mode so that I'd get email reports from systems to tell me when they received emails from my domain which failed DMARC. I started getting them from all over the place. Pretty much all related to mailing lists breaking the DKIM signature by rewriting headers or the body. My conclusion was: If any email address on your domain subscribes to one or more mailing lists, DMARC will break your email. I disabled it. I don't see myself enabling it again any time soon.

We noticed noticed last month that one of our Yahoo! Groups mailing lists would randomly drop emails[1]. We couldn't find any consistent behavior to it. Wonder if this is the culprit.

[1] https://twitter.com/jmathai/status/440529845198790656

So what should Yahoo do? Change settings to say "Actually, anyone in the world can send emails from @yahoo.com now."?

(Honest question.)

Yes. If they want their users to be able to use mailing lists.

Is that the trade-off? Either we neuter SPF et al or we break mailing lists?

I vote for SPF et al.

Yes, that is the tradeoff.

SPF doesn't really come into it. Mailing lists use their own sender envelope. The problem is, when a mailing list makes changes to an email which breaks the DKIM signature. But the sender uses DMARC to say that DKIM must pass.

Another fix would be for all mailing lists to be updated to not make any changes to messages which might break DKIM. E.g by adding [listname] to the subject line, or messing with other headers, or adding signatures to the body.

The tradeoff isn't that bad: it's possible to make mailing lists work with DMARC, see for example their suggestions in the FAQ: http://dmarc.org/faq.html#s_3.

It can be argued that the required changes are very burdensome and not mailing-list-friendly. The mail body modifications seem to me like something mailing lists could drop taking advantage of the list-* headers instead. The harder usability issue arises from the fact that DMARC imposes a different way of setting the from header potentially breaking all those filters we've set up.

DMARC claim both issues can be solved using "Original Authentication Results" header but since it requires the receiving MTA to trust the mailing list the administrative overhead here just doesn't scale and will likely end up being pushed onto the list admins.

Also, SPF does come into it since DMARC requires "alignment" between the from domains in the envelope and the header (see again the FAQ answer above).

My point about SPF not coming into it, was made because SPF will fail in a forwarding/list setup. DKIM can work in this setup, so that's what we want to make work, but which all mailing list software seems to break.

SPF is not the problem here, SPF determines what mail server can send yahoo mail. The issue here is DMARC which determines (amongst other things) what to do when someone puts yahoo.com in the "From:" field. Yahoo has set it up so that mail gets rejected in that case, even if you add the proper "Resent-From" header to be something non-yahoo.

DMARC.org has very clear remedies.

Q: I operate a mailing list and I want to interoperate with DMARC, what should I do?

A: DMARC introduces the concept of aligned identifiers. It means the domain in the from header must match the d= in the DKIM signature and the domain in the mail from envelope. You have a few solutions:

- operate as a strict forwarder, where the message is not changed and the validity of the DKIM signature is preserved - introduce an "Original Authentication Results" header to indicate you have performed the authentication and you are validating it - take ownership of the email, by removing the DKIM signature and putting your own as well as changing the from header in the email to contain an email address within your mailing list domain.

Spoofing is a huge issue for all email customers. DMARC was started, in part, to deal with the coming problems that were foreseen here. Mailing Lists don't have to forge or spoof to work. They can adjust and everyone is better off.

(the post above has been largely copy-pasted from http://dmarc.org/faq.html#s_3)

Interesting point for the discussion on whether MLMs are allowed to modify the from header is in the section 3.6.2 of rfc 2822: http://tools.ietf.org/html/rfc2822#section-3.6.2. The intended meaning of the from field is to indicate the author of a message which is explicitly allowed to be different than the sender. Thus list-originated communication like digest messages should be sent with the from header of the list, but messages forwarded by the MLM should be sent with the from header indicating the original author. In the absence of the sender header it can be assumed to be the same as the from header. Thus, DMARC could use the sender header instead of the from header and fall back to the from header only when sender is absent. This way MLMs would have a way of avoiding the issue by supplying the sender header. Unfortunately, DMARC chose not to use the sender header citing abuse and bugs in some MUAs which don't display the sender header to the user correctly: http://www.ietf.org/mail-archive/web/dmarc/current/msg00064.....

As for the "Original Authentication Results" it doesn't solve the problem for most lists since it requires the destination domain to explicitly trust the list, see http://www.dmarc.org/pipermail/dmarc-discuss/2012-February/0... and http://tools.ietf.org/id/draft-kucherawy-original-authres-00.... Few list admins could afford getting a trust explicitly established with every domain where the members happen to have mailboxes.

> Mailing Lists don't have to forge or spoof to work.

Using email correctly per RFCs isn't "forging" or "spoofing".

That this doesn't work with DMARC because DMARC chose instead to break the world because it preferred to support the existing broken behavior over (rather than only as far as was consistent with also supporting) standardized, documented semantics of email headers is if not a fatal flaw in DMARC, at least something that greatly limits its utility.

For this to fail, wouldn't the mailing list have to send the message on to its subscribers listing it as "From" the Yahoo email address? In that case, it's the mailing list doing it 'wrong' (in the eyes of SPF, DKIM, et al, anyway), as they should be sending it 'from' their mailing list email address, not the original person who sent the message. This isn't a new problem as SPF has required mailing lists to do this for years now.

*UPDATE: Clarified 'wrong' wrt the various protocols.

Breaking the entire world and then complaining that "they are doing it wrong" goes over like a lead balloon.

Its a messy state of affairs to have to workaround issues that dont match your particular models, but its often necessary to have things work at all.

Do note that mailing lists and the way they send mail predate DMARC, DKIM and SPF by far, why don't they better account for this extremely prevalent model of email usage. Why is it that mailing lists that are "broken".

Mailing lists are a tiny, tiny fraction of email and used mostly by us techie types. So, I thinking 'breaking the entire world' is a bit hyperbolic.

I'd meant 'broken' with respect to DKIM, SPF et al (and have updated the post to reflect that).

I am going to take a guess and say that you are not/have not been involved in any somewhat serious activism. Activists love mailing lists.

I'd wager the vast majority of folks on the internet don't even know what a participatory mailing list is. And I say that as someone who has been a part of multiple lists for nearly 20 years.

I'm on ten or twelve different mailing lists. Only two have anything whatsoever to do with tech. The rest are - as per dfc above - politics-related.

So I don't view lists as somehow a techie habit - quite the opposite, in fact. They work for politics because the technical 'barrier to entry' is very low. You don't need to know anything more about computers than how to send and receive email in a client of your choice. Given that in most political groups you're dealing with students, pensioners and everyone in between, this is a distinct evolutionary advantage.

I have no idea how the total volume of list-mail breaks down between tech stuff and muggle concerns. But in any situation where you have to communicate between people of different generations with wildly different technical expertise, lists are an ideal lowest-common-denominator.

Until, that is, Yahoo or whoever cut their users off, and muggins here has to explain to uncomprehending people why their messages aren't appearing.

Guess it's going to be a long week for me. sigh

As an anecdote, both of my parents have dealt with (unrelated) private "listservs" associated with their work.

With just my eldest, 10-yo daughter, I have subscribed to 9 mailing lists at 3 different schools in two countries, to discuss things with various groups of parents, with none of these lists being started by tech folk.

The mailing lists I am subscribed to -- openbsd-misc, roundcube-dev, and backuppc-users -- all do this. So does the mailing list I admin, for a local hackers group.

As well as every other mailing list in recent memory. (Including the LKML. I just checked.)

Whether it's "right" or "wrong" might be an argument that someone will find worth having, but I don't think it can be argued that this isn't common practice.

We shouldn't have to keep this giant security hole in email (spoofing) just because mailing lists, which predate the spam and spoofing issues, don't want to update how they operate. Accommodating =reject DMARC policy is simple for the mailing lists. A policy of "Because that's the way we always did it" is dumb. We didn't use to recycle, now we do...because it is better. DMARC is better than allowing spoofing. Update your mailing list. Stop forging sender. Move on.

They often do that (resend with the receiver being the person). Mailing lists have intense debates over the correctness of replying to the sender by default or to the list by default (and whether or not you want to add cc or not). Many also munge the subject line with things like [The AWESOMEList] because forever and ever people's mail clients could only filter on subject lines or body text and rarely additional headers.

I appreciate where Yahoo's heart is, but this wasn't really well thought out on their part.

> For this to fail, wouldn't the mailing list have to send the message on to its subscribers listing it as "From" the Yahoo email address?

Yes, which many mailing lists do.

> In that case, it's the mailing list doing it wrong

Is it? Not having the actual originator as the "From:" seems to be "doing it wrong".

Exactly, and "From:" has "always" been expected to possibly be a different identity to the sender of the e-mail: We have "Sender:" to indicate where the message was sent from when "From:" does not.

That's bizarre. Correct behavior used to be to send "from" the original sender, with the list address in the "reply-to" field. That way everyone knows who posted it, but discussion is still directed to the list.

No, it's not. A maling list should not ever touch the Reply-To: header field. See also:


Note the (also very old) opposing argument: http://marc.merlins.org/netrants/reply-to-useful.html

Personally, I've almost never wanted to privately reply to a mailing list posts (and I'm under the impression that doing so in most contexts is vaguely rude), but redundant CCs in reply-all chains are ugly, so I prefer munging Reply-To.

Except the opposing argument is severely broken, of course. It starts with not even comprehending the RFC he is quoting ("include the address of that service in the "Reply-To" field of all messages submitted to the teleconference", which clearly does not sanction the "teleconference" itself munging the header), and after that essentially turning logic on its head and appealing to ignorance ("People who don't know how what the working solution looks like are asking for a broken solution! If software does something else than it should do by definition, it's the user's mistake, and therefore the user's fault! [...]").

As for redundant Ccs: That is what Mail-Followup-To is there for, see also:


edit: Oh, and also, in particular, he completely ignores the fact that Reply-To munging deletes existing Reply-To headers, and thus breaks things in a way that even the best MUA cannot possibly work around - the original Reply-To isn't there anymore, so it cannot possibly offer you an option to reply to the original Reply-To or to the munged Reply-To.

What? Every mailing list I've ever subscribed to has the original sender in the "From" field and the mailing list address in the "To" field. Otherwise how would you know who sent the message to the list?

I think SPF is happy if the bounce address (envelope from) is replaced but the From: header is intact.

To pass DMARC, you need to have either DKIM pass, or SPF pass when the envelope sender and the From: header are aligned (whatever aligned means... generally within the same domain).

Because the envelope from headers are very likely changed to the mailing list, the SPF pass doesn't help with DMARC. And since the mailing lists very likely alter the messages (to put footers), the messages probably don't pass DKIM either.

I'm amused to note that this probably applies to Yahoo Groups as well, since it uses the yahoogroups.com domain, not a yahoo.com domain.

Looking at the headers of a recent message from a Yahoo user over Yahoo groups it seems like it would be the case:

  dkim=pass header.i=@yahoogroups.com;
  dmarc=pass (p=REJECT dis=NONE) header.from=yahoo.com
Of course, anyone who is using Yahoo Groups regularly has probably noticed that even with last year's redesign it's not getting much attention.

This and SpamCop flagging a bunch of Google's mail server IPs broke a bunch of email last week.

Please forgive my naivety—why are mailing lists forging from addresses in the first place? Have they just been fragilely dependent for years on the exploitation of an authentication vulnerability?

Fair question. The basic internet email spec has virtually no security features whatsoever, and is completely unauthenticated. Mailing list management software doesn't forge sender information, but rather often retains it and generally trusts incoming headers. Back in the old days, some folks even referred to discussion lists as "reflectors."

The proper usage of SMTP mail headers is outlined in RFC2822 (originally RFC822), and the definition of the headers From, Sender, Resent-From, etc. The rules for specifying sender information are spelled out in 3.6.2. [0]

That said, system behavior also depends on if the MLM software is running behind a mail transport agent that enforces authentication protocols for incoming emails, scans for viruses, etc.

When discussion list owners are concerned about receiving forged posts, they usually use list moderation features so they can ensure emails do not get distributed that haven't been reviewed first. But the biggest problem for MLMs isn't usually dealing with impostors, but rather blocking email-borne viruses and misconfigured auto-responders that could cause bogus emails to get reflected out to subscribers.

The behavior of the outgoing From header from MLM software typically depends on the configuration of the list. Some lists (especially digests) are configured so outgoing messages are "From" the list itself. But most discussion lists are configured to retain the original From line, while clarifying their role as an email proxy through other headers.

[0] http://tools.ietf.org/html/rfc2822#section-3.6.2

I don't use Yahoo anymore because it runs very slow for me..

So would this explain a spam email in my hotmail inbox that had the same From&To, with no mention of my actual email in the raw message source?

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | DMCA | Apply to YC | Contact