

Google Groups is Dead  - RedWolves
http://ejohn.org/blog/google-groups-is-dead/

======
Maciek416
This is a problem I see plaguing several code-related groups that I'm
following, and it is very annoying (and up till now, mystifying) to see not
only showing up in regular message flow, but also in abridged once-a-day
summaries.

It's nice to see Google maintaining stuff like Groups and Code, but they don't
seem to get very good maintenance. I've been finding myself saying the phrase
"I wish they'd move this project away from google code and onto github" quite
frequently lately. I suspect I'll be saying something similar for groups very
soon..

~~~
antonovka
_I've been finding myself saying the phrase "I wish they'd move this project
away from google code and onto github" quite frequently lately._

Funny, I usually find myself saying "I wish they'd move this project away from
github, provide a proper API-stable versioned release, properly document the
project and APIs, and otherwise behave like a well-maintained open source
project" a lot lately. =)

If they did all that, then I wouldn't need to fork the project, maintain any
local patches/modifications, etc.

~~~
Maciek416
Good points.

You're right that many github-hosted projects suffer from overly-brief
documentation. Perhaps Github attracts a certain type of project maintainer at
the moment..

------
aravindhanv
John,

Would you be interested in taking a look at Zoho Discussions
(<http://discussions.zoho.com/home>) for your need? It is a powerful solution
for setting up a user / developer community and is an offering from the Zoho
suite of products. Zoho Discussions provides a lot of features
(<http://discussions.zoho.com/features>) comparable (and in many respects
better than phpBB).

Feel free to contact me at dhan [at] zohocorp [dot] com or support [at]
zohodiscussions [dot] com to discuss this further. Will be more than happy to
support you in any way we can. :)

And since we use jQuery across Zoho, and greatly benefit from the productivity
it offers, I am happy to offer the entire solution for free for your website -
for as long as you choose to run it.

Regards

Dhan

Product Manager, Zoho Discussions

------
dangrover
I used Mailman for the discussion group for beta testers of my app, and people
were always confused about how it works or why they were suddenly getting
emails from other people. But once I switched to Google Groups, people get it
and participate a lot more.

~~~
davidw
Yeah, there is something of a distinction between a communication channel for
something that's already "made it", and a project that still needs a low
barrier to entry. You can impose higher hurdles for people to jump over to
participate without really losing much if your project is already 'famous'.

However, if you're trying to gain traction, having something that's easy to
get started with is important, and Google Groups fits that bill pretty well:
1) people with Google accounts can sign up easily. 2) You can treat it as
either a web-based forum, or a mailing list, which allows people to interact
in the way that's best for them.

Indeed, I just switched Hecl over to using Google Groups, from a SourceForge
mailing list, and so far I'm reasonably happy, despite a few spam problems.

~~~
niels_olson
> there is something of a distinction between a communication channel for
> something that's already "made it"

> _MADE IT_

This was true for us even on a small student-organized webapp project. Even
some of the guys on the team protested using MailMan and went on safari
looking for alternatives while we were prototyping, then when the project was
officially adopted by the college and all students were told "yeah, the old
channels exist, but this is a new, officially sanctioned channel also",
suddenly there was a lot more activity on MailMan.

------
jeresig
Sorry everyone, my server has completely fallen over. Feel free to delete this
post and I can re-post when it's back.

~~~
mdemare
Please do repost, it's a good read. What are you going to switch to?

------
shimon
What's a good alternative? The spam problems in Google Groups totally suck,
especially for very active groups, but it's a familiar system, so even non-
techies can sign up and post fairly easily.

Is Yahoo groups even worse?

~~~
mdemare
I wonder if Google is neglecting Groups because they think Wave will replace
it?

~~~
cake
I don't think Wave will replace it, it's different purposes.

They have switched their own support forums from Google Groups to a much more
usable classical forum : <http://www.google.com/support/forum>

Google Groups was such a mess, I never understood what was really going on.

------
mdemare
Google has a cache:

[http://74.125.77.132/search?hl=en&safe=off&rls=en-
us...](http://74.125.77.132/search?hl=en&safe=off&rls=en-
us&q=cache%3Aejohn.org%2Fblog%2Fgoogle-groups-is-dead%2F&aq=f&oq=&aqi=)

~~~
scorpion032
Google cache is not dead.

------
richcollins
I haven't had much trouble with it. I just moderate the first post by a user
and let the rest through. This has prevented any spam from getting through to
my 1200+ member group.

~~~
jamesbritt
Same here. Perhaps I'm not moderating large enough groups for this to be an
issue, but it's never been a hassle.

What I really like is that I get mail telling me there's a moderated post, and
I can approve simply by replaying to the meta-message.

Now, it is annoying that every so often users with approved posts are not
correctly then set as auto-approve, so there's some occasional intervention
needed, but Google groups has worked quite well for the 4 or so groups I look
after.

------
gord
Google could solve the spam problem in groups, they've solved it in gmail.

That they havent, is most likely a symptom of their size and having so many
balls in the air.

------
est
Google Groups has severe privacy leak. If you post through the Google Groups
Web UI, you can view the poster's IP & user-agent via individual message
headers.

~~~
bartl
So?...

The idea is likely to more easily identify spammers.

------
edgeztv
An easy way for Google to fix the problem:

Lemma: most users are subscribed to Google Groups with their Gmail addresses.

Algorithm: GG should broadcast a new message to the Gmail accounts first and
wait. If Gmail flags it as spam for p% of the receiving accounts, it should
notify GG. GG should then send the message to spam (e.g. hold it for
moderation), and not broadcast it to the rest of the subscribers. If Gmail
gives it a green light, let it through.

~~~
bartl
Google Groups is basically just a window to newsgroups (AKA usenet). You can
post to usenet directly, bypassing Google' posting police.

------
brianobush
The moderated groups that I read/contribute to are doing well. Spam isn't an
issue there.

~~~
jeresig
You should chat with the moderators of those groups - I'm positive that
they're having a similar experience.

For people that have already been whitelisted in (and assuming that you don't
get spoofed) the experience is great. For those running the group? Not so
much.

~~~
boucher
Our list is obviously smaller, but with somewhere around 1000 members, I don't
find the moderation duties difficult to keep up with. And we've only let spam
through a couple of times.

------
anigbrowl
Rather than central moderation, which is appropriate in some cases but would
lead to silly turf fights on others, GG ought to implement a simple voting
system, not unlike YouTube comments (where particularly stupid things
disappear after 6 downvotes).

Google really ought to consider the idea of putting NNTP back; it's obvious
they don't really want to develop Groups as a tool (we can still hope for Wave
to save the day, but...) and so GG looks more and more like the zombified
corpse of Usenet. Perhaps if there was a push for Groups to have it's own
Google Code API we might see some innovation.

~~~
mcav
Groups _does_ have a 1-5 star rating system; I've noticed spam often gets one-
star votes in groups I frequent. But that doesn't really address the problem;
they ought to have a "report spam" featuer as well.

~~~
davidw
So the spammers just start reporting a bunch of good emails as spam in order
to confuse the moderation system.

~~~
jacoblyles
This actually happens in webmail quite a bit as well. Spammers open up a bunch
of accounts, spam themselves, then mark their messages as "not spam". But the
major vendors have figured this out.

------
GeneralMaximus
Freelists is a nice service for FOSS projects. You might want to check it out.

Link: <http://freelists.org>

------
ErrantX
I cant help but agree - I have to admit whenever I click through the support
tabs on a projects site and see it go to a Google group my heart sinks.

------
percept
I wish Google Groups worked out better for moderators, because as a user it's
the only message system I enjoy using (versus my 300th phpBB or vBulletin
registration/confirmation/login cycle).

I know some groups have promoted frequent posters to moderator status and this
helps with the workload (though it sounds like the jQuery group already does
this and it's still too much).

~~~
forensic
Even vBulletin requires manual spam control these days.

Spammers are more than happy to pay sweatshop wages to people who manually
register and log in to site after site and post their spam until the account
is hand-banned by a moderator.

------
sammyo
A moderated usenet group with the moderation being a 'reletivly' simple filter
works well. For misc.writing.screenplays.moderated the moderation is basically
just no cross-posting. It's public, a few spams slip through but are easily
ignored (other than when a writer has a lovely snark reply). Getting a
moderated group is non trivial but no too difficult.

------
Coax
Google Groups needs to switch to a Stack Exchange paradigm. I'm hard pressed
to see what the disadvantage would be.

~~~
JimmyL
Stack Exchange is designed for one type of interaction: answering focused
questions.

It's not appropriate for just putting an idea out there and discussing it, or
for that matter even having a discussion about answers to an individual
question.

------
mpk
So is this link.

~~~
phsr
I guess I got in right before it died:

As far as I'm concerned, Google Groups is dead.

For the jQuery project we've run all of our community discussions through
Google Group mailing lists for the past three years. At this moment the main
jQuery group is the second most popular programming group (next to Android
developers) clocking in at over 21,000 members. We also have the jQuery Dev
and jQuery UI groups. The main jQuery group averages around 83-143 messages
per day. I also use Google Groups for discussion on a number of my other
projects (Processing.js, Env.js, Sizzle.js, and TestSwarm).

This post isn't so much about the usefulness of mailing lists as a discussion
medium, it's the complete failure of Google Groups as an adequate purveyor of
public discussion software. For the jQuery project we're already in the
process of moving the full discussion area to a forum that we control. We
should have it set up, and everything moved over, within the next month or
two.

There is one area in which Google Groups continues to shine: Private, or
restricted, mailing list discussions. However any attempts at using it for a
public discussion medium are completely futile.

The primary problem with Google Groups boils down to a systemic failure to
contain and manage spam. Only a bottom-up overhaul of the Google Groups system
would be able to fix the problems that every Google Group faces.

To better illustrate the problem, let's step through the common experience of
running a Google Group.

The Beginning

When you create a public group everything will go well for a couple days, at
most. Without fail an onslaught of spam will start to come through your group
- I've even seen it happen within the first day. It happens to every group and
doesn't matter how well you advertise it (or try to hide it). After having
watched Google Groups for as long as I have I can only assume that there
exists no spam filtering whatsoever. Or, if there is any, it's the most
grossly incompetent spam filter I've ever seen.

When these spam messages start to come to your group a couple things will
happen. First, you may not even notice the spams coming through. Since you're
likely reading the list in a competent email client (such as Gmail) it'll
detect the messages and dump them into your personal spam folder. Don't be
surprised if you visit your group and see a pile of spammy messages sitting
there greeting your new visitors.

Most email client spam detection software is smart. It looks for common points
of failure and tries to take care of the root problem. One such tactic is to
realize that a lot of spam is coming from a single address (like a Google
Group) and start to flag most of it as it comes through (regardless of the
actual content). The result is that much of your list is being flagged as a
false positive. In the case of Gmail people will then start to un-flag the
falsely-binned group messages. This works well until the system starts to
think that all group messages are ok - and here comes the spam again.

To fight the spam you'll likely start flagging emails as "spam" in the groups
interface. This works well (the user is permanently banned and the message
deleted) - until a couple hours pass, that is. You'll see the spammer return,
with a slightly different username, posting the same exact spam messages.
Flagging a user/message as spam does absolutely nothing to train the groups
spam detection system (for reasons that aren't entirely clear and only be
explained by incompetence).

It's a horrible game of cat and mouse with the spam destroying the quality of
your group. It's at this point that you say "enough is enough" and you turn on
moderation for your group.

Moderation

Google Group moderation seems like a palatable idea but in practice is
aggravating and crippling. To start, it creates a horrible first-participation
experience for your users. For example, let's say you go to bed at the same
time as someone in Tokyo attempts to post a message to the group; you won't be
able to moderate the message through for many hours (and that's assuming that
you moderate messages during your work day). While the experience is much
worse than instant posting it is par for the course for most moderation
systems.

Of course, this would assume that Google Groups actually informs the users
that their message has been held in moderation. Looking through the moderation
queue you can see users attempting to submit their message over-and-over
again, wondering why it isn't working. Eventually they'll just give up in
frustration.

In order to combat this you'll typically need to bring on a bunch of people to
help with the moderation duties. In the case of the main jQuery and jQuery UI
groups we divvy up the moderation based upon the time of day and week (and
where the moderator lives). This is incredibly frustrating but still
manageable.

This moderation load also assumes that you are able to successfully navigate
the abysmal Google Groups moderation user interface. It's a horrible quagmire
of radio buttons and un-evenly spaced rows with no visual delineation. I've
provided an example of the interface below (BEWARE: Contains Not Safe For Work
text).

When you begin moderating all the radio buttons start on the "Ignore" column,
it's your duty to move all the messages to the right columns. The "Spam" and
"Always Allow" columns were added just recently (thank goodness) - the
moderation process use to be much worse.

With a user interface this bad mistakes happen. Sometimes spam accidentally
slips through, sometimes users get completely banned. I estimate that this
happens about once in every couple hundred messages. With 84-143 messages
coming to the main jQuery Google Group every day that means that there'll be
at least a few users banned and a few spam coming through every week.

While the occasional spam slipping through is a reality of the web,
accidentally banning users is unacceptable - but it does happen, even when you
don't mean to. For example, here's a message that I got from a user just
today:

    
    
        I seem to have been banned from the jQuery Google Group for a reason
        I'm not aware of.
        > The owner of this group has banned you from this group.
    
        Not sure if this is another Groups glitch or not. I can still access
        the jQuery UI group, though.
        My account is under email: XXXXX@gmail.com
    
        Please advise. Thanks.
    

Nothing quite like insulting, confusing, and scaring your users, due to a
poorly-designed user interface and abysmal spam detection. This is the reality
that Google Group owners have to live with on a daily basis.

All of this changed a couple weeks ago.

End Game

The final straw was placed upon my patience with the Google Groups system a
few weeks ago. Spammers are now spoofing the email addresses of existing group
participants to sneak their messages through. Previously you would've seen a
delightful "FREE MOVIE DOWNLOADS" spam from "freemovies123@gmail.com" - but
now you'll see it coming from existing group users - or even the group
moderators themselves. This cheat completely bypasses the moderation system
since the spammers are pretending to be pre-moderated users.

The Google Groups system is completely fooled. The spam message comes in
claiming to be from an existing group participant - and according to the
Google Groups interface there is no difference. If you click the user's name
you'll be taken to a full listing of that user's posts (with the spam messages
delightfully interspersed).

For example, here's a user whose email address is being spoofed and an email
that was actually sent to Google Groups. Note that the actual email is coming
from a .ch domain and not from the actual Gmail server.

The only "cure" to this problem is to watch for a spam message to come through
and then force that user into a permanent state of moderation. Of course, you
then have to be careful not to bring that user into a "pre-approved" state the
next time you clean up the moderation queue. There is no way to keep track of
which users should be kept in a always-moderate state and which should skip
moderation.

At this very moment my own email address, and the email addresses of most of
the jQuery group moderators, are being spoofed by spammers. This means that we
(the owner and moderators of the group) have to moderate our own messages
before we post, for fear of letting a spoofed spam through. This will likely
happen indefinitely since Google Groups has been notoriously slow to fix
problems with the site.

On top of all of this, Google Groups actually strips out many of the original
spam indicators from the message when it re-broadcasts it to the full list.
This means that when the message finally arrives at a user's email client it
actually looks like it came from the spoofed user. Since I'm currently being
spoofed I've actually had a bunch of my legitimate email end up in spam
folders as a result. Having my email address become flagged as a spammer is
positively infuriating. The fact that Google Groups is silently sitting by and
blindly letting this happen communicates one thing to me: Google Groups is
dead, time to move on as quickly as possible.

Moving On

I've completely given up on Google Groups - and I'm not the only one. Feel
free to ask any Google Group moderator and I'm certain that you'll only get a
sad shake of their head. The situation is completely untenable - which is why
the jQuery team is actively working to get all our lists off of Google Groups
as quickly as possible.

To give you an idea of the overall level of quality that Google Groups
exhibits here is an anecdote: A couple weeks ago the jQuery UI Google Group
was completely deleted for no apparent reason. It was gone for the better part
of a day before it was restored. The only mechanism for contacting support,
even in a situation as serious as that, is to post on a public Google Group.
We were never received any response from an admin regarding the missing group.
Are there backups of group data? Who knows! Forget it, life is too short for
the stress and aggravation that Google Groups provides.

While Google Groups provides a mechanism for exporting a CSV members list it
provides no way to export the full message archive for a group. With over
120,000 messages tied up in the main jQuery Google Group alone it's going to
take some significant work to get everything out and move on. Our only avenue
of escape (short of screen scraping the entire Google Group archive) is doing
an IMAP dump of my personal Gmail account and extracting all the jQuery group
posts from it. I'm sure that experience will be absolutely delightful as well.

Bye Google Groups, you won't be missed.

~~~
xtho
Isn't the problem rather that e-mail (or rather smtp) is dead (or should be).
It seems to me the only solution to this problem is a web-based BBS or replace
e-mail with something else. Or is there a mailling list server that requires
users to authenticate before accepting a post?

~~~
jeresig
Not really a problem with email. It's more a series of critical errors on the
part of the Google Groups system. If they were to implement Gmail-quality spam
filtering then most of my concerns would be moot (since I never would have had
to turn on moderation and these sketchy spoofers would've been caught right at
the gate).

~~~
mblakele
<http://jquery.markmail.org/> seems to have a good chunk of the lists
archived. They can probably get you a copy (marked up in XML) if you ask -
that might be better than scraping.

~~~
kylemathews
And it's handy for showing all the spam that's showed up:
<http://jquery.markmail.org/search/?q=porn>

------
omouse
I just went to see if my ISP still had a usenet server I could use so I could
use a spam-filtering tool at home but it seems that Bell has discontinued it.
Looking at a list of usenet servers, it seems that more than half of US ISPs
and the major Canadian ISPs no longer have usenet servers :/

------
yannis
I would be glad to see the jQuery discussions moved to a normal forum and
moderated accordingly. A good idea would be to use a HN style forum.

Google has failed to contain spam, even on discussion lists for their own
offerings.

It is actually good advise to curate your own data.

------
tdavis
Or, "Why I Use Mailman and Spam Filtering". I just talked to a list moderator
about this issue yesterday, actually. It's highly annoying and some _project
contributors_ have actually posted on lists to say "Sorry, there's too much
spam, I'm unsubscribing".

------
imraj
I suppose google can't win at every project they undertake, the law of
averages is bound to catch up. of course, there are other google projects with
varying success, I use one called GWT whose succes is hard to measure

------
ankeshk
Cant google just add their spam filters for all the group messages too?

~~~
brianobush
not so easy. email is sent to thousands (if not millions) of address. when
someone presses spam in gmail, most likely if the mail is truly spam hundreds
of others marked the same mail as spam. the email could be quickly added to
their training data, the sending agent black listed, etc.

Now groups is a different thing. How many of you mark messages that are spam?
A few I am sure, but the point is the offending person is long gone and the
message is already been posted. Harder problem unless you can wall off the
source and control the users access.

------
giardini
Google should split USENET from Google Groups. The first is useful, the second
is questionable. The first is suffering terribly from it's conjunction (by
Google) with the second.

------
ido
And by extension, so is their usenet archive.

It's really puzzling, since gmail already has a decent spam filter - can't
they apply that to groups?

------
snewe
As the article points out, Google Groups for private lists is awesome. Free
and includes a gig of storage.

------
motters
I moderate a few small google groups, and have been having similar spam
problems.

------
dennisgorelik
I wonder if Google would replace management of Google Groups team...

------
brown9-2
Is the 404 a meta joke?

------
c00p3r
Go back to the school and learn what NNTP and Usenet were. =)

