
Anatomy of Gmane v2 - sohkamyung
http://home.gmane.org/2016/09/14/anatomy-gmane-v2/
======
textmode
The beauty of Gmane v1 from a user persepctive was its simplicity.

[http://gmane.org/lists.php](http://gmane.org/lists.php) <\-- a simple gzipped
list of all the groups and their corresponding gmane category prefixes

ftp -4o lists.gz [http://gmane.org/lists.php](http://gmane.org/lists.php)

zgrep and a little bit of sed was all that was needed.

[http://download.gmane.org/gmane.[listname]/[start]/[end]](http://download.gmane.org/gmane.\[listname\]/\[start\]/\[end\])
<\-- bulk download list messages in the range [start] to [end]; no brackets
needed of course.

There was no JSON or other nonsense. No pages upon pages of "API
documentation". A user could figure it out in a matter of minutes.

This was an easy way to retrieve chunks of a mailing list, from circa 2002
onwards, in a format that was easy to work with, using only basic UNIX
utilities.

It was like dumping a mail spool.

Given the Sirens of gratuitous complexity that never stop singing on today's
web, I fear for the future of this project.

But the point of this comment is to say thanks to the author of _Gmane v1_.
Great project, great execution.

~~~
alphapapa
Yeah, I still don't know why Lars wouldn't let them have the existing front-
end code. I understand that maybe it wasn't pretty under the hood, but who
cares? It worked great for many years and was extremely useful. It could
always be cleaned up and refactored a bit at a time.

Now they're having to reinvent the wheel. And while I shan't presume to tell
them how to do it (especially since it's at their expense), I wonder if the
infrastructure they're cooking up will be as future-proof. It certainly won't
be as simple to reimplement if Gmane changes hands again someday, and it won't
be conducive to setting up other instances of Gmane on a smaller scale.

------
anc84
Any chance of donating a full dump of all the data to
[https://archive.org](https://archive.org) to preserve this treasure trove for
future generations? Imagine if the dejanews newsgroup archives had not ended
up in Google's horrific JS interface but open for anyone.

------
Tepix
borderline unreadable.

[http://contrastrebellion.com/](http://contrastrebellion.com/)

~~~
alphapapa
Unfortunately, quite true. Look what a difference simply disabling the
stylesheet makes:

Before: [http://i.imgur.com/xXLyKgD.png](http://i.imgur.com/xXLyKgD.png)
After: [http://i.imgur.com/DhBXUNA.png](http://i.imgur.com/DhBXUNA.png)

I'll take the second one every time. The first one is like some kind of artsy
poster design, except it's supposed to be an article for reading.

~~~
Sir_Cmpwn
The latter is also a lot more in the spirit of gmane and less in the spirit of
new hipster web design. Unix tradition places utility before form.

------
mrweasel
I wonder why they'd choose to mix Python and PHP and not just pick one
language for EVERYTHING. Seems like you'd risk having to implement some
functionality twice.

Also does anyone know how much data GMANE has?

~~~
jcranmer
Suprisingly, not that large. My archive of NNTP messages has about 10M
messages in only 5.6GB of space, with no compression. SMTP messages accumulate
far more header cruft (in particular, chatty Received headers and often times
DKIM as well in practice), so I'd expect 10M SMTP messages to be about 2-5
times as large as the same number of NNTP messages.

The total size of gmane is only on the order of a few TB at best. I think the
author said that it was only 2TB--for 14 years of history!

------
chaz6
I don't care what they do with the front end so long as the core nntp-email
gateway remains intact.

~~~
alphapapa
I almost feel the same way: I love reading mailing lists and RSS feeds with
Gnus via NNTP.

But the Web interface to mailing lists and groups is also extremely useful.
Nothing like clicking a Gmane link in LWN and being able to instantly browse
the thread like a newsgroup.

