this is interesting, but in a way it's also really scary from my quick look at it it seems that is sql to store the emails. any reason to use that over mbox, or milder, which you can then easily index?
it also implements a small imap client(which may work fine for just reading gmail, but imap is a scary thing, everyone uses it a little differently)
but even for that, if you don't care about fancy guis, you're probably still better off running mutt, imap, mu on a shell you have somewhere
edit: i didn't pay enough attention, as was mentioned it's using imaplib, which is the same library offlineimap uses. i do believe though that a lot the work that happens is how people deal with the rfcs. i remember reading about frustrations of the offlineimap author with different servers, and clients. but basically imaplib itself is just a basic protocol implementation, everything else happens on top of it.
and while you can indeed easily store your terabytes of data in postgresql and you can easily setup full text search, it's still much more portable and easily to handle backup and syncing purposes in different formats. Archiveopteryx looks interesting though, so thanks for the hint. if i never write a service to handle mails for other people i'll revisit it.
Arnt and Ahbijit know more about the email RFCs than probably anyone else on the planet. Arnt has been involved in writing several RFCs himself. Aox is blazing fast and your mailboxes scale as large as you want. Millions of emails in a single mail folder is not a problem. It's also really nice that it handles deduplication of your mail at several levels.
Thanks, I've never heard about that project, imap was cyrus, courier or dovecot for me (I run the latter).
The dovecot developers claimed in various mailing list threads, that a SQL based backend would not make sense, not even performance wise. Interesting to see a different approach.
What irks me is the FAQ entry claiming "If you have trouble with user authentication, edit the pg_hba.conf to allow trust or md5"... I - don't think that this is a good idea?
Trust is not a good idea, in my opinion. But I know that some people disagree with me, and I have better things to do than discuss the issue with them.
MD5 is fine, I think. If you allow randoms from the entire internet to connect, allowing MD5 won't hurt you much more. If you restrict connections to e.g. your own IP addresses using a packet filter or firewall, MD5 definitely will not hurt you.
> it seems that is sql to store the emails. any reason to use that over mbox, or milder, which you can then easily index?
If you do that, synchronizing the index with the data becomes your responsibility. And you're most likely indexing with a database anyway. So the next logical step is to put the message itself in the database, and then you get synchronization for free.
It might actually be better to use ElasticSearch as the datastore for this particular application instead of MySQL or the like.
If you choose your layout carefully SQL should be fine. Certainly you can index on whatever you like that way. You also get the potential to do clever things like storing message bodies or mime parts once and foreign-keying them into individual mailboxes (if I email 30 people a single 1M file I should only really need to store it once).
This actually looks almost like what I want out of a webmail client except it's lacking hotkeys. I don't know how people manage email without keyboard shortcuts.
It's not so much that it's one file (although that makes for fun times if the underlying filesystem doesn't support proper write locks), after all most databases also use just one file -- it's more the fact that individual messages are only recognized by lines starting with "From: " -- which presents some obvious challenges with escaping (that some programs manage to get wrong). I also imagine it'd be challenging to maintain an effective index as mails are deleted...
Well, there are MTAs, which one may use to distribute mail to different mailboxes. Also, no one is obliged to not truncate their mbox files after a while. I just encrypt my set of files and tuck them in a tarball, and store that in a specific folder with older tarballs.
How deep have you gone with the library and with an imap client implementation? There is definitely a lot of variation from imap server to imap server. Things come back with weird and broken encodings sometimes (the server will tell you that a message is unicode and then return invalid unicode), commands that seem to be read-only will have persistent effects, etc. IMAP is nowhere near as nice to work with as its standard and interface imply.
Gmail IMAP's IDLE support is broken (probably intentionally). It will report new messages, but it won't report changes to the number of unread messages (or which messages).
This means your clients don't automatically update read message stats, which is a pain (most surprising thing I noticed moving to Gmail from hosting IMAP on my home server).
> IMAP is nowhere near as nice to work with as its standard and interface imply.
I prefer to say that it is just as horribly awful as it's abomination of a ill conceived standard implies.
Pretty much "everything" is wrong about IMAP, from allowing the use of message sequence numbers that can change under you, to the horribly overcomplicated support for addressing mime parts, to the overcomplicated grammar...
If it were me I might look at building somthing on top of notmuch. I use the emacs package for notmuch and it's awesome, it has search and tagging capabilities very similar to gmail and handles conversations nicely.
I was enthusiastic about this being able to replace Roundcube/Gmail (what I currently use), but it's rather ugly (no offense to the author.)
If nobody else is going to, I may start work on some better styles for it. Also, I'm not really sure why there are two panels and profile pictures; neither gmail nor Roundcube nor Outlook do this. (Gmail and Outlook do put profile pictures to the side of the email itself when opened, however.)
Ah, the rising tide of open source software. Whatever was done a decade ago will show up in open source. Companies need to invent new things to stay competitive.
Just wanted to mention. The source of this link is http://redd.it/22bvy4. I have posted link only to reddit and then it appears here. There are some my answers on questions which can be simular to this thread.
I welcome this new mailclient. Unfortunately it currently leans differently from what I prefer, but perhaps it is close enough that the source code will form a useful basis for what I prefer (no html (and a decent html>text renderer for the brain dead services that don't provide (useful) text parts), no rich editing, threaded -- not gmail style conversations).
At any rate it seems like a more viable (for me option) than eg hacking on prayer (because, I think it's just silly to try and work with something exposed to so much bad user input/data as email -- in C):
One additional difference between GMail/AppleMail "conversations" and Thunderbird/Evolution "threading" is that the formers do merge conversations from different IMAP folders (as long as the auto-threading rules kick in), and do merge duplicate mails automagically (e.g. if you have the same mail in "Sent" and in "Mailing List Foo", they will not be displayed twice in the conversation). This for instance allows to have full conversation for emails in Inbox, which is usually not working in Thunderbird since the emails are usually split between Inbox and Sent.
IOW, a "conversation" is displayed in any folder that contains at least one message which is part of it, and contains all messages that are logically part of it, even if they're not stored in that folder. A "thread" is a visual arrangement of the emails in the specific folder, but doesn't cross the folder limits.
Another important point of the user interface is the "mute" button, that hides (archives) the conversation and doesn't allow it to popup anymore even if new messages arrive (in any folder).
I believe it means "flat" threaded email. You care about all messages that are part of a conversation, but you are not concerned about the exact structure of a conversation, so you list all messages of a thread in time-received order.
That is what I was afraid of. It did not make any sense to me why one would advertise to the HN crowd that even though the mail client may not provide the best threaded message display at least it does not provide the worst case scenario of flat sorted by date emails. I really wanted to like mailpile, but the email threading is terrible for serious use with mailing lists.
I think the killer feature of Gmail is the priority box view. It is your personal todo list, your second stage spam filter, etc. Would be great if that could be implemented in a mail client. I would instantly use it!
This is a project for people who like to read and write source code, not for people looking for a drop in g-mail replacement. All the feature requests and UI complaints make no sense here.
it also implements a small imap client(which may work fine for just reading gmail, but imap is a scary thing, everyone uses it a little differently)
https://github.com/naspeh/mailr/blob/master/mailr/syncer.py
that said i guess it's a nice example for writing a mail client in python.
you might want to use mailpile which had it's first release a while back:
http://mailpile.is
but even for that, if you don't care about fancy guis, you're probably still better off running mutt, imap, mu on a shell you have somewhere
edit: i didn't pay enough attention, as was mentioned it's using imaplib, which is the same library offlineimap uses. i do believe though that a lot the work that happens is how people deal with the rfcs. i remember reading about frustrations of the offlineimap author with different servers, and clients. but basically imaplib itself is just a basic protocol implementation, everything else happens on top of it.
and while you can indeed easily store your terabytes of data in postgresql and you can easily setup full text search, it's still much more portable and easily to handle backup and syncing purposes in different formats. Archiveopteryx looks interesting though, so thanks for the hint. if i never write a service to handle mails for other people i'll revisit it.