I used to use mbsync but when my employer migrated from Cyrus to Exchange IMAP I couldn't get mbsync to work with it, but OfflineIMAP worked just fine.
Now they've migrated to GMail, which works fine with both, and even though OfflineIMAP is more flexible & easier to customize I'll probably switch back at some point.
OfflineIMAP regularly pegs my CPU at >100% just downloading a few dozen E-Mails, whereas mbsync's CPU usage never went above 2-3%. When my laptop fan starts spinning it's usually OfflineIMAP in the background.
So if you're looking for an IMAP->Maildir retriever or backup solution consider trying out mbsync first.
It's a PAIN to switch between these programs, or even using the same data when switching between different servers (e.g. Cyrus -> Exchange). Inexplicably neither of them have some sort of reindexing mode where they can take an existing Maildir and reconstruct their local database from local/remote Message-Id's. They both use & maintain an opaque local database. That's why I haven't simply gone back to mbsync.
I'll second this: I used OfflineIMAP at first, but switched to mbsynrc after being unhappy with OfflineIMAP's performance. (It also seemed to hang semi-sporadically for me, which made it a pain to launch as a cron job.)
Lots of factors affect which tool is the better for a given user, of course, but I personally have been happier with mbsynrc.
I haven't dug into why it uses that CPU in any detail. Looking at its output / tcpdump of what it's doing it goes to >100% CPU for 10-30 seconds when it's finished fetching the status from the remote & is presumably in some tight inner loop comparing it with local data to decide what to do before it starts downloading messages, and then briefly spends a few more seconds at >100% after the messages are downloaded, presumably updating its local state.
When it's just issuing IMAP commands or downloading messages from the remote it sits at 1-5% CPU.
As I pointed out OfflineIMAP works just fine for me, but in retrospect I'd be happier if I had stayed with mbsync. I don't need any feature from OfflineIMAP that it doesn't have, and it both uses less CPU/Memory & it finishes deciding what to download & downloading it in way less time.
Thank you for mentioning mbsync, I will keep it in mind.
What goes wrong with plain text responses in Outlook, by the way?
For what it's worth.
Alternatively, since everything's downloaded locally as mboxes / maildirs, use grep / ag / ripgrep, etc.
I can't remember the exact reasoning now but several years on I am using dovecot's excellent dsync protocol over SSH to keep my laptop in sync with a private mail server, and have never looked back.
Combined with pine, exim and the excellent mairix (surprised not to see that mentioned yet) I enjoy a decent search on a fast MailDir implementation, with full offline support.
I have been using this for years and it's great software. One tip, store each different email account in it's own "database" my crontab looks like this:
0 0,6,12,18 * * * /usr/local/bin/gmvault sync -d /XXXXXXX/email-backup/joshstrange --resume email@example.com >> ~/mail-backup-logs/joshstrange.com.log 2>&1
0 1,7,13,19 * * * /usr/local/bin/gmvault sync -d /XXXXXXX/email-backup/otheremail --resume firstname.lastname@example.org >> ~/mail-backup-logs/otheremail.com.log 2>&1
A professional mail host ('professional' as in 'someone that somehows makes money by hosting email') is way more unlikely to lose data than most of us are.
Making backup is great by the way, but for such task I'd strongly recommend the good old fetchmail or the newer (and better) fdm (https://github.com/ft/fdm). Both support imap(s). Both can save to either mailbox or maildir.
If you need a local copy, go for fdm and a local copy of your mailbox, plain and simple.
I think their point is not that you're less likely to lose it yourself, it's that the probability that both you and the provider losing it at the same time is extremely unlikely. I.e., two points of failure required to lose your data.
The code probably works, but don't expect any help or updates.
Not all dead projects are actually dead. Some are just "completed".
I started using offlineimap in 2008, which feels like a long time ago but of course is a lot more recent than the first appearance of imap.
The first version of offlineimap (1.0) was released in 2002.
If you have already made your experiences with storing email in a sql database, it would be very appreciated if you would like to write about the pitfalls to avoid and the problems that occured.
Thank you very much for your attention!
There's also "Archiveopteryx":
Another question is what do you want to do? I've not played that much with sql store for mail - but notmuch (and mailutils) make good use of Xapian for quick index/search of mail, if that is your end goal.
* We have more than 25K IMAP accounts running across 3 different 8core machine.
* Application is written over our custom async web server and our nio tcp client.
* Handling more than 50k connection per server instance.
Running a standalone client is much more easier because we have to concentrate only on syncing/fetching 1 account per instance. Rather in our IMAP Client Server we have to deal with multiple accounts with connection restrictions,failover handling, proper retry mechanism etc...
We have done this backend engineering for one our product.
That works, but you have to program a bit lua in order to setup imapfilter correctly.
Would OfflineIMAP download 'more' data than i already have backed up?
It seems to me all this mail solutions requires that
the password of the mail server must be somewhere
visible in a poor plain text file which is a no-go
There's an addon that lets you store Thunderbird's passwords with Gnome's keyring. That said, in the end you must either type it each time or it must be stored in a way that can be read in an automated way.
I can't compare it but I have been using gmvault for quite a while and love it.
Python v3.4+ [STALLED]
- For READING mail, my offlineimap knows about multiple accounts and puts them in different directories on disk
- mu indexes the PARENT of all those directories, so as far as mu is concerned, there's a single mail store with ALL the mail. The m: filter can be used to select specific directories, which map to different accounts here
- For SENDING mail, mu4e knows about multiple contexts that are detected based on rules you can set (for instance, when replying to email, it'll set the context to whichever account the mail I'm replying to was sent). Or one can set the context explicitly with the ';' binding.
Works quite well
Link to the docs: http://www.lesbonscomptes.com/recoll/usermanual/usermanual.h...