Hacker News new | comments | show | ask | jobs | submit login
Host your own contacts and calendars and share them across devices (partofthething.com)
227 points by acidburnNSA on July 24, 2017 | hide | past | web | favorite | 97 comments

People looking for easier ways, Nextcloud/Owncloud already does this and you also get with file sync/backup/share for computer/smartphone and more as a bonus; all you need is just a LAMP server.

I tried used Owncloud... and stopped. I tried using radicale... and stopped.

The software was fragile, slow, and broke often. Maybe it's better now (a few years later). But at the time it was negatively impacting productivity.

I've moved to Sogo. Zero problems after install.

File sync is Resilio. Bittorrent sync was unusable, the new re-branded (and fixed) Resilio works fine.

Just my $0.02

As with many things open source a couple of years is a long, long time; Nextcloud has improved a lot as far as I can tell, for me it's decently usable and surprisingly problem free.

How's Sogo with regards to maintainability and upgradability?

I have been researching this space recently, what turned me off was them scanning ports and then reporting to ISPs, supposedly because people were running vulnerable versions, but reporting could make problems for users because some ISPs don't allow servers on home connections.

And here's the money quote: "The effort has been quite successful. Of the tens of thousands server owners who were informed, over 5% had upgraded already in the first ten days."[1] In other words 95% were left vulnerable and they were responsible for it. Does not inspire trust.

[1] https://nextcloud.com/blog/nextcloud-releases-security-scann...

Sogo: zero maintenance. No problem upgrades.

It's really, really, nice.

sogo does not release stable packages anymore (it's only for paid users which is basically businesses). How are you updating sogo?

https://sogo.nu/download.html has all the new releases, where did you find that "only for paid users" info?

Have used systems like Zimbra in the past but SOGo looks pretty neat and inviting. Appreciate the intro.

I use http://mailinabox.email which totally automates running your own mail server. Highly recommend taking a look at it!

What about spam?

I use a self-maintained version of the same email stack (Dovecot, Postfix, Spamassassin) and Spam is pretty much not a problem. I train the Bayes filter on my collection of 10,000 spams and a few thousand hams every 6 months and, while I do get an occasional false negative I feel that spam is pretty much fully under control. Self-hosting email is fun you guys!

As long as you set up SPF, DMARC, DKIM, reverse DNS, and SSL/TLS, plus do that all correctly. Oh, and make sure you pick a provider or host whose IP ranges aren't damaged goods. And don't get me started about sender reputation with the various email providers, or the often erroneous blacklists (and fighting to get your server/IPs off and kept off).

IMO, there are limitless other things I'd rather be doing than these!

You're right that all those things are needed, and it did take me a while to get them all working. But now I know about tools like mail-tester [1] that analyze your emails and tell you exactly how to fix them. Once you know what to do, it's really not that much actual work to set them up.

[1] https://www.mail-tester.com/

How feasible would running a mailing list (say 10k subs, one email/week) be off your setup?

We're doing something like that.

The problem is that you need a lot more volume to be able to use the feedback loop mechanisms ( https://blog.returnpath.com/what-is-a-feedback-loop/ ) efficiently.

Sure, it all works, but it's still the old piece of shit SMTP wild wild west, because when it stops working, you can do nothing. (But usually going to the saloon, having a rough night and waiting will solve things.)

You forgot DNSSEC +DANE

You do not need to set up DNSSEC+DANE, and, in fact, doing so is very likely to make your system less reliable; the primary function DNSSEC has in practice is to cause outages.

Observe how few of the major sites are DNSSEC-signed. In reality the only purpose signing has is allowing your site to vanish from view of the few DNS resolvers dumb enough to do DNSSEC validation.

Self-hosting email is fun you guys!

I figured that it would be, but trying to set it up was the single most frustrating thing I've ever experienced running a Linux box, and I never did get anything properly working after many days of trying. The principles seemed straightforward enough and there was plenty of software available, but what I really wanted was to self-host the mail store and IMAP server but send/receive via ISPs or other services who are better at administering a full-time SMTP setup than I am, and somehow I just never could find the magic incantations in the settings files to make that go. :-(

Did you try exim4? That was pretty good a few years back, but honestly I took the advice to offload to service and don't run a "real" one anymore. I do on my dev box as localhost so I can test authentication scripts and the like but that's all.

Well, for outbound you'd presumably want an "authenticated smarthost" with setup details on your side varying by what mail server you're using locally.

For inbound, you'd presumably want the server using fetchmail or getmail to then push them into your local mailboxes. Should be plenty of documentation on those projects.

There was plenty of documentation, and what I was trying to do was along the lines you mentioned. My problem was that if anything there were too many different moving parts in the software and too much documentation. Individually it was quite clear what each part was supposed to be doing. Collectively, it simply didn't work: messages weren't reliably sent or delivered, sometimes generating various kinds of logs or error messages, other times silently as far as I could tell. The hard part was trying to pin down which part of the system wasn't set up properly, and I simply didn't have and couldn't find the knowledge to solve that problem within a reasonable amount of time.

Given that fundamentally neither email protocols nor mail store structure on a typical Linux box are particularly complicated, and that all of the software I was using was well established and reputable, it was an exceptionally frustrating experience. Someone with more skill than me would surely have installed the relevant half dozen or so packages from the distro, edited a few configuration files to set up the remote mail server details and identify the local processes to each other, and had everything working within a few minutes. Unfortunately, without that knowledge the sheer number of options and possibilities was overwhelming and I never got that far. What I really needed was a one-page HOWTO for the most simple use case and a setting for each package that said "don't try to do anything clever or unusual", but what I had at the time was literally hundreds of screens' worth of detailed parameters and config file options, each written according to the conventions and assumptions of their own package.

Why don't you just download your mails from your provider via getmail or the like?

I could, but the object of the exercise was to be able to fetch the mail to a local server and then access it from any device on my network (including potentially from a mobile device via VPN). I also wanted to do things like spam filtering and automatically filing incoming mail according to various rules, the same as I'd do with a standalone mail client, and sending all outgoing mail via a system that could keep copies. This is well beyond the scope of running something like fetchmail on its own.

Most email clients allow the user to configure a separate outbound email server.

I self-host my email with a very similar setup, and while alright I wouldn't go so far as to call it fun. It's funny how pushing messages to my iOS device is actually harder to configure than sending emails. (hacked-together exchange support / IMAP push etc)

I am commenting to confirm that nextcloud works very very well.

I am running the official docker image with an apache2 frontend (http/https) and it pretty much works flawlessly.

They don't offer client side encryption yet, do they?

Just checked the GH issue and it is still open: https://github.com/owncloud/client/issues/4327 and https://github.com/nextcloud/server/issues/5145

I have about 1TB of data that I tried to sync with NextCloud. The Windows client crashed every night, so I finally gave up and moved to Syncthing. Has anybody else had a similar experience?

For that much data rclone might be better.

I don't have any experience with their Windows client, but webdav with nextcloud has been painfully slow in my experience.

This is a reasonable recommendation, aligned with the summary of the article.

While this is nice, and much better than using e.g Google, it is even better to use an end-to-end encrypted solution. Messaging has underwent the same transition in the last few years. Trusting the server, even if yours, is just not enough.

I created https://www.etesync.com for that purpose exactly, it does end-to-end encrypted sync for Android and desktop for your contacts and calendar.

Looks really cool! I have been looking for a solution similar to this one. Signed up.

One thing that I miss though is a nice web interface for calendar and contacts. Especially if I want to do some sort of cleanup of data. Is this functionality planned? Or is there some open source web calendaring software that you would recommend?

Glad you like it.

I've also been looking for such a solution for years, and except for the short-lived Flock by the Open Whisper Systems people, I couldn't find anything. That's why I finally decided to create it almost a year ago.

Edit: you just edited your reply while I was writing mine, to answer your additional question:

For a variety of reasons, the browser is not a safe environment for such an application, at least not currently. There is however a CalDAV/CardDAV gateway (beta) that you can host locally and essentially use any application that supports those to quickly edit your data, or alternatively, if you are a programmer, there's an open source (like everything from EteSync) python API (both low-level and high-level) that you can use to manipulate the data.

When communicating with your own server using https, it is already end-to-end encrypted.

While this may feel true, there are a few reasons why it's not.

1. This assumes https can be trusted, that is, MITM is not a possibility. This is a fair assumption for many people, but not for people whore are scared of state actors, which is common for journalists in and citizens of some countries. - This can be mitigated to an extent with certificate pinning, but I don't think all software support it (correctly?) and it's another thing to get wrong.

2. This assumes your server is trusted. It is not. If it's hosted in a remote location (e.g. VPS or even metal in a server farm), it can't be trusted. If it's at your home, based on how much of a target you are, it can't be trusted. Even if your server is physically secure, this assumes it won't be hacked, and you should never assume a public facing server won't be.

Of course this all depends on your threat models, and this may not apply to you, but https to your own server is definitely not the same, and given that adding end-to-end is almost "free" for most users, it's a good idea to error on the side of safety and just do it.

Re: 1. You can deploy additional security measures such as certificate pinning or a VPN tunnel to your CalDAV server if you have a high security requirement.

Re: 2. If your server is untrusted (because it's a remote virtual server) or hacked, e2e will not protect you.

1. That's exactly what I said, though you are just patching a broken system here, why not just use a system that's resilient to all of this in the first place like with end to end?

2. How so? EteSync for example has a git like integrity verification (just with HMAC instead of hash), so it's easy to check consistency across clients, and the server can't forge anything. The worst the server/MITM can do is stop syncing a specific client which would be easy to detect. A rogue server can't even omit specific changes, only stop sync. So I don't agree with your assertion.

I've done this before. It becomes tedious to maintain, but I can imagine myself doing it again in the future to allow me to do what cannot be done in the common clouds (iCloud, G Suite, O365).

I would love a whitelist-only inbox. I'm sick of spam, marketing mail, etc. I'd like one public email address that catches all the garbage. Then, I'd like a private email address that accepts emails from my contacts, and bounces the rest.

> Then, I'd like a private email address that accepts emails from my contacts, and bounces the rest.

You're doomed. Because at least one of your correspondents is going to upload their contacts to some service, and that service is now aware of you.

I think OP intends that mailbox to be the one that only accepts mail from a whitelist of addresses.

This is really easy to set up using Dovecot Sieve [1]. I set it up the other day to bounce reject messages that had too many swear words as a proof-of-concept for an idea a Facebook friend had about making sure people stay nice in email. The syntax of the server-side filters is a bit strange but there's even a Thunderbird plugin that allows you to edit them from the client.

[1] https://wiki2.dovecot.org/Pigeonhole/Sieve

Bouncing is a bad idea as it can be abused for spam:

1. Fake the origin to actual destination of choice

2. Your server bounces the original message wrapped by the SMTP error report.

3. Actual destination receives error report containing spam content from YOUR server

A proper solution would be to REJECT the message before it is accepted. However, Dovecot Pigenhole doesn't support that (yet).

I've been using radicale[0] for this for some time, it's a bit quirky with thunderbird but I've had tremendous success with Apple Calendar for iOS and the Contacts application.

And it's using standards (Carddav, Calddav) so you can be somewhat secure in the knowledge that other programs can implement the protocol (or should). :)

[0]: http://radicale.org/

EDIT: I guess I should have read the post, but eitherway I'm going to keep this comment up as a review of the software.

Check out Mailinabox. This Project does most of it for you. https://mailinabox.email/

That's nice, thank you for the link.

After spending way too much time messing around with stuff like DAV and syncing using same, my current scheduling and reminder system is based on this:

* https://www.roaringpenguin.com/products/remind/

Every day at 6am my server runs a cron job. If there are appointments I get an email and an SMS. The server also makes a unique noise over the house PA system (I live alone) but that is mostly because I can.

What exactly was wrong with a DAV server? I can't imagine it being more complicated than what you've settled with.

DAV was always incredibly unreliable when you tried to host it yourself. Remember the original web servers + browsers, as one would support some new feature and another would break it? It was like that.

SyncML was pretty rock solid and interoperable when you could find support for it, but the damn thing was such a nightmare to implement that the only open source implementations were buried inside groupware solutions.

Something I've been looking for is a web service and android app that I can self-host that will allow me to read and send SMSs from my desktop. The important thing here is self hosted.

I think something like this would be really useful. I've seen services that provide this via a 3rd party hosted service but I don't trust someone at some random company not to read my messages.

> I think something like this would be really useful. I've seen services that provide this via a 3rd party hosted service but I don't trust someone at some random company not to read my messages.

... do you know how SMS works? Specifically that it is sent in plaintext for anyone who cares to listen anywhere along the path?

Maybe try Signal. It has a desktop app.

> ... do you know how SMS works?

Yes. Does making a bad situation worse help anyone?

I'd use Signal/RedPhone if even 5% of my contact knew what that was.

Send an email to:


For example, with Verizon:


Doesn't address the self hosting part but this is an easy way to send SMS from a desktop/laptop.

This should really be handled by the carrier. Look for a carrier that allows you to use your number from anywhere via SIP or WebRTC.

I'd rather have a self hosted service, that I can modify if need be, than be tied to a carrier.

In that case you'd need an app on your Android phone to relay SMSes between the phone network and your web app. The advantage of doing it via the carrier is that the phone is taken out of the loop and you're directly sending to the carrier via the internet.

> In that case you'd need an app on your Android phone to relay SMSes between the phone network and your web app.

I'm fine with that.

> The advantage of doing it via the carrier is that the phone is taken out of the loop and you're directly sending to the carrier via the internet.

The disadvantage of doing that is you than need to stay with that carrier. I'd rather be able to change if my carrier starts doing things that are shifty without needing to change the way I interact with my phone.

Nextcloud (and therefore probably Owncloud) has a 3rd party app, though only for reading at the moment.

I started with Radicale a few years back (5+?). I found it too brittle, and switched to Baikal - http://baikal-server.com/ . I've been very pleased with Baikal for since the switch (again, 5+)

Baikal was going to be my second choice but I never got there because Radicale went pretty well.

I've been trying to do this with inf-cloud, and baikal



It mostly works so far, but I'm definitely looking for a simpler, easier to implement solution. I looked at Radicale before, but I forget why I passed it over. I'll have to look into this.

Don't redact information with a blur filter! It can be reversed! Use solid color boxes.

Gaussian blur can be reversed? I assumed it was destructive. Thanks for the tip.

How would it work for email invitations automatically being added as calendar entries?

There is a message-based interoperability protocol for iCalendar: https://datatracker.ietf.org/doc/rfc5546/ and https://datatracker.ietf.org/doc/rfc6047/.

This is what Google Mail + Calendar are using, including features like RSVP, etc. It's all there, just passing iCal objects around.

No open-source calendar server that I know of supports this - they all rely on your email client managing this client-side (when you accept an invitation it adds it to your calendar).

The only thing I miss in Google's thing is ability to allow people w/o tech savviness and w/o google account to sporadically add things to my calendar and have them marked as unconfirmed (like semi-transparent). I don't care if they'll recieve any further info, I want people to add their things to my life.

The subject of "who the hell isn't tech savvy enough and doesn't have an account already" is quickly answered with: I'm an uncle, my niece knows her way around the computer and knows the calendar, but has no idea why'd she need an e-mail account, and wouldn't bother to even try at this point.

I've been looking a Zimbra [https://www.zimbra.com/] for a while. I like the general idea of the product. Even the free version is feature-filled. My only problem with it is that it seems to be a resource hog. You'd have to get a fairly fat Linode to run it.

I've used garden variety email hosting auto-magically setup on VPS's from such luminaries as GoDaddy without problems for many years. I have never worried about how to setup an email server, just get a cheap VPS, setup email addresses and you are good to go. Given this experience I have to admit to not understanding why Zimbra is so fat.

I'm a fan of Zimbra. In reality...a Linode 4096 handles 5 "normal" users just fine, and a few here and there users, possibly more...that was all I needed. Tack on Z-push and you have a nice Activesync implementation as well. Then it is just like gmail just plugin the server and password and calendar, contacts, email, and whatnot all sync as you would expect. http://z-push.org/

I probably have a dozen email addresses across five companies that I need to manage. I thought Zimbra would be the answer but server requirements sky-rocket quickly.

> I’m trying to learn ways to minimize my reliance upon large companies for handling my day-to-day personal data.

Except that the moment you "share them across devices", at least one large company will silently grab your contacts anyway. And several others will try to, too, with one excuse or another.

To be fair, though, I read this as "I don't want a company closing down/sunsetting a service to end my ability to sync my data", not "I don't want companies to have my data".

So though you have a point, I don't think it goes against the post.

Thanks, this is primarily what I was going for. I'm also trying to be just a bit harder to snoop on but do realize that I'm nowhere near unsnoopable.

Isn't the whole point of running your own servers for mail/calendar/contacts/etc. that you don't have to rely on synchronising things via anyone else's infrastructure?

I want to have multiple devices able to access my important data.

I have no interest in sharing that data with Google, Apple, Facebook, LinkedIn, or $SMALL_TIME_APP_DEVELOPER and will actively avoid any system that requires me to do so.

Yes. And it's this that made me write my own zero-knowledge sync service. I encrypt my stuff on the client, sync it to a server using store-and-forward, and pull it down from other devices whenever they have an internet connection.

It's essentially file share for database records, and I remain surprised that something like this isn't in wide use already.

Docs haven't been updated in years, but here's a description of the thing back when I was focused on the client rather than the sync engine -


Why would anyone get your contacts if you sync them via your own CarDav server? You don't need to sync them to iCloud / Siri on iOS, and you can use non-Google Android phones.

As a few examples, there are a variety of social-network applications (LinkedIn, Facebook) that request access to your platform specific contacts application. They then do a variety of interesting things with them, such as suggesting users with numbers they have "on file" as potential new "Friends", etc. This is seemingly server-side.

Whatever you're using to sync contacts, without syncing them to this contact app, it seems (to me) as useful as keeping a textfile full of phone numbers in Dropbox. You won't have any useful prompts telling you that it's Dave phoning you, John texting you, or Alice's number to phone/text.

This is, admittedly, neatly worked around by keeping your permission requests under control, but... stuff like this[1], from before the granular permission system on Android was available, are annoying.

1: http://bgr.com/2011/08/12/facebook-stole-every-contact-and-p...

That's not an automatism nor inevitable, though. I don't have either of these apps on my phone. And both iOS and now Android have these dynamic permission dialogs where you can refuse access to your contacts. It may be inconvenient to use a messaging app without granting it access to your contacts, but I think all of them work that way on iOS. Refusing to work without access would be against the iOS guidelines iirc.

But that wasn't the article's concern at all, as you pointed out in another comment.

What about my contacts on Facebook, Google, Microsoft, Apple, Instagram, Twitter, LinkedIn, Skype, Snapchat, WhatsApp, WeChat, etc.

At least some of those services will export vCards, which I assume Radicale can import, though I haven't used it.

Radicale is really just the server and doesn't have its own import functionality. Instead, the clients support importing vCards. In the article, the CardBook Thunderbird plugin was used for this purpose.

Cloudron and yunohost have radicale, email and nextcloud. Sandtsorm lacks all of them, last I checked.

anyone has experience with this


I'm looking into a similar solution, and found roundcube.net features to be exactly what I need

honestly win-win in 2017 would be to securely orchestrate pesonal cal,dns,mail,notes,drive on one of the container hosting platform for reasonable cost.

while doable. cost is still far more common shared hosting or accepting a mail on google.

You lack critical reading skills.

His concern is not that they companies would grab a copy, his concern is that he used to rely on them for syncing. He no longer has to rely on them. Google can shut down calendar and contacts and he would still be fine.

We detached this subthread from https://news.ycombinator.com/item?id=14837015 and marked it off-topic.

You are too sensitive. My statement was literally (and I do mean literally) an objective statement of fact. Nothing more nothing less.

This ultra-politically correct mentality is what makes it hard to battle people like Trump.

The comment violates the guidelines by jumping to a personal attack with “You lack critical reading skills.” We need you to not do that when commenting here.

Please don't use such stark language on HN. There's a much friendlier way of saying that, something along the lines of "I think you misunderstood the author's concerns".

You are left-coast/euro sensitive. Nothing that he said was incorrect, non factual, or otherwise out of line.

It's against the guidelines. They are very explicit about such things:

Be civil. Don't say things you wouldn't say in a face-to-face conversation. Avoid gratuitous negativity.

When disagreeing, please reply to the argument instead of calling names. E.g. "That is idiotic; 1 + 1 is 2, not 3" can be shortened to "1 + 1 is 2, not 3."

Please don't insinuate that someone hasn't read an article. "Did you even read the article? It mentions that" can be shortened to "The article mentions that."


It's not called "left-coast/euro sensitive", it's called manners.

I have no idea what you mean by "left-coast/euro-sensitive".

However, I will put what lorenzhs said a bit more bluntly.

Please don't go out of your way to be a dick - there is no need for it, and it benefits nobody.

Hosting calendar and contacts on a personal server accessible over the web smells like a potential security nightmare to me. For example, how do you make sure that whatever third party tool you use to sync your phone with the web server has been well designed to protect against a large attack surface?

I'm sure it's a bit dangerous, but when you run it as done in the article, behind an Apache Reverse Proxy with Apache authentication, you're relying hugely on Apache for the sync security, and that has been well designed to protect against a very large attack surface.

Another advantage is that you're just different. So if someone (or some State) attacks a commonly-used service, this will be protected simply by being different.

You'll need to be more specific. Is it worse or better than running your own mail server? Your own git repo? Your own VPN server? In what ways, and why?

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact