
My Self-Hosted Life - webworxshop
https://webworxshop.com/2016/10/10/my-self-hosted-life
======
oridecon
For me the problem is making stuff secure (..enough, and that would require my
"full" attention).

That's why I would love if a company like Mozilla, that offers stuff I support
like Firefox, started offering a paid "Google suite" with custom domains. Or
at least e-mail and storage. I understand most of it is a race to the bottom
but it might be viable if it was open source, community driven. What I care
about is knowing my data is not being sold and paying for not having to
monitor logs, update servers and all of that.

I do use the cloud but only for encrypted tar files.

I do know about alternatives like the ones listed at privacytools.io, but it's
all about trust (in before someone tells me why I shouldn't trust Mozilla).

~~~
lucb1e
Of all the companies, I think Mozilla is one of the best ones these days. I'd
not be surprised to hear there has been bad behavior from an employee or some
corporate manager, but overall, they still stand for something.

------
luxpir
Completely sympathize with the reasoning behind this.

Have done similar myself, trying to make the most of low-resource software to
run on low-powered hardware.

Self-hosting email, while a PITA at times (must use mailinabox next time...
but spf, dmarc, tls everywhere wasn't too bad to set up, just the multiple
accounts were fun) is extremely satisfying. And no, it doesn't mostly end up
in a gmail inbox. But even if it does, Prism leaks resulted in Google de-
pwning themselves (internal encryption), supposedly. For now, etc.

Otherwise RSS (newsbeuter), XMPP (finch), email (mutt), IRC (irssi), Drupal
with nginx + php opcode caching + microcaching, Hugo sites with nginx,
letsencrypt/certbot. SSH tunnels for socks and remote connections. Cheap
router with tomato firmware... Quite amazing what you can run on a pi, router
and low end VPS with a bit of time and effort.

Even transitioning much of my word processing to markdown/latex/pandoc for
dead simple clean and quick publishing.

I may be completely deluded, but for me the future is still the command line,
low powered hardware and FOSS. Its strength is in its fluid development that
seems able to endure decades of change and advances, weathering rises and
falls of shinier things.

More power to the OP, may many more share their non-shiny setups.

------
startling
I was interested in Piwik until I noticed the many, many open security issues:

[https://github.com/piwik/piwik/issues?q=is%3Aopen+is%3Aissue...](https://github.com/piwik/piwik/issues?q=is%3Aopen+is%3Aissue+label%3A%22c%3A+Security%22)

Includes an issue from January about insecure random-number-generation:
[https://github.com/piwik/piwik/issues/9473](https://github.com/piwik/piwik/issues/9473)

and one from March claiming that it doesn't reliably use TLS to update
plugins:
[https://github.com/piwik/piwik/issues/7582](https://github.com/piwik/piwik/issues/7582)

It bothers me a lot when people advocate self-hosted services with really bad
security posture to increase privacy (or "control", as the author says).

~~~
educar
Having seen the issue trackers of many closed source SaaS products, let me
tell you that this is hardly anything. If anything, it's great that all this
is in the open and I can assess and prepare myself in terms of
sysadministration accordingly.

~~~
startling
Maybe!

But for example Piwik's main competitor is Google Analytics. It's reasonable
to not trust Google for privacy reasons, but Google employs one the largest
security teams in the world and produces very well-regarded research. I trust
Google with securing Google Analytics more than I trust Piwik's developers
with securing Piwik.

------
tomc1985
Yes, snicker at the failing server of the self-hosting guy, ye who are made
rich by SaaS.

I applaud him for denying money to an industry that doesn't deserve it

~~~
eropple
SaaS isn't IaaS, and administrative incompetence happens on both IaaS and
self-hosted.

~~~
tomc1985
It's software as a service, is it not?

And administrative incompetence? Dude had his server temporarily overrun with
traffic. C'mon.

~~~
eropple
No, you would run Wordpress on EC2 or Google Compute; that would be IaaS, not
SaaS.

As far as competence goes: you're submitting yourself to Hacker News and your
Wordpress site isn't caching content? Yes, that's incompetence.

~~~
tomc1985
Sure, if you're a Hacker News(tm) web developer.

But computin' is such a big field...

~~~
eropple
Or if I'm a high-school student in 2005. Because I was building Wordpress
sites for people then. And staticize (or, later, staticize-reloaded) was on
everything I built. If you can't step to a high-school student in a backwater
state who had access to state-of-the-PHP _eleven years ago_ when you want to
start virtue-signaling your way to glory (and don't pretend creating an
account just to post a story is anything but), you are not building a argument
for yourself.

The best part is, though? For all his very manly self-reliance, this blog of
his _is_ on a DigitalOcean droplet. He isn't "denying" anything to anyone,
whether they deserve it or not (and that's a laughable sentiment in your OP
anyway, _while we 're at it_).

~~~
tomc1985
Well, technically Linode. Not DO, but close enough :P That is pretty funny
though.

My sentiments remain. I want a world of self-reliant experts, not halfwitted
idea-people who see technology as a means to an end. That may not be this
world, but one can dream.

PS: He admits as such in the blog that it is hosted on a VPS. And no one
person is going to be able to step to another on every possible criteria.
(Frankly, I read your high-school 2005 PHP staticize story with probably the
same eye-rolling you gave that dude, cause i was building static HTML in
elementary school in the 90s and was doing dynamic flash sites before AJAX
became a think in the mid-aughts... and the dude sitting one cubicle over
would roll HIS eyes at me because he used to program simulators and missile
code for the military in the 70s and 80s. Holier-than-thou can backfire)

~~~
eropple
I was building static sites in elementary school in the nineties, too. I was
building _correct things_ that _worked in a business context_ in high school
in 2005. In your haste to praise the guy for nonsense virtues of "self-
reliance"\--and let me be real for a sec, I've been paid in the last year to
debug kernel drivers on physical hardware and architect self-bootstrapping,
multi-tier, cross-datacenter systems, I could be said to be big on "self-
reliance"\--you missed the forest for the trees: _his stuff doesn 't work_.

Setting up mail is easy. Setting up a web server that doesn't go down under
medium load is easy. If you want to lecture others about this sort of thing,
you had best come correct. Because those of us who _do_ \--and who especially
those of us who are capable of using tools hosted in either our own racks or
Amazon's without trouble, who are likely to have a problem with the shitty
virtue-signaling in his post--will call you on it. Because the world is not so
simple as to scream that "SaaS is bad!" and, _sometimes_ , there are economics
of scale worth leveraging. There is no virtue in refusing them; there are
places for self-hosting and there are places for cloud systems and ascribing
the rise of cloud services to "halfwitted idea-people" is a sign of a
profoundly juvenile understanding of both the technologies and the economics
involved.

------
zodPod
Kind of comically ironic. I can't currently load the article.. Hug of death
I'm guessing? So much for self-hosting?

~~~
dest
Keeping the site online despite being self hosted and on the front page of HN
might be the ultimate challenge for the people fond of self hosting!

~~~
netule
Not using Wordpress to serve static blog content should be a good start.

Also running the database, webserver, mail server, etc. off a single machine
is just asking for trouble.

~~~
eropple
Wordpress can trivially handle "static blog content" with any of a number of
caching plugins. (And you get a lot more out of it, in my experience, than
trying to use Jekyll or the like.)

nginx, PHP-FPM and APC, your caching tool of choice, done.

~~~
mmagin
And yet after all these years, they've never bothered to roll functionality
like that into the base install.

~~~
eropple
I much prefer, as somebody who uses this stuff pretty heavily, to make the
choice of caching particulars myself and to not have to unscrew their
defaults. A cache API would be fine, but I'd rather have as few decisions made
inside Wordpress as possible.

~~~
mmagin
I know that attitude well, but really this is a problem that's persisted for
years and they have basically acted like it's not their problem, which leaves
me unimpressed.

------
zekevermillion
This is the first I've heard of Syncthing. Would be interested to know if hn
people think it is reasonably secure. Finding a good self-hosted sync option
is hard, one of the last places where I'm reluctant to give up SaaS (using
Egnyte now since ~2010). Waiting for magic folders from Least Authority. And
then even if they put down zcash long enough to finish the magic folders
project, someone still has to build a GUI for idiots like me. Is Syncthing
security OK for relatively normal people who don't have any secrets of
interest to the Russians?

~~~
Sir_Substance
Syncthing has a good reputation in every department except the usability one.
It's workflow is generally considered a bit clumsy, but it's a good product.

You could also look into librevault if you're interested in pursuing this
avenue.

------
update
Good post! I run a homeserver too. It serves media content (1080p) to everyone
in the house and never has any trouble. It's main purposes are running Plex
and nginx, and soon, e-mail. Only port 80 is accessible to the internet atm
(it's fun to read the access logs & see the latest exploit attempts)

It's hard because you have to really be on top of security, but it's worth it.

> You might say that this is a bit of a cop out, since this all runs on a
> virtual machine, which itself runs on someone else’s computer. I would
> agree, however it’s a nice middle ground between going all out with your own
> servers and running everything in the cloud. To me the reality that the VPS
> is in the cloud is obscured by the ability to control every detail of its
> running software.

Hmm. Contrarily, this part convinced me to completely self-host all aspects of
my internet life.

Also, sorry about your blog going down! What was the cause of that?

------
aj_ashton
> I have taken the decision to self-host as much of my online services as
> possible, rather than relying on the cloud (since that’s just someone else’s
> computer)

I understand the desire to take control, and have been on a similar kick
lately to reduce my dependence on The Cloud. But I'm not an experienced
network or system admin and I'm not concerned about using Someone Else's
Computer when it's convenient for me. What I am worried about is my data being
locked in to any particular service or being a pain to migrate somewhere else
if I want.

I've also been documenting my progress, if anyone's interested. It's on a not-
self-hosted-but-still-extremely-portable GitHub Pages site:
[http://ajashton.ca/decloud/](http://ajashton.ca/decloud/)

------
newsat13
For those looking into self-hosting, there are many great options these days.
[https://github.com/Kickball/awesome-selfhosted#self-
hosting-...](https://github.com/Kickball/awesome-selfhosted#self-hosting-
solutions) has a full list.

------
trizic
I believe Monit could have solved this, check it out:
[https://mmonit.com/monit/](https://mmonit.com/monit/)

------
zelon88
I second your self-hosted life. "The Cloud" is just someone else's computer
running Apache instead of mine. I will never pay for someone else's servers.
That just makes no sense to me.

~~~
rabino
Do you homeschool your kids? Grow your own produce? Kill your own cows?

I mean, why don't you let experts do what they do and you spend your time
doing what you do?

Unless you do servers. In that case, sure. Whatever.

~~~
newsat13
You do own your own phone right? Owning a server on the cloud is no different.
There will be a future shortly where everyone keeps their data on their own
servers instead of handing out to all these corporations.

~~~
rabino
I own my phone, but the data is still in the cloud.

I'm not sure I agree with your version of the future. People don't know enough
( and don't want to) about security, scalability, availability, etc.

~~~
newsat13
OK, since you own your phone. Do you have any clue about security and
availability of your phone? Why do you think millions can own and "run" their
own phone but not own and run their server?

------
codeddesign
What about just the simple fact of internet connection? Bandwidth limitations?
Are you on a business line? If not, I probably wouldn't be a very happy
neighbor ;)

------
tomc1985
This is how computer nerds have been doing it since the 80s.

------
lwhalen
FTFA:

For those that know me, I’ve made no secret of the fact that I believe that
you are better off doing something yourself than outsourcing the task to
someone else, especially in areas that you are interested in or have some
expertise. For me this has particular value in the case of my computing. As a
result, I have taken the decision to self-host as much of my online services
as possible, rather than relying on the cloud (since that’s just someone
else’s computer). I’ve been working on this for years (actually the whole time
this blog has been dark and before) and at this stage I’m mostly there: almost
all of my digital life is provided by Open Source software, running under my
control.

This post will detail what I’m using and how it all fits together. I’m not
going to go into technical specifics since otherwise this post would be huge,
perhaps I’ll focus on some of that in future posts (feel free to make requests
in the comments). Also, please note that my setup is by no means finished and
probably never will be, it’s an ongoing project and it has become pretty much
my main hobby to install and maintain this stuff. In the Cloud

I’m going to start right here, with this blog, since that was where the whole
thing really started. This blog existed well before my undertaking to self-
host. In the early days it lived on a shared hosting plan provided by
Dreamhost. The site has always run WordPress, although I’ve toyed with the
idea of moving to a static site over the years, I’ve just never quite managed
it. In 2011 I moved the site to a shiny new VPS provided by Linode, where it
has lived ever since. There is also a Piwik install for tracking website stats
(which I’ve blogged about before).

The main motivation behind the VPS was to install and configure my own mail
server setup, something which I ranted about shortly after. This setup has be
serving myself and various family members well since then, with really very
little maintenance on my part (almost everything is automated).

There have been various other uses for the VPS over time, many of which
haven’t stuck. Probably the most successful has been an installation of TT-
RSS, which started life on my home server and at some point moved to the VPS
for convenience of access. I’ve also dabbled with various chat applications,
mainly XMPP based, but they’ve never really been that useful due to the
network effect of no-one else using them! At this stage email has become my
primary form of communication.

You might say that this is a bit of a cop out, since this all runs on a
virtual machine, which itself runs on someone else’s computer. I would agree,
however it’s a nice middle ground between going all out with your own servers
and running everything in the cloud. To me the reality that the VPS is in the
cloud is obscured by the ability to control every detail of its running
software. Its also pretty nice for services which I want to be reliable, since
Linode almost never skips a beat. At Home

So the VPS is one thing and is really used for critical services or stuff that
needs to be accessible to the wider Internet (like this site), but the real
magic happens on my home servers (yes, there is more than one). My main server
(now on its second hardware iteration) started life as a MythTV system and
still does a great job in this respect. Many other services have been added
over time, such as an MQTT broker (mosquitto), git server (gitolite+gitweb), a
calendar/contacts server (Radicale) and file synchronisation (Syncthing). At
some point I also switched out the MythTV frontend and replaced it with XBMC
(now Kodi).

In the last couple of years I’ve been moving further down the home automation
route, rather than just sensing and logging via MQTT. I’ve finally settled on
Home Assistant as my automation controller and UI, along with an instance of
Node-RED to do some miscellaneous processing. This all runs on the main
server, with a Raspberry Pi 2 in the garage functioning as what I like to call
‘the gateway’ (it has a couple of radios and some sensors connected and runs
another instance of Node-RED to shuttle this data to MQTT). In addition I have
my home CCTV set up using a couple of webcams and MotionEye. One of the
cameras is located remotely and connected to another Raspberry Pi (this time
an old model B) and streams back to the main server with mjpg-streamer.

I also run a pfsense based firewall to protect my network and provide remote
VPN access. This runs on an old netbook with an extra USB ethernet adapter.
The internal network is partitioned using VLANs to provide a separate
firewalled subnet for the home automation gear, some of which is cheap Chinese
stuff which needs to be forcibly prevented from talking to the cloud. The
networking gear consists of two TP-Link routers, flashed with OpenWRT which
provides nice VLAN support. These have been configured to just provide
switching and wireless access points and delegate all the firewalling, DNS and
DHCP stuff to the firewall.

Within the last year or so I’ve been working on streamlining the management of
all of this. The principle focus of this has been monitoring all the services
I’ve got running. For this I’ve settled on Nagios, which I run in a separate
VM hosted on the main home server. Although complex to set up, I can’t talk
highly enough of Nagios, it’s brilliant and it saves me so much time just by
knowing what is going on on my network. Email notifications from Nagios of
course go via my own mail server! I’ve also played around with collectd,
InfluxDB and Grafana for performance graphing, although I’ve yet to deploy
this to everything. Conclusion and The Future

So that was a probably non-exhaustive list of my self-hosting activities. I’m
sure I’ve probably forgotten many things and of course there are the huge
amounts of supporting software that I haven’t mentioned. As I said, I’m now at
the stage where this meets almost all my computing needs although there are a
few areas where I want to improve.

The main thing is automating and persisting my configuration, since I’m still
mostly doing things manually. For this I’ve settled on a combination of
Ansible and Docker. I’ve played extensively with both but haven’t really made
much progress with deploying them for much more than testing purposes.

I’m also constantly evaluating new software to fill gaps in my ecosystem. I’m
currently looking at Rocket.Chat and Hubot to provide a chat based interface
for remote administration, but don’t have a usable system yet. I’m also toying
with the idea of a Gitlab server to replace the gitolite+gitweb system and to
utilise the CI in my automation strategy, but I’ve heard it requires a bit in
terms of resources (incidently gitlab.com is really the only 3rd party service
I heavily use).

That I am able to do this at all is a testament to the power of Free and Open
Source software and cheap commodity hardware. I find it pretty awesome to
think that almost every interaction I have online utilises my own
infrastructure and that it works tirelessly for me 24/7.

I’m only just getting started documenting my setup here, for instance this
post hasn’t touched on any of the client applications I use on my phone and
desktop machines. I’m also going to do some more technical posts on various
aspects as time goes on, so please stay tuned (or even subscribe to the RSS
feed or mailing list!).

------
codeddesign
What to do when your self hosted life crashes? Get a new life?

------
notliketherest
Wait is this a joke? Like if you self host, you're stuff gonna break and you
have to deal with the maintenance headaches? ¯\\_(ツ)_/¯

------
nbevans
"Error establishing a database connection" Oh the irony :) Poor guy!

------
cameroncooper
"Error establishing a database connection"

~~~
memming
We did it once again!

------
bnchrch

      My Self-Hosted Life
      "Error establishing a database connection"
    

I think this may have been one of the more unintentionally ironic and
hilarious blog posts Ive seen in awhile.

While I run my own server for some personal projects I'd never advise people
to do it unless they absolutely have to. The maintenance, security updates and
embarrassing events like these for the most part are not worth it.

~~~
lucb1e
Just as many non-self hosted Wordpress installs go down. It's Wordpress that's
extremely slow.

Edit: Besides, the article mentions "the VPS is one thing and is really used
for critical services or stuff that needs to be accessible to the wider
Internet (like this site), but the real magic happens on my home servers".
That isn't the self-hosted part anyway. /end of edit.

> I'd never advise people to do it unless they absolutely have to.

I'd compare it to work. When doing a project for a client, all else being
equal, you'd rather have them pay a bit more so you have more time to develop
a great product. Rush jobs are never fun and you usually have to cut corners.
If you can make your code nice, it can be made flexible and neatly extendable.

Same for hosting: yeah it takes a little more time than getting shared
hosting, but it gives you so much freedom to install and configure your own
software. Save for perhaps mail servers, it's pretty much always worth it in
the long run if you can afford it in the short run.

