To piggy back on the question, has anyone had a good experience using Prometheus and Grafana for monitoring? I'm looking into trying it. I've looked into Zenoss but from what I gather it's slow.
I love it. Coupled with Alertmanager it's a really easy to use and powerful platform. We also started serving custom Prometheus exporters for our product usage which has been helpful and really easy to implement.
To me, this solution is too much leaning toward graphing and that's not the primary feature you need to health check services.
Also Prometheus only saves 15 days worth of data by default and since they aren't willing to develop down sampling, it's not good to keep any long term history of the metric but you can overcome this by using InfluxDB as the storage but I would rather use something like monit which is simpler and easier to setup and gets the actual job done well.
we <3 prometheus & Grafana :) Mostly because it gets us the classic monitoring of hosts/services like nagios, etc but also gets us app-specific metrics as well.
We are using it and it's great, so easy to also plug in your own metrics from whatever application you're running. I insisted with the management that we needed it on a new project and they were like 'fine if you insist, we do have Google analytics and uptime Robot you know', now they call me as soon as it's down (we installed it on a crappy server because it was just my fixation :) )
Another happy user here. Only we use Telegraf to gather the metrics on the servers, I've found that it's a bit more flexible than the normal node exporter.
For alerts we have a mix of grafana and AlerManager.
Yes, of course, Its a great tool to track statistics and write alerts. Prometheus is designed to scrape the stats from the services directly, but its also possible to do some kind of active checks. Maybe for checking the ssl cert expiry date you can use the blackbox exporter. https://github.com/prometheus/blackbox_exporter
