Hacker News new | past | comments | ask | show | jobs | submit login

Free multi-node TSDB solution sound cool! I wonder if someone tried to use TimescaleDB as remote-storage for some heavy-loaded Prometheus [1] setups.

[1] https://prometheus.io/


We wrote one of the first remote backends to Prometheus that supports both the remote_read/remote_write interface: https://prometheus.io/docs/operating/integrations/#remote-en...

Given how much interest we had using TimescaleDB for this, we recently built and released (in beta) a new "full-stack" of Prometheus + TimescaleDB + Grafana that comes fully configured and "just works" out-of-the-box:

- https://tsdb.co/prom-design-doc

- https://github.com/timescale/timescale-prometheus

This is so nice that design doc is opened for commenting - so much good thoughts there. Thank you for sharing!

After reading that I have a two questions:

1. While the integration with Prometheus sounds great it still requires to run pretty complicated system behind it. The distributed TimescaleDB could require a lot of knowledge to operate and additionally a connector that could become a one more point of failure. Have you considered to merge connector into Timescale to make setup more simple and robust?

2. Significant part of my everyday work is connected with writing PromQL queries and I often check week/month ranges while plotting timeseries. And I heard many complains that remote-read might be very expensive when it touches a lot of data. Do you consider possibility to support PromQL in TimescaleDB to avoid remote-read bottleneck?

Personally, I have a good experience working with Thanos and VictoriaMetrics because of seamless usage experience - same queries, same Grafana dashboards, same alert rules. Would love to see more products that support the same standards for timeseries data.

Edited: formatting.

1. Even though it is newer, distributed TimescaleDB is probably more robust and easier to operate (and already more operationally mature) than other local storage options for Prometheus metrics, in part thanks to the underlying maturity of Postgres.

2. Yes, supporting PromQL directly (ie not via remote_read) is already in internal testing. Coming very soon.

Would really appreciate feedback if/when you get to try it out yourself. Please feel free to ping me directly: ajay (at) timescale.com

I've had luck with https://thanos.io/ for a big (~1 billion timeseries across all our DCs) Prometheus scale out project. Horizontally sharded Prometheus that can be queried and alerted on in a unified view with object store backend.

I remember being very impressed with numbers from the following tweet https://twitter.com/this_is_tckb/status/1256649880434606080.

I'm wondering what is the cost of your setup to handle billions of timeseries?

Do you consider TimescaleDB as replacement for Thanos? Would be nice to read some operational and performance comparisons for real world cases.

You may also want to checkout https://eng.uber.com/m3 which is a highly available RF=3 multi-node TSDB metrics backend and is used with heavy Prometheus workloads and is used to ingest tens of millions of timeseries per second.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact