
Ask HN: Do we have a metrics store? - zrannie
For data scientists, we spent much of the time creating and optimizing features - thus, there&#x27;re multiple feature stores at tech companies to build on previous&#x2F;crowdsourcing experience.<p>When it comes to metrics design, especially with data coming from different areas like ads and content in a video, how to measure a &quot;success&quot; is no longer simple as CTR etc.<p>Ask: do we also have a metrics store to pick up brains from different data scientists&#x2F;companies? Would love to hear if there&#x27;s one.
======
valyala
Take a look at ClickHouse [1] and VictoriaMetrics [2]. Both solutions share
architecture details and are optimized for high performance and low resource
usage. They can handle trillons of rows (i.e. more than 10^12 rows) on a
single node and can scale to multiple nodes.

[1] [https://clickhouse.tech/](https://clickhouse.tech/)

[2]
[https://github.com/VictoriaMetrics/VictoriaMetrics](https://github.com/VictoriaMetrics/VictoriaMetrics)

------
gas9S9zw3P9c
There is Prometheus [1]. It has bindings for pretty much all languages and
integrates nicely with Grafana [2], which is used to plot Prometheus queries
onto nice-looking dashboards. I have been using Prometheus for many projects
and couldn't be happier with it. It's incredibly small, fast, reliable, and
memory-efficient. On one of my clusters, it has been 2 years without any
prometheus downtime.

Most commonly used infra services, e.g. databases like postgres, Minio, Kafka,
Docker, or k8s, ship with Prometheus metrics out of the box, so you don't need
to setup anything to monitor these. Just enable the metrics endpoint. It also
integrates with long-term storage (like timescaledb) for more advanced queries
on historical data.

Alternatively, there is InfluxDB. I haven't used it, so I can't speak to the
differences. These seem to be the two big ones, I'm sure there is a whole
range of smaller "startup-ish" competitors/projects as well. Splunk or other
log analysis systems are another option I see commonly used. They are not a
metrics store, but many companies seem to "abuse" their logs extract metrics.
IMO not a good idea.

[1] [https://prometheus.io/](https://prometheus.io/)

[2] [https://grafana.com/](https://grafana.com/)

