Hacker News new | past | comments | ask | show | jobs | submit login

I'm Mike Schroll, CTO of DNSFilter - Happy to answer any questions about our experiences with InfluxDB or TimescaleDB over the last 2 years.



When you talked about 150M queries per day; are those inserts or reads? That's about 1,700 per second, which to be honest doesn't seem like a lot for time series metrics ingest. I would expect a single node on most TSDBs to be at least ~100 times that performance.

Can you talk about the data you were ingesting? Was it numerics, text, or something else?

(Disclosure, I work for TSDB provider IRONdb http://irondb.io)


Hi there -- We are currently ingesting 150M/day -- though it's not evenly distributed -- probably peaks around 3,000qps

Agreed that it's not too much yet. I think the element which kills us is needing to do rollup tables, and then query against both the raw data and rollup data for our customer analytics dashboard.

The data is a combination of date/time, strings, and ints. 22 fields.


I know I mentioned this in another comment, but you really should check out Clickhouse. They have a table engine for exactly this purpose.

You create a table with the raw logs, then a materialized view (or another actual table) which declaratively does the rollup for you in real time.

https://clickhouse.yandex/docs/en/table_engines/aggregatingm...

A full example: https://www.altinity.com/blog/2017/7/10/clickhouse-aggregate...




Join us for AI Startup School this June 16-17 in San Francisco!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: