Hacker News new | past | comments | ask | show | jobs | submit login

I'm a huge fan of Splunk but always want to keep my eye open for alternatives. My use case is mostly security analytics against event content and patterns, and for that the Splunk Processing Language is very well suited.

That said I find it's fairly tedious to do a lot of time-series analysis and pattern discovery/anomaly detction across rich event models (think aws cloudtrail events).

Anything TimescaleDB can help with here? Are there case studies you can point us to? It feels like there is probably home for both just in my domain and quite obviously in the broader context of large enterprise ops/security.

Yes, we hear Splunk complaints quite often. :)

Here is a doc on using TimescaleDB as a horizontally-scalable, easy-to-deploy, operationally-mature data store for Prometheus data (i.e., metrics), put together by another of our engineering teams:

Building an open-source analytical platform for Prometheus


I'm also happy to discuss privately if you'd prefer - ajay (at) timescale.com.

A couple examples from Timescale users which might be relevant to your use case:

ShiftLeft - code analysis and security scanning to catch vulnerabilities [https://blog.shiftleft.io/time-series-at-shiftleft-e1f981969...]

k6 - a load testing tool that scales to 100k concurrent users, analyzes performance over time, etc. [https://www.timescale.com/case-studies/k6]

If you want to talk specific scenarios, you can reach out alex @ timescale or on Slack - slack.timescale.com.

I use TimescaleDB for mass storage and query of security events (up to 100s of millions) - the speed of queries and aggregate queries even on a single node is very impressive.

I haven't done anything with regards to anomaly/trend detection yet, but it's planned. Not really sure where you see a database (TimescaleDB) fitting into that though?

We're in that scale domain where everything is a pain in the ass but not obviously outside the scope of commercial solutions. I just checked and we're averaging ~500k events per second in the five areas I'm interested in.

I feel that we could probably use a time-series database to reflect our streams as 'last observed state' type collections as well as do the aggregations that we need to feed back into anomaly detection.

I'd like to also use something like that to create a 'heat map service' where you can feed a property/window/range and get back scalar for color coding and possibly a slice of values for sparkline type UI.

Without getting hands on, though, it's hard to say for sure.

@jcims I'm really interested to see if we can help. If you're open to discussing, please feel free to email me: ajay (at) timescale.com

It wouldn't be me reaching out but I'll put a bug in the right person's ear. This has been something I've been thinking about for a bit, the HN post is just a bit serendipitous.

Sounds good, thanks!

Hi GordonS, would love to hear about your use case if you don't mind sharing! ajay (at) timescale.com

What makes it tedious?

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact