
Ask HN: Splunk Alternatives? - bhattchaitanya
At our company we love Splunk for its amazing query capabilities, dashboards, rich set of APIs, speed etc. 
Are there any credible competitors for this tool? We are open to both proprietary or opensource. 
Splunk is too expensive and is not cost-effective for our business and we are desperately looking for an alternative. any guidance would help.
======
luizfelberti
I'll give a non-orthodox suggestion: ClickHouse

You'll need to manage some stuff yourself, and assemble your own dashboards
and stuff, so there will be some labor involved. That being said, I doubt it
will be more painful than managing an ELK stack: there are just too many ways
you can destabilize a cluster with it.

ClickHouse clusters from my experience are ridiculously scalable, fast, and
stable. There are several other accounts to back that up, and a good case
study is Cloudflare, which uses it to store and query all of their logs and
metrics from all data centers (that's quite a few PB of data).

There are some projects on GitHub you can use to get inspired, but what you
need is pretty much a ClickHouse cluster, Grafana, and a Log Shipper.

------
kodyo
Elasticsearch (well, the whole ELK stack)

[https://www.elastic.co/](https://www.elastic.co/)

Depending on your volume and situation, there are hosted options or you can
roll your own on-prem.

~~~
sporkland
+1 on elastic search. We swapped splunk out for es + some inhouse pipelines
and it's been a decent replacement. It's not 100% the same. We can't throw
random garbage logs at it and create structure, we pushed structured logging
onto service owners, but it's covered a lot of our common use cases e.g. what
happened with this request?

We haven't moved off of splunk for all of our logs, but have reduced the
volume going there significantly.

------
ram_rar
At my company, we used to pay 7 figures to splunk, until we realized how much
data do we actually need. We cut down a lot of unnecessary logging and sent
only the things that were needed.

Most of the data that was needed by business folks was sent to segment in
terms of events and for genuine errors were sent to sentry with stack traces
and converted most of the logs to metrics.

This helped us cut down a lot of $$ for logging and monitoring. Elastic search
is a great alternative, but it takes an entire team to maintain it. I'm not
sure, if its worth the time and effort.

------
pych
It'll really depend on your use-case. If you're not doing _too much_ volume
(less than 1-2TB) and you're willing to put in the work, ELK stack (as others
mentioned) will be more cost-effective from a software cost perspective.
You'll end up making up those costs in person-hours though, as you'll lose the
benefits of managed/SaaS.

If you'd rather not spend the time managing an ELK stack, there's a lot of
logging options (disclaimer: I work at one of them,
[https://logdna.com](https://logdna.com)). It'd be helpful to unpack what you
mean by "credible" competitors however. Our product, and most others in our
space frankly, can't match Splunk in terms of feature-set today, so knowing
what is most important to you would be helpful. For example, if you're looking
for basic log storage and search, you'll have tons of options. If you need
compliance, that would narrow the field a bit.

I'm happy to chat if you're interested in giving us (LogDNA) a shot or if you
have other general questions, feel free to shoot me an email, I'm just
peter@[ourcompanydomain]. I'm not in sales so I don't really have any
incentive to push you towards us unless it makes sense, and a lot of our
competitors have great products as well so happy to try and point you in the
right direction.

~~~
pych
Quick follow-up on my last comment, if you're interested in trying us out, a
few highlights: \- don't need stress about structuring logs beforehand, we'll
parse common log types automatically, parse JSON automatically, and you can
create custom parsers after the fact \- it takes two kubectl command to dump
all your kubernetes logs onto us, and we'll add metadata after the fact like
pod name, container name, namespace, etc (we also have a few dozen other
integrations/ingestion options, of varying levels of quality/support) \- we're
responsive with customer feedback, and love to talk to customers about how we
can make our product better

Hopefully that's helpful information!

------
mindcrash
Graylog: [https://www.graylog.org/](https://www.graylog.org/)

It can also integrate in your existing Splunk setup if you want:
[https://www.graylog.org/post/graylog-splunk-integration-
is-n...](https://www.graylog.org/post/graylog-splunk-integration-is-now-here)

(which allows you to ship data to Graylog and/or Splunk and setup analytics in
both as needed)

------
castillar76
It's important to consider whether your requirements include long-term
retention and search of data, or only recent search and dashboards. Many of
the open-source solutions such as Graylog-free or ELK don't retain information
long-term by default, so if long-term data retention is your goal, you should
take that into account (Graylog offers it, but only for $$$).

On the other hand, if your primary use-case is near-term searches and reviews
of data (e.g. "we just need to see the last 90 days of information for
troubleshooting and stats"), you'll be pleasantly surprised with the
capabilities in both Graylog and ELK without the additional overhead of
Splunk. I'll say, too, that I found working with Filebeat for ingestion to be
a lot easier and automation-friendly than working with Splunk forwarders—so
much so that we're basically adopting an "only use Splunk for indexing"
approach to ingestion and using everything else to get data into it.

------
stuntkite
I had to go through this last year with my previous company. Splunk's
licensing is outrageous. They cap the data transfer on a PAID license
arbitrarily. Even getting coordinated with their sales to just pay them was a
chore. They do nothing to foster open development on their platform. IMO, the
open offerings actually seem designed to make it impossible to
improve/contribute/extend in a cost effective way. It is also ill equipped to
deal with Kubernetes even though they did release a reasonable if Windows Guy
bloated example of a k8s deployment at the beginning of this year. I was
completely baffled when I found out it was a Django project. It feels like
something from the depths of enterprise Java.

Elasticsearch (or ELK/EFK) is really eating their lunch but if you have any
experience managing an Elasticsearch cluster at scale you might have some
reservations about it. If you'd like to audit it I might suggest using a Helm
chart like this one[0] to deploy a full stack to a Kubernetes cluster. Even if
you aren't using k8s for production it's a fast way to get a handle on it. You
can deploy a test cluster easy on GKE quick for nearly no money. If that seems
too inaccessible, there are a few good docker-compose[1] implementations too
that can get you going right away.

I don't think it's quite ready for production, but we were already using
Prometheus and Grafana so I was auditing Loki[2] and was pleasantly surprised.
Though at the time I determined it would not be a substitute at scale for
Splunk or ELK but could be viable for many people and the project is moving
fast to do cool stuff.

[0] [https://github.com/helm/charts/tree/master/stable/elastic-
st...](https://github.com/helm/charts/tree/master/stable/elastic-stack)

[1] [https://github.com/deviantony/docker-
elk](https://github.com/deviantony/docker-elk)

[2] [https://grafana.com/oss/loki](https://grafana.com/oss/loki)

~~~
trm42
In my previous job Splunk payments were ok enough not to warrant change but I
spent couple of days trying to figure how to renew the license and pay for it.
Their license portal was so horroble and full of funnel loops that it was next
to impossible to renew the license.

------
YuriGrinshteyn
I work at Google Cloud, and this is a common request. If you're on GCP, you
can build your own using logging exports and BigQuery. Otherwise, I typically
recommend Elastic.

------
swengw
Scalyr is great, but I'm not sure if it's any cheaper.
[https://www.scalyr.com/](https://www.scalyr.com/)

------
pranaygp
Depending on what you're looking for, you might not want to spend a lot of
time maintaining your own stack for dashboard.

For really good user/account dashboards, which is the most common first tool
companies build, you check try windsor.io which does most things out of the
box with no setup

If you're looking to build custom dashboard and are okay with spending time
actually coding and maintaining it (just code snippets, not a full ELK stack)
try retool.com

~~~
bhattchaitanya
Thanks I will checkout windsor and retool

------
mamcx
I work in the past on a big virtual ecommerce company and their solution was
to move all their nosql(primary store)/log data to postgresql for reporting
and analysis.

I think that is smart.

P.D: Better to not use nosql but well marketing of nosql back in the days...

p.D2: Most logs are pure NOISE. What I wish to have in some ways to reduce
them when incoming and the distilled put in a rdbms. I think this is the best
for most but not see much info about this.?

------
serinjune
No one mentioned GrayLog. They're a relatively new player in the game, but
they show great promise. I should mention that their dashboards are probably
not as mature as Splunk and some of the other platforms, but I've met a few
people who use it and they seem happy!
[https://www.graylog.org/](https://www.graylog.org/)

~~~
hwj
I was using Graylog but I wasn't happy because it took ~8GB RAM and ~20% CPU
(IIRC).

------
garganshum
I work at Datadog and we've been pretty successful at attracting customers
over from Splunk depending upon the use case. Others in industry are
SumoLogic, ELK etc. You can check us out here if you want
[https://www.datadoghq.com/](https://www.datadoghq.com/)

~~~
bhattchaitanya
Thanks ! Do you guys sell logging separately or should we buy both metrics and
logs to use your platform ?

------
solatic
I work at Coralogix [https://coralogix.com](https://coralogix.com)

We're a relatively new, relatively small player in this space, differentiating
by real-time anomaly detection and tooling to help our customers keep costs
low (by filtering out logs known to be irrelevant).

------
alexandercrohde
I work DevOps at a 60 person company. I wanted Splunk very badly, but due to
pricing had to compromise.

We got SumoLogic. I'm very happy with it relative to ELK.

It's almost as good and much cheaper. The limitations would be that dashboards
aren't as customizable and that some advance searches are harder.

~~~
bhattchaitanya
Thanks for your input!! Is sumo Logic pure Saas or hosted ? How do they scale
at load ?

~~~
alexandercrohde
They host it for you. My experience has been that they scale fine.

I don't know about your use case specifically, but for our case (~100GB of
searchable data at any given moment), searching takes a few seconds for basic
queries.

------
jonahbenton
Elastic has been getting a lot of business from priced out Splunk users.

------
Fnoord
AlienVault, or its FOSS version, OSSIM. [1]

[1]
[https://www.alienvault.com/products/ossim](https://www.alienvault.com/products/ossim)

~~~
twunde
Alienvault (now AT&T Cybersecurity) is really focused on the SIEM space and is
only ok at that. While it has a lot of features, it's not very flexible. For
example, neither the vulnerability scanner nor the compliance scanner can send
out alerts upon newly found vulnerabilities/compliance issues. My feeling is
that if you're used to using most of Splunk's capabilities, Alienvault will be
disappointing

------
aspleenic
Full disclosure: I work at Humio. Great, cost-efficient monitoring, sub-second
ingest latency, on-prem or in the cloud. Feel free to check it out - humio.com

Runs smoother than the ELK stack.

------
somedanishguy
I’d have to mention humio.com They’re mainly just for logs, but I really love
their query style.

~~~
rafaelgarrido
We recently migrated from splunk to humio at work and not only their query is
easier to work with but creating dashboards and reports is simpler and faster.
Also, their UI is extremely responsive. Kudos for humio.

------
not_a_cop75
Elastic which used to be called Elasticsearch. There is a free community tier
of this, though according to recent drama, not sure how much longer it will
remain open as such.

~~~
not_a_cop75
Wow....generous person. Thanks for the downvote. I have an identical answer to
like 1/5 of the people here. I guess even popularity is a great reason to
censor!

