
Graylog2: Java, Ruby, MongoDB-powered log management, monitoring, and alerting - mattyb
http://thechangelog.com/post/3504643018/graylog2-server-java-ruby-mongodb-log-management
======
viraptor
It was fun to try out this software. It lacks some features in the interface
though... With some more automatic processing and UI improvements, it could
definitely be a low-end rival to splunk, but right now it can only do a
standard search on custom attributes or whole text (not with a proper full-
text index though) or show stuff "around that message". That means basics
only.

But with the development happening all the time, it's definitely a project
worth keeping in bookmarks. If it can get some of the features available in
octopussy without looking just as fugly, I'm in :)

------
kordless
Be sure to check out LogStash as well. Uses a similar approach with MongoDB,
plus ElasticSearch for...search: <http://code.google.com/p/logstash/>

------
devinfoley
This looks great! Is anybody using it in production though? I can't find any
case studies on their site.

~~~
aedocw
We are using it and loving it. More important than just shipping syslog to
graylog2, we're using it to monitor 30+ servers in a cluster. Each server has
a "health agent" that sends all the important metrics that happened in the
last minute, packed into a GELF message. This could easily scale to 1000+
servers. On the monitoring side, one server dips into MongoDB to pull out the
records and maintain records and graphs of everything that's going on
(including sending alerts when a machine fails to check in as frequently as it
should.)

Graylog2 is pretty solid, and the people working on it are extremely
responsive. If you suggest features that make sense, they'll probably be
implemented within a few weeks (if not a few days!)

~~~
kordless
Let me just caution those with highly scalable expectations. A large MongoDB
instance is going to weight in at around 1-10TB. Here's an updated list of
production MongoDB instances for reference:
<http://www.mongodb.org/display/DOCS/Production+Deployments>

At both Splunk and Loggly I've seen customers sending in multi-tens-of-
millions events a day from a handful of boxes without even breaking a sweat.
It wouldn't take long to blow through several TB of storage in MongoDB if you
were storing all that log data with a retention of a few months.

Log volumes vary from use-case to use-case though, and I absolutely love the
GrayLog2 guys and the way they listen to their users. It's definitely a great
tool for a job that is usually a real pain in the ass!

