
Centralising logs with rsyslog and parsing them with Graylog extractors - babolivier
https://brendan.abolivier.bzh/logs-rsyslog-graylog/
======
dozzie
Parsing this way (in Graylog) is so very... manual. I'd rather go with
rsyslog's liblognorm, either in rsyslog itself or in a separate daemon, like
logdevourer I wrote. The rules then can be written in a regular text editor
and versioned.

Also: what happens if there is a typo in one of the rules and a log entry
doesn't get parsed? Does Graylog/grok have some indication that the log
doesn't match? And where the mismatch starts? Because liblognorm does.

~~~
babolivier
I'm not saying that's the best way to do it, just that it's how we did it back
then. Of course there might be better ways to do it. I was in charge of
exploring this way (which we didn't end up keeping, as I hint at at the end of
the post), and thought it would be valuable to share what I learned with
people.

Regarding your latter point, as I mentionned, a pattern fails silently if it
doesn't match a log entry while parsing incoming entries. That's why Graylog
requires you to load a log entry example when creating an extractor, and let
you try it while defining the pattern (with the "Try" button next to the
pattern input field). I didn't put enough emphasis on this feature in my post,
though.

~~~
dozzie
> That's why Graylog requires you to load a log entry example when creating an
> extractor

So it still fails silently when the log format changes slightly, e.g. upstream
fixes spelling or grammar in one of the log messages produced.

~~~
babolivier
Yep. You might be able to configure Graylog to alert you when an extractor
fails, but I didn't go this far (as I said, we switched to another solution
since then).

