
Ask HN: Real-time log viewing - 87
The powerful solutions seem to focus on post-hoc log analysis. The thing I&#x27;m working on produces a lot of logs. Logs that I like to look at when debugging. Viewing them is a bit overwhelming.<p>I&#x27;d like a viewer that: 
-filters by JSON properties<p>-formats JSON logs into human readable messages<p>-can fold large messages<p>-lets you save sets of filters<p>-lets you simultaneously view the same set of messages using different filters<p>Right now I settled for splitting logs into different files based on filters. Which is okayish but doesn&#x27;t work well for teammates. And even at mediocre volumes Emacs will choke.
======
java-man
\- are these files local or remote?

\- do you need tail?

\- could you provide a link to a typical log file (with sensitive info removed
of course)?

~~~
87
They're local, the logging logic is completely under my control, I'm fine with
piping it to wherever as long as it does what I want.

For example the simplest JSON message would be:

    
    
      {"timestamp":"2018-06- 29T17:25:47.857Z",
      "module":"main",
      "msg":"[\"trying to reconnect\"]",
      "level":5}
    

while the human-readable representation for the same message is:

    
    
      global 17:06:067 trying to reconnect
    

So I'll have one main log file that gets everything in JSON, but that's only
useful for post-hoc analysis. I can control what makes it to stdout at runtime
but that's just a single view. So right now I've hard-coded the logging to
spit out human-readable messages to different files based on the module and
level properties. I open the files in auto-revert-tail-mode (emacs).

Not sure what you mean do I need tail? If you mean the tail program then yea
that's what I effectively do at the moment. If you mean always seeing the
latest logs as they come in - yes.

So the ideal software would just read that one file (or db) that has
everything in JSON (or machine readable format) and let you have different
"views" over the same data. Packaged in a nice (web?) GUI.

It shouldn't be too hard to implement but I'm hoping there's something out
there already.

~~~
java-man
There is probably something out already -

[https://softwarerecs.stackexchange.com/questions/37881/log-v...](https://softwarerecs.stackexchange.com/questions/37881/log-
viewer-with-json-support)

[https://github.com/gistia/json-log-viewer](https://github.com/gistia/json-
log-viewer)

[https://github.com/brocode/fblog](https://github.com/brocode/fblog)

I am currently writing a log viewer / terminal combo for internal use.
Handling milti-line json with tailing is a bit of a challenge though.

~~~
87
Oh I put the newlines on HN for readability, the actual logs don't have that.
There are numerous tools that handle json logs on the cli, for example there's
jq so you could

    
    
      tail -F my.log | jq <filter>

This is okay, but changing filters isn't trivial, running the commands and
displaying side-by-side isn't trivial etc etc.

So say you can develop a decent emacs workflow using that, but once you get a
new dev you cannot really pass it on. And if they're not much of a cli person
then yea...

Thanks for gistia/json-log-viewer I'll check it out, though it doesn't seem to
solve that much it's still interesting.

There's some nice Kibana stuff out there, but it seems single-view-at-a-time
focused too

