Thank you, that's interesting. A small feedback about the website - it uses the "Space Mono" font for the installation snippet - it is a monospace font, but surprisingly, it collapses "fi" into a ligature, making the font subtly and weirdly non-monospace. This is wrong.
lnav is something I keep coming back to over and over really _wanting_ to like, but I've never managed to figure it out. I'm not sure exactly what the problem is, but I find the docs confusing and incomplete, and I always end up getting stuck and going back to Vim and/or VisiData.
Does anyone have any good tutorials or resources apart from the official ones?
I'm the author of lnav .. and not a very good writer, apologies.
I guess my main question would be, what are you expecting to get out of lnav? I use it primarily for merging log files together and just jumping around trying to understand what was happening. It has a bunch of other functionality, like using SQL for analysis, but that's not something I use regularly.
Really, a lot of the benefits of lnav are automatic, like uncompressing files, detecting log formats, tailing... So, if that's not something that comes up for you, it might just not be the tool for you.
I actually have this "not getting it" problem with VisiData/multitail. I start them up and they don't behave like I would expect when pressing hotkeys.
Thanks for writing lnav, it's a fantastic tool. I constantly take advantage of the automatic benefits you mentioned, and also love the ability to navigate a log minute by minute, interleave multiple logs within the same timeline, navigate through errors or warnings, and how easy it is to deal with spammy logs by using filter-in and filter-out. Thanks!
lnav is amazing and I use it often. I do have a list of gripes where I think it could be improved, so I'm just going to dump them here in case you're interested:
- regex101 support for quickly defining custom formats is just awesome. Versioning support is slightly broken however, probably because regex101 changed something, so there's no easy way to update the format once you've initially imported it.
- I feel like there's missing opportunity for integration between various features.
- There are lots of different filtering capabilities, but there is no unified treatment of them. For example, `:hide-lines-before` and `:filter-out` are at their core the same type of operation: filtering. I should be able to pull up a list of all filters that are currently active and easily add new ones and toggle or delete existing ones.
- I would expect to be able to create a new view of the data using SQL `SELECT`. A select statement is fundamentally about filtering out some rows (log lines), which feels like a filter, and selecting some particular columns (log fields) and hiding others. The latter point seems like it could be something that should be handled when https://github.com/tstack/lnav/issues/1274 is resolved.
> - regex101 support for quickly defining custom formats is just awesome. Versioning support is slightly broken however, probably because regex101 changed something, so there's no easy way to update the format once you've initially imported it.
There is a `pull` sub-command and it looks like it still works. Running the following will generate a patch file with the updated regex:
lnav -m format <format-name> regex std regex101 pull
It creates a patch file since the original file might've been modified.
> - There are lots of different filtering capabilities, but there is no unified treatment of them. For example, `:hide-lines-before` and `:filter-out` are at their core the same type of operation: filtering. I should be able to pull up a list of all filters that are currently active and easily add new ones and toggle or delete existing ones.
> - I would expect to be able to create a new view of the data using SQL `SELECT`. A select statement is fundamentally about filtering out some rows (log lines), which feels like a filter, and selecting some particular columns (log fields) and hiding others. The latter point seems like it could be something that should be handled when https://github.com/tstack/lnav/issues/1274 is resolved.
> I start them up and they don't behave like I would expect when pressing hotkeys
That's funny, because that's been my experience with lnav! Not saying there's anything wrong with it though.
It's been a while since last time I tried it so I don't recall the exact stumbling blocks I ran into, but I think it was mostly around hotkeys not doing what I expected, lnav not recognizing log types I think it should have (Apache/Tomcat), and not correctly loading custom log parsers.
If you don't mind next time I try it I can give you more concrete info.
> That's funny, because that's been my experience with lnav! Not saying there's anything wrong with it though.
I tried to use the hotkeys from less/more/vim so that it would be somewhat familiar. I think people are frequently tripped up if files are not recognized as a log and just treated as text. Files treated as plain text are separated from log files, so it can be a bit confusing. Not entirely sure how to improve the experience there.
> lnav not recognizing log types I think it should have (Apache/Tomcat)
There are quite a few log formats builtin. But, since log output formats can be customized by admins, it's possible they deviate from the builtin ones and things won't "just work".
> not correctly loading custom log parsers
I've tried to improve error messages a bunch[1] and make it easier to trouble shoot configuration issues[2]. I'm sure more could be done, I just don't quite know what folks are tripping over without feedback.
> If you don't mind next time I try it I can give you more concrete info.
Feel free to file github issues or email support@lnav.org
You're too modest. I wrote a custom format correctly on the first try; CouchDB. Keys are chosen rationally. People won't care about SQL until they realize that the later lnav versions allow `;select * from access_log where 0x00 in decode(log_body, 'base64')`
it's worth it just for the navigation helpers alone: '2' will bring you to the next 20-minute-after-the-hour entry (pressing again takes you to the next hour+20 minutes, same for the other 0-5 numbers). E will bring you to the next error, W to the next warning. And O will take you to the next entry with the same correlation ID (request/session/whatever .. you define it). And adding shift to all those keys reverses the direction. And naturally we're just scratching the surface.
I recently started appreciating lnav with JSON logs, where you can recreate a normal log-like experience by picking fields that are displayed (but you can still press a key to see all the JSON fields when you need them). I do wish there was support for switching formats so I could switch between different "views" over the same data, maybe it will be possible someday :)
lnav was a godsend with a particular project several years ago where we had a server bombarded with IOT messages and had to create some order from all the chaos. I actually went and donated some money to the project then, it really made my life easier.
> I do wish there was support for switching formats so I could switch between different "views" over the same data, maybe it will be possible someday :)
I tried it recently but got overwhelmed by its interface. I didn't understand all the colors and what I could do at each point. I'll retry later as it seems worth it as replacement of grep/sed/.... , but I'll have to reserve enought time to slowly go through it.
Would anyone have a recommendation for a log mining / analysis tool? I am particularly interested in automated identification of message sequences across multiple users, but anything beyond a simple log browser would be helpful. (I work with hundreds of GBs of Logcat logs.)
It's still just looking at one server right? If you have a fleet of webservers, or microservices in containers, ELK or similar seems like a requirement.
That doesn't seem .. wise. lnav has support for filtering, bookmarking, and attaching tags/comments[1] to log messages so that editing the log file isn't required. The filters, bookmarks, tags, and comments are saved separately so they can be restored when the file(s) are reopened.
I’ve made a Caddy server lnav configuration file a while a go, for who’s struggling to parse the default JSON logs produced by Caddy.
https://gist.github.com/vjanssens/3c6fb8393d87346323d939f172...