
Show HN: Log.io (Realtime log monitoring, powered by node.js + socket.io) - msmathers
http://logio.org/
======
viraptor
> Log.io has no persistence layer. Harvesters are informed of file changes via
> inotify, and log messages hop from harvester to server to web client via
> socket.io.

This struck me as really odd. Why is it done that way? Most stuff is logged
via syslog. Those things are already in the buffer and it seems the local
nodes are doubling the seek+read of what was just written. Downsides:

\- double disk action for every log line

\- inotify overhead for the system

\- logging requires local disk space

\- disk io delays mean collection delays

\- when disk get full / fail on write you don't collect the message about the
failure if one was attempted to be written

\- when syslog daemon decides that messages are coming in too fast, it might
drop them from the for_the_disk buffers - this way the only copy was lost (not
100% sure, but I believe syslog-ng can send to one destination even if another
is blocked)

I really can't see positive sides of this approach. Logstash seems to handle
everything log.io does and much more... am I missing something?

~~~
guan
A lot of things don’t use syslog. For example, default configurations of
Apache write logs directly without syslog. The benefit of this method is that
it is general enough to work for most people without configuration changes,
and I assume they could always add syslog support later.

~~~
viraptor
Log.io assumes multiple nodes. If you want any kind of visit statistics in
that scenario, you're already collecting the logs somehow. Unless you rolled
out some homemade script for that purpose, you're probably using remote syslog
anyways.

Or do you store your logs on a random server that request happened to hit /
session was bound to?

------
joshklein
I know very little about system administration, but the comment I want to add
to this conversation is: wow, this is a tremendously good product website.

1\. Show the glamour shot (even just a screen shot of text makes it
compelling). 2\. Quickly explain the problem you're solving. 3\. Quickly
explain how you solve the problem. 4\. List the most compelling RTBs ("reasons
to believe")

Just a heads up: it wasn't immediately clear that I could click on the
screenshot for a demo. I'd make that a huge call-to-action. The product can
sell itself.

------
xal
Also realize that if you are connected over websockets all data will be
transferred unencrypted even if you put log.io behind https. A much better fit
for these cases is SSE which works fine over https:

<http://www.html5rocks.com/tutorials/eventsource/basics/>

~~~
drivebyacct2
How have I never heard of this before? (Oh, no Firefox or IE support).

Also, I don't believe your claims about WebSockets are true. I thought wss://
would allow for SSL secured WebSocket connections...

edit: Googling indeed seems to indicate that wss is ws over TLS. See:
<http://websocket.org/echo.html> for an example.

~~~
Sidnicious
Modern HTTPS uses TLS too.

------
alex1
It's cool to see log files streamed in real time to your browser but most of
the time it scrolls too fast to get any useful information out of it (unless
you use regex filters of course). It would be cool to have graphs that show,
say, browser usage (or IP geolocation or operating system or page visited or
any other metric that can be gathered from log files), being updated in real
time.

~~~
xtacy
This is probably what you described: <https://github.com/stagas/maptail>

------
panarky
I like this, and I'd use it in my projects if I knew it was reliable.

I've seen socket.io drop messages and I wonder if it's happening here.

Open both Chrome and Firefox at the same time and watch logio_server in each
browser. After several minutes I start seeing different messages in each
browser. Is socket.io not delivering every message to each socket?

~~~
Rauchg
[author of socket.io]

It's possible that log.io is not trying to send absolutely every message
(which is the reason I introduced the volatile flag for the upcoming version:
<https://github.com/learnboost/socket.io-node/tree/develop>)

------
skrebbel
Oh man, and I just coded essentially the same (with persistence through, for
we need history) with CouchDB over the last 2 months. If only you'd have
launched a bit sooner!

ps. Maybe it's me, but 2 hours ago your site worked great in Opera, and now
the design seems broken. If you changed some CSS, then that might be it :)

------
coenhyde
You know what would be cool. Write your own npm that does logging which would
open up a socket connection to your server bypassing sending logs to disk and
being watched by inotify.

~~~
tedjdziuba
Yeah that would be cool. Even better if it could be done over UDP, so the log
messages are fire-and-forget from the client side. Even BETTER if there was a
standardized way for all programs regardless of implementation language to
write to said socket/log.

Yeah, they thought that up 30 years ago. It's called syslog.

~~~
viraptor
There are some issues with syslog. Fortunately an extension is ready:
<http://www.graylog2.org/about/gelf> \- can be structured and is not limited
in size anymore.

~~~
iloveponies
Except my Cisco switches, HP printers etc won't be talking that anytime soon.

In addition, graylog appears to be "solving" something RFC 5424 already has
with structured messages.

~~~
viraptor
Tbh, I haven't seen any standard syslog daemon supporting structured messages
- except for apple's syslogd and syslog-ng. Others mostly cut the message to
1024 or so bytes and each one handles new-lines in a different way. So far it
seems noone is sending messages that way and noone is expecting them. If a new
protocol is the way to push structured logging forwards, I'm all for it. It
doesn't mean those two protocols cannot coexist. It doesn't mean a typical
logging pipeline cannot include syslog->gelf converter which adds annotations
/ parses the message.

It also seems that the structured messages handling assumes the message fits
in a packet and can be processed dropped when convenient depending on its
size. That's not something I expect from a logging system. Gelf adds chunking
/ compression just in case.

~~~
wladimir
rsyslog also supports structured RFC5424 messages. Others will support it in
time.

I agree it's not the most perfect format, but it's usable, and already much
better than freeform text messages if you intend to do programmatic
processing.

Another interesting initiative in normalized logging is CEE
(<http://cee.mitre.org/>). The 'lognorm' and 'libee' libraries
(<http://www.liblognorm.com/>) are early attempts to implement this.

Note that I haven't looked at 'gelf' yet so I don't know how it compares.

------
sciurus
This looks similar to logstash, which has been on my tools to try list for a
while.

"logstash is a tool for managing events and logs. You can use it to collect
logs, parse them, and store them for later use (like, for searching). Speaking
of searching, logstash comes with a web interface for searching and drilling
into all of your logs. It is fully free and fully open source."

<http://logstash.net/>

------
franze
very, very cool.

node.js is also my (current) tool of choice for real time logfile analysis, i
use a slightly different approach.

i ssh funnel the logfiles to my machine i.e.:

    
    
       ssh server.example.com sudo tail -f /var/log/nginx/example.com_access.log > test.log
    

then use this ( _disclaimer: coded it during a hackweek_ ) lib
<https://github.com/franzenzenhofer/nolog> to attach events to the logfile

    
    
       nolog('./test.log').shoutIf('googlebot', /Googlebot/i ).on('googlebot', function(data) { ... });
    

basically you

    
    
       - watch a logfile
       - shoutIf you come across a given pattern (can be regex or function)
       - listen if a shout event occurs
    

i will definitly look into log.io and see if there is a way to plug nolog
somewhere in there.

an important addition to log.io would be log servers in php, asp(x) and all
the other legacy languages. my experience is that webservices that allow
and/or use modern development approches already have a suitable logfile
observation, but all the millions of sites stuck with yesteryear really
struggle with it.

------
prule
vsConsole is a similar application that solves the same problem, but based on
java technologies.

* Java Agents run on the servers * vsConsole is a web application that runs on Tomcat

Use a browser to select the desired log file, and you can tail it. (When the
browser polls the web app, the web app will contact the agent for the latest
changes to the file).

View the demo at:

* [http://demo.vamonossoftware.com/vsconsole/file/index#1:DEV:8...](http://demo.vamonossoftware.com/vsconsole/file/index#1:DEV:8081:/tmp/example.log)

More information at * <http://demo.vamonossoftware.com/>

vsConsole is aimed at development teams, not as a production log monitor. For
example, while developers and testers are working with an application, when
they want to see whats going on in the log, they can just click a button - no
unix accounts, ssh, tailing, needing to know where the logs are etc.

A new version which looks a lot better, and has simple messaging and
application monitoring will be out soon.

------
ericflo
I built something similar a while back during the Django Dash, but there's no
server software to install because it just uses ssh and tail:
<https://github.com/ericflo/servertail>

This looks more polished and complete though from a user interface
perspective, nice job!

------
ianl
This is really cool; very impressed!

------
robinduckett
Cool, but doesn't run on CentOS. Init files are coded exclusively for Debian,
and I don't have time to convert them.

------
oomkiller
Couldn't get it to start in Ubuntu (couldn't find module "connect", totally
unsupported on Centos.

~~~
te_chris
did you use npm for packages? did you install connect?

~~~
oomkiller
Yep.

------
marcamillion
This would be awesome as a plugin/add-on for Heroku - if that's not already in
the works.

------
noinput
nice, however unless you left it out deliberately to fire the error log, you
should add a favicon to your site.

~~~
whimsy
Could you explain your concern? I don't understand why a favicon is important
in this case.

~~~
tptacek
Look at the app; favico 404's are the entire error log.

------
mmaunder
ssh user@server tail -f logfile

It's even encrypted.

~~~
viraptor
Try that with N servers and M log files where nodes join / leave the cluster
at any time.

~~~
cdavid
As long as N is not too big, remote syslog works well for this case.

------
jschrf
Very cool!

------
tedjdziuba
Cool, you just reinvented syslog, poorly.

~~~
e1ven
Syslog is cool and all, but this is a nice UI for displaying the logs in the
browser, so everyone in the company can watch them, or you can easily open it
up on an iPad, and hang it on the wall.

Could I use SSH over iTerm, and display multiple windows in Screen? Sure. But
this is easier/quicker.

It would be nice if it didn't need to constantly hit local disk on every
machine, though.

~~~
viraptor
What is a usecase for everyone in the company to see raw logs? Nagios / zabbix
dashboard - sure. Some visits analytics - maybe. General stats / resource
usage - not for everyone, but ok.

But why would you ever inflict raw, unprocessed, scrolling logs on anyone
other than developers working on a specific bug? (they'd expect an accessible
backwards search too)

~~~
e1ven
I was thinking it would be fun, in a somewhat similar vein to Google
displaying Search Queries on the wall -
<http://www.flickr.com/photos/kchrist/21051526/>

That said, I agree- Historical Search is a necessary feature for it to be very
useful to debugging.

