

Show HN: My weekend project, realtime log viewer in Flask - llambda
http://logger.conceptis.org/

======
llambda
So if anyone's wondering how this works, it's actually quite simple thanks to
juggernaut. I used Flask to build the web app but because it's basically just
serving up a page that really wasn't necessary. You could actually write this
all with something like gevent, however Flask lends itself to extensibility
and you could certainly use this as a component of a larger app.

The primary functionality goes like this: Essentially I wrote a small script
to read off a log file, that runs concurrently as a greenlet via gevent.
Anything it reads is pushed to juggernaut which uses socketio to publish that
along a channel. The frontend reads off that channel, pushing in realtime.

I actually spent more time styling it than I did coding it. The VT terminal
style is very intentional and I'm pretty happy with the overall result of the
look (hey, CSS3 is nice!). I also included a couple of alternate color
schemes: blue and orange.

Hope you like it! :)

~~~
jemeshsu
I understand Flask can not do socketio because it is WSGI app. Any idea if
socketio will be supported by WSGI at all?

~~~
ak217
In addition to this, I recently discovered that Flask doesn't support
Transfer-Encoding: chunked (which is part of the HTTP/1.1 spec), and doesn't
fail gracefully either (when receiving a chunked request). Then I discussed it
with Armin and realized that the WSGI spec prevents Flask from doing either.
(See also <http://www.b-list.org/weblog/2009/aug/10/wsgi/>)

Time to fork WSGI?

ETA: Upon further investigation, there's a big discussion out there on what to
replace WSGI with, here's one entry point: [http://dirtsimple.org/2011/08/is-
wsgi-lite-library-or-protoc...](http://dirtsimple.org/2011/08/is-wsgi-lite-
library-or-protocol-and.html)

------
a_a_r_o_n
Cool. And not to take away from it all, but ...

You can do the same or better with multitail locally tailing a remote tail
over ssh, and you get multitail's color schemes for various log types like
mail or httpd.

    
    
        multitail -s 2 -sn 2,2 -C \
        -l "ssh -t xxx.yyy.zzz.123 tail -n 100 -f /var/log/nginx/a*" \
        -l "ssh -t xxx.yyy.zzz.123 tail -n 100 -f /var/log/nginx/e*" \
        -l "ssh -t xxx.yyy.zzz.123 tail -n 100 -f /var/log/uwsgi/*" \
        -l "ssh -t xxx.yyy.zzz.123 tail -n 100 -f /var/log/ufw.log"

~~~
bluemoon
That's pretty cool, hopefully we will have better support for multiple logs
soon

------
bflesch
What is really bugging me as a programmer: We live in the 21st century and
people still include user-supplied data in their pages as if Cross-Site
Scripting (XSS) is totally unknown.

I get a bad feeling when these simple attacks are possible; because this shows
that there are fundamental flaws in some people's coding styles. I know this
sounds harsh, and I recognize it is llamdba's weekend project, but these flaws
show neglection of fundamentals in handling data.

Apart from this, this live log is very entertaining!

~~~
llambda
You're absolutely right, I overlooked the fact that the log data wasn't being
escaped (something that I had realized in a previous chat application I had
made last week). A quick fix is to use flask's built-in escape function,
something like: line = escape(line). The server is now escaping lines off the
log.

------
pingswept
Thank you for the entertainment, unknown people of HN.

X.X.X.X - - [27/Dec/2011:00:26:07 +0000] "GET /hey-i-can-send-secret-messages
HTTP/1.1" 404 211 "-" "Mozilla/5.0 (X11; Linux i686) AppleWebKit/535.2 (KHTML,
like Gecko) Chrome/15.0.874.121 Safari/535.2"

Y.Y.Y.Y - - [27/Dec/2011:00:26:19 +0000] "GET
/I%20can%20recieve%20secret%20messages%20too HTTP/1.1" 404 211 "-"
"Mozilla/5.0 (Windows NT 6.1; WOW64; rv:8.0) Gecko/20100101 Firefox/8.0"

X.X.X.X - - [27/Dec/2011:00:26:25 +0000] "GET /hey-i-can-send-secret-
messages/maybe-al-qaeda-could-use-this HTTP/1.1" 404 211 "-" "Mozilla/5.0
(X11; Linux i686) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.874.121
Safari/535.2"

X.X.X.X - - [27/Dec/2011:00:26:50 +0000] "GET /ooh-you-can-recieve-them?
HTTP/1.1" 404 211 "-" "Mozilla/5.0 (X11; Linux i686) AppleWebKit/535.2 (KHTML,
like Gecko) Chrome/15.0.874.121 Safari/535.2"

Z.Z.Z.Z - - [27/Dec/2011:00:27:05 +0000] "GET /they+are+already+using+it
HTTP/1.1" 404 211 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:8.0)
Gecko/20100101 Firefox/8.0"

X.X.X.X - - [27/Dec/2011:00:27:34 +0000] "GET /hello-decatur! HTTP/1.1" 404
211 "-" "Mozilla/5.0 (X11; Linux i686) AppleWebKit/535.2 (KHTML, like Gecko)
Chrome/15.0.874.121 Safari/535.2"

Y.Y.Y.Y - - [27/Dec/2011:00:27:56 +0000] "GET
/this%20is%20alqaeda%20checking%20in...%20Hussain%20do%20you%20copy???
HTTP/1.1" 404 211 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:8.0)
Gecko/20100101 Firefox/8.0"

X.X.X.X - - [27/Dec/2011:00:28:15 +0000] "GET /200-hussain-found HTTP/1.1" 404
211 "-" "Mozilla/5.0 (X11; Linux i686) AppleWebKit/535.2 (KHTML, like Gecko)
Chrome/15.0.874.121 Safari/535.2"

~~~
FrankAbagnale
I coudn't respond to hussain :(

------
joshbaptiste
I can immediately envision some use cases for this RTM at the workplace, yet
our log file throughput is extremely high, so I would have to incorporate some
type of throttling mechanism, Great Project...

------
jakubw
Nice demo, thanks for enlightening me of juggernaut's existence! If I were to
be picky, it's hard to read anything because the scroll position moves to the
end even if you explicitly control it, which it shouldn't. Plus, there's this
reliability problem that between your Flask code that serves the last 100
lines and the realtime communication, you can miss some entries. And it's a
bit inefficient such as loading the whole log file just to grab the last 100
lines. Perhaps this comment should be on GitHub...

~~~
llambda
Yeah it's definitely a little rough around the edges. If you're interested,
I'd be more than happy to merge your suggested changes via a pull request on
GitHub.

------
pingswept
I was planning on writing something like this in the next few weeks for my
Flask app that runs on the Rascal (see profile for details), but now I won't
have to.

Thanks, Max.

~~~
llambda
You're most welcome. Please feel free to contact me if you have questions
about setting it up. Also if there's changes you'd like to make that would
benefit the application in general, I'll be happy to accept pull requests on
GitHub.

~~~
pingswept
Roger that. I'll let you know if I come up with anything good.

------
jiggity
It's poetic. It's like walking into a room filled with mirrors.

------
amix
Why is it that Flask is highlighted when juggernaut, node.js and redis do the
realtime communication? :-/

~~~
llambda
Because it's not built in a JS framework such as Node (it just uses those
things as a secondary service). Which up until recently was kind of a pain.
Juggernaut makes life easier and as a result there has been an increase in
realtime Flask applications which is pretty cool if you're a Python web dev.

------
septnuits
It's fun to watch. You can communicate with your browser agent.

------
FrankAbagnale
Response 500 :(. I was having fun chatting with GET requests..

------
marcuswestin
XSS By bflesch@bf-itservice.de :) nice work

~~~
godDLL
<https://github.com/maxcountryman/logmon/issues/1>

------
thehodge
Seems to have some XSS vulnerabilities

~~~
bluemoon
Yeah we had the data just return instead of escaping the value.

