

scales: Greplin's new open source Python server metrics library - rwalker
https://github.com/Greplin/scales

======
rarrrrrr
We open sourced something similar called StatGrabber awhile back, including
Perl and Python client libraries. We tend to avoid threads, and so it instead
uses non blocking UDP (guaranteed delivery on localhost) to a collector
daemon, which aggregates and then delivers the info to Ganglia for graphing.
<https://spideroak.com/code> if anyone is curious.

The client modules simply emit non-blocking UDP packets and get on with their
business, avoiding slowing down their response time. You can graph 4 types of
stats: counters (ex: transactions) averages (ex: size of transactions),
accumulators (ex: bandwidth used) and elapsed time (ex: time per transaction)

It's pretty nice to see Ganglia graphing system metrics along side of all the
stats we have the backend emit. For example, one of the stats graphed are
revenue events. There's clear relationship between network health issues and
revenue.

~~~
gnubardt
We wrote something similar at Brightcove to collect system statistics and
publish them graphite. It runs as an independent service and focuses on OS
(not application) level metrics.

<https://github.com/BrightcoveOS/Diamond>

~~~
pjscott
That looks really convenient and useful.

------
mattlong
Correct me if I'm mistaken, but it seems like this wouldn't really work with a
web server that has several worker processes like uwsgi in preforking mode.
Each worker process would be sandboxed to its own STATS object and attempting
to serve the HTTP/Graphite server in its own background thread; all of which
would be trying to access the same port.

~~~
pjscott
Correct. This is intended more for daemons with one or two processes per
server, or for handy debugging of a single instance on a development machine.
In order to monitor something like preforking uwsgi, you would need a way to
aggregate stats from all the processes, which this library doesn't do.

(You can change the port the HTTP server listens on, though. If you have ten
workers, you can have them listen on ten different ports.)

