

Ask HN: Monitoring historical server resource usage on linux? - apollo

I want to be able to find performance bottlenecks after usage spikes, so I would like a tool which monitors and logs cpu, memory, disk io over time on each of my servers. These are ubuntu servers.<p>If you have any other suggestions for tools I can use to help find bottlenecks, I'd appreciate that too; I'm new to this.
======
ezmobius
Collectd, I swear by it, running it on thousands of hosts without issues.

~~~
apollo
What do you use to generate graphs?

~~~
dryicerx
collectd can spit out rrd files, and there are tons of rrd graphers out there.

I use drraw... it's really old and boring, but I like it for it's simplistic
and I am used to it. (it's just a perl script you run on your web server, no
DB needed)

------
jbellis
There's a ton of projects, none of which is truly easy to deploy, all of which
kinda suck, and all of which store data as RRDs (which is the worst possible
solution except for all the others, as the saying goes).

This is an area that could use some new blood, but apparently the existing
solutions work barely well enough.

------
lg
You probably know about sar, but a lot of people may not; it can get advanced
with all the options, but for simple cpu spot checks:

    
    
      sar -u X Y
    

prints Y lines of utilization info, one every X seconds.

------
rv77ax
<http://www.solarisinternals.com/wiki/index.php/Dim_STAT>

<quote> _All STAT data are collected from standard Solaris or Linux programs
(vmstat, iostat, etc.) or some special (like psSTAT for users/processes
activity) and saved in MySQL database. Collected data are accessed via Web
interface and can be presented in several manner (interactive or static
graphs, text, HTML tables)._

 _dim_STAT can be used for On-Line monitoring one or several hosts on the same
time. As well, data may be easily post loaded from output files of stat
commands and analyzed in the same manner. At any time collecting from new stat
commands may be added to the tool (via Add-On interface) and enlarge your view
on application workload, RDBMS, your personal STAT program, etc._ </quote>

------
neondiet
Where I'm working at the moment we use Zabbix to monitor system availability
and collect performance and trend data for 184 servers [that generate 8701
monitored items of data]. The Zabbix folk do all their development on Ubuntu
and it's their preferred platform; but it works just as well on Redhat,
CentOS, etc and has client agents for a wide variety of platforms.

------
jws
I keep munin, <http://munin.projects.linpro.no/> installed on all my servers.
When things begin to look unwell, it lets you look back to see what resource,
temperature, voltage, or activity might have changed.

It is alleged to be easy to add your own data points to it as well.

~~~
JustRick
+1 for Munin. Very easy to install and deploy. There is a central collecting
server and each host being monitored runs a fairly lightweight agent. You can
write plugins in any language (bash, Perl, Ruby, PHP etc) to graph custom data
points. I prefer Munin over Cacti because Munin's config is in simple text-
based config files which can be scripted.

------
carl_
I use zabbix over ~<1000 physical and virtual hosts for historical resource
usage and alerting.

Cacti for network traffic monitoring (and alerting using thold plugin).

Smokeping for network latency and availability monitoring and alerting.

------
duskwuff
Ganglia is pretty good, although it's oriented more towards detecting problems
than diagnosing them post-mortem.

------
josephruscio
Are you looking only at free solutions? Or is proprietary software an option?

~~~
apollo
Free, but feel free to recommend proprietary solutions; I'm curious about
what's out there

~~~
josephruscio
The startup I work for sells a product called "Load Monitor" (I know, real
dynamic name) that provides application-centric, comprehensive monitoring
(including cross-host rollup) with both a real time dashboard and historical
access to collected data. Probably overkill if you only have a few servers,
but it's nice if you have a few 1000:
<http://www.librato.com/products/load_monitor>

------
ApolloRising
You can try Munin, MRTG, or CACTI

------
brianr
Cacti.

------
maelstrom
rrdtool

