eHour development blog

Reading Dmitri Maximovich’s blog posting about plotting JVM memory into MRTG graphs had to trigger my interest as I’m a statistics addict. Earlier on I tried to get JVM stats using SNMP but it took me more than 30 minutes to get any data so I dropped it. The interesting thing about Dmitri’s method is that it uses jstat which is far more easier, read his posting for more details.

Anyway, I’m not using MRTG anymore but RRDTool. RRDTool is a lot more flexible and graphs aren’t created every 5 minutes but whenever you request them – at least with rrdcgi.

To get JVM data into RRD is easy. First create a RRD database with 6 datasources; a capacity and utilization datasource for eden, old & permanent size. The following script creates a heapsize.rrd which is the actual database.

[code lang="bash"]
/usr/bin/rrdtool create \
heapsize.rrd \
--start N \
--step 300 \
DS:edencap:GAUGE:600:0:10000000 \
DS:edenut:GAUGE:600:0:10000000 \
DS:oldcap:GAUGE:600:0:10000000 \
DS:oldut:GAUGE:600:0:10000000 \
DS:permcap:GAUGE:600:0:10000000 \
DS:permut:GAUGE:600:0:10000000 \
RRA:AVERAGE:0.5:1:600 \
RRA:AVERAGE:0.5:6:700 \
RRA:AVERAGE:0.5:24:775 \
RRA:AVERAGE:0.5:288:797 \
RRA:MAX:0.5:1:600 \
RRA:MAX:0.5:6:700 \
RRA:MAX:0.5:24:775 \
RRA:MAX:0.5:288:797
[/code]

With the database created it needs to be filled every 5 minutes or so. Slightly modifying Dmitri’s MRTG script the JVM data is fed into rrd using rrdupdate. In the following example I query a Tomcat instance, change the ‘grep tomcat’ if you want to monitor something else.
Add this script as a cronjob to have it executed every 5 minutes. Also make sure that it can access the heapsize.rrd file.

[code lang="bash"]
#!/bin/sh
JAVA_HOME=/opt/java
PID=`sudo $JAVA_HOME/bin/jps -lv | grep tomcat| awk '{print $1}'`
DATA=`sudo $JAVA_HOME/bin/jstat -gc $PID | grep -v S0C`

EC=`echo $DATA | awk '{print $5}'`
EU=`echo $DATA | awk '{print $6}'`
OC=`echo $DATA | awk '{print $7}'`
OU=`echo $DATA | awk '{print $8}'`
PC=`echo $DATA | awk '{print $9}'`
PU=`echo $DATA | awk '{print $10}'`

`/usr/bin/rrdupdate heapsize.rrd N:$EC:$EU:$OC:$OU:$PC:$PU`
[/code]

The last thing to do is creating the graph with RRDCGI. RRDCGI is part of the RRDTool and parses a HTML template replacing RRD::Graph tags with the… graph
Store the following cgi somewhere on your webserver (don’t forget it’s a CGI so it must be executable) and make sure your webserver has read access on the heapsize.rrd file.

The end result is a graph stacking the three space capacities on eachother so you’ll have a clear view of the total usage. Current utilization is drawn into each space.

[code lang="html"]
#!/usr/bin/rrdcgi

--rigid
--height=120
--width=400
--start=-1d
--alt-autoscale-max
--vertical-label="byte"
DEF:edencap_=/var/rrd/heapsize.rrd:edencap:AVERAGE
DEF:edenut_=/var/rrd/heapsize.rrd:edenut:AVERAGE
DEF:oldcap_=/var/rrd/heapsize.rrd:oldcap:AVERAGE
DEF:oldut_=/var/rrd/heapsize.rrd:oldut:AVERAGE
DEF:permcap_=/var/rrd/heapsize.rrd:permcap:AVERAGE
DEF:permut_=/var/rrd/heapsize.rrd:permut:AVERAGE
CDEF:edencap=edencap_,1024,*
CDEF:edenut=edenut_,1024,*
CDEF:oldcap=oldcap_,1024,*
CDEF:oldutval=oldut_,1024,*
CDEF:oldutgrap=oldutval,edencap,+
CDEF:permcap=permcap_,1024,*
CDEF:permutval=permut_,1024,*
CDEF:permutgraph_=permutval,edencap,+
CDEF:permutgraph=permutgraph_,oldcap,+

AREA:edencap#00cf00:"eden cap "
GPRINT:edencap:LAST:"Current\:%8.2lf %s"
GPRINT:edencap:AVERAGE:"Avg\:%8.2lf %s"
GPRINT:edencap:MAX:"Max\:%8.2lf %s\n"

STACK:oldcap#8fea00:"old cap "
GPRINT:oldcap:LAST:"Current\:%8.2lf %s"
GPRINT:oldcap:AVERAGE:"Avg\:%8.2lf %s"
GPRINT:oldcap:MAX:"Max\:%8.2lf %s\n"

STACK:permcap#EACA00:"perm cap "
GPRINT:permcap:LAST:"Current\:%8.2lf %s"
GPRINT:permcap:AVERAGE:"Avg\:%8.2lf %s"
GPRINT:permcap:MAX:"Max\:%8.2lf %s\n"

LINE1:edenut#0000ff:"eden util"
GPRINT:edenut:LAST:"Current\:%8.2lf %s"
GPRINT:edenut:AVERAGE:"Avg\:%8.2lf %s"
GPRINT:edenut:MAX:"Max\:%8.2lf %s\n"

LINE1:oldutgrap#002f8f:"old util "
GPRINT:oldutval:LAST:"Current\:%8.2lf %s"
GPRINT:oldutval:AVERAGE:"Avg\:%8.2lf %s"
GPRINT:oldutval:MAX:"Max\:%8.2lf %s\n"

LINE1:permutgraph#008fea:"perm util"
GPRINT:permutval:LAST:"Current\:%8.2lf %s"
GPRINT:permutval:AVERAGE:"Avg\:%8.2lf %s"
GPRINT:permutval:MAX:"Max\:%8.2lf %s\n">

[/code]

This entry was posted on Thursday, October 25th, 2007 at 9:29 pm and is filed under The Rest. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.