Finally eHour 0.7 is out ! I underestimated the move from Struts to Wicket quite a bit so this new version took a while. Not that Wicket is that hard to learn but the amount of code that had to be redone was huge..

Anyway, eHour now comes in two flavours – a standalone version and a WAR file that you can deploy under Tomcat or any other servlet container. The datamodel is still backwards compatible with 0.6 so the upgrade should be easy.

Another drastic change is the license change. eHour is no longer distributed under the GPL license although the only difference with the GPL will be that it’s no longer open source but just free. A real license is still to be defined.

Release notes:

General
——-
The user interface is completely rewritten using Apache Wicket.

Bugs fixed
———-
#1761496 – Unable to print 1 assignment
#1720193 – Quick dates have no effect
#1782491 – ‘null’ assignmnent start & end date results in nullpointer
#1780975 – No validation for entering date
#1753473 – Time Sheet empty after login
#1751252 – Error when adding assignments to user account
#1743637 – Unrequested End Date Changing
#1743634 – Edit Project Assignment Day Listing
#1740848 – Unable to delete assignment
#1725554 – E-Mail check too restricitve
#1688620 – Project assignment date mismatch
#1668143 – Admin, assignment showns non-consultant roles
#1670797 – submitted hours not checked on server

Known issues
————
Print timesheet doesn’t work in the standalone version.
All pages containing a dropdown date picker load slowly.

 

After 5 months of part-time development the Struts-to-Wicket conversion is complete. Download the standalone version from here.
Some additional testing is required but this is most likely the next release. If you encounter any problems, let me know.

 

Preview 2 is available for download !

Changelog:

  • form validation is back (downgraded wicket 1.3b4 to b3 fixed it)
  • admin pages save fixed (hibernate sequence was missing in the ddl scripts)
  • lots of minor stuff

Known issues

  • creating a new user sometimes throws an error although the data is saved
  • print month doesn’t work
  • the stylesheet of reporting is a messed up
 

Reading Dmitri Maximovich’s blog posting about plotting JVM memory into MRTG graphs had to trigger my interest as I’m a statistics addict. Earlier on I tried to get JVM stats using SNMP but it took me more than 30 minutes to get any data so I dropped it. The interesting thing about Dmitri’s method is that it uses jstat which is far more easier, read his posting for more details.

Anyway, I’m not using MRTG anymore but RRDTool. RRDTool is a lot more flexible and graphs aren’t created every 5 minutes but whenever you request them – at least with rrdcgi.

To get JVM data into RRD is easy. First create a RRD database with 6 datasources; a capacity and utilization datasource for eden, old & permanent size. The following script creates a heapsize.rrd which is the actual database.

/usr/bin/rrdtool create
heapsize.rrd
--start N
--step 300
DS:edencap:GAUGE:600:0:10000000
DS:edenut:GAUGE:600:0:10000000
DS:oldcap:GAUGE:600:0:10000000
DS:oldut:GAUGE:600:0:10000000
DS:permcap:GAUGE:600:0:10000000
DS:permut:GAUGE:600:0:10000000
RRA:AVERAGE:0.5:1:600
RRA:AVERAGE:0.5:6:700
RRA:AVERAGE:0.5:24:775
RRA:AVERAGE:0.5:288:797
RRA:MAX:0.5:1:600
RRA:MAX:0.5:6:700
RRA:MAX:0.5:24:775
RRA:MAX:0.5:288:797

With the database created it needs to be filled every 5 minutes or so. Slightly modifying Dmitri’s MRTG script the JVM data is fed into rrd using rrdupdate. In the following example I query a Tomcat instance, change the ‘grep tomcat’ if you want to monitor something else.
Add this script as a cronjob to have it executed every 5 minutes. Also make sure that it can access the heapsize.rrd file.

#!/bin/sh
JAVA_HOME=/opt/java
PID=`sudo $JAVA_HOME/bin/jps -lv | grep tomcat| awk '{print $1}'`
DATA=`sudo $JAVA_HOME/bin/jstat -gc $PID | grep -v S0C`

EC=`echo $DATA | awk '{print $5}'`
EU=`echo $DATA | awk '{print $6}'`
OC=`echo $DATA | awk '{print $7}'`
OU=`echo $DATA | awk '{print $8}'`
PC=`echo $DATA | awk '{print $9}'`
PU=`echo $DATA | awk '{print $10}'`

`/usr/bin/rrdupdate heapsize.rrd N:$EC:$EU:$OC:$OU:$PC:$PU`

The last thing to do is creating the graph with RRDCGI. RRDCGI is part of the RRDTool and parses a HTML template replacing RRD::Graph tags with the… graph ;)
Store the following cgi somewhere on your webserver (don’t forget it’s a CGI so it must be executable) and make sure your webserver has read access on the heapsize.rrd file.

The end result is a graph stacking the three space capacities on eachother so you’ll have a clear view of the total usage. Current utilization is drawn into each space.

#!/usr/bin/rrdcgi

<HTML>
<HEAD><TITLE>stats</TITLE>
<META HTTP-EQUIV="CACHE-CONTROL" CONTENT="NO-CACHE">
<META HTTP-EQUIV="PRAGMA" CONTENT="NO-CACHE">
<meta http-equiv="refresh" content="300">
</HEAD>
<BODY>

<RRD::GRAPH heapsize.gif  --title="tomcat heapsize"
--rigid
--height=120
--width=400
--start=-1d
--alt-autoscale-max
--vertical-label="byte"
          DEF:edencap_=/var/rrd/heapsize.rrd:edencap:AVERAGE
          DEF:edenut_=/var/rrd/heapsize.rrd:edenut:AVERAGE
          DEF:oldcap_=/var/rrd/heapsize.rrd:oldcap:AVERAGE
          DEF:oldut_=/var/rrd/heapsize.rrd:oldut:AVERAGE
          DEF:permcap_=/var/rrd/heapsize.rrd:permcap:AVERAGE
          DEF:permut_=/var/rrd/heapsize.rrd:permut:AVERAGE
        CDEF:edencap=edencap_,1024,*
        CDEF:edenut=edenut_,1024,*
        CDEF:oldcap=oldcap_,1024,*
        CDEF:oldutval=oldut_,1024,*
        CDEF:oldutgrap=oldutval,edencap,+
        CDEF:permcap=permcap_,1024,*
        CDEF:permutval=permut_,1024,*
        CDEF:permutgraph_=permutval,edencap,+
        CDEF:permutgraph=permutgraph_,oldcap,+

        AREA:edencap#00cf00:"eden cap "
        GPRINT:edencap:LAST:"Current:%8.2lf %s"
        GPRINT:edencap:AVERAGE:"Avg:%8.2lf %s"
        GPRINT:edencap:MAX:"Max:%8.2lf %sn"

        STACK:oldcap#8fea00:"old cap  "
        GPRINT:oldcap:LAST:"Current:%8.2lf %s"
        GPRINT:oldcap:AVERAGE:"Avg:%8.2lf %s"
        GPRINT:oldcap:MAX:"Max:%8.2lf %sn"

        STACK:permcap#EACA00:"perm cap "
        GPRINT:permcap:LAST:"Current:%8.2lf %s"
        GPRINT:permcap:AVERAGE:"Avg:%8.2lf %s"
        GPRINT:permcap:MAX:"Max:%8.2lf %sn"

        LINE1:edenut#0000ff:"eden util"
        GPRINT:edenut:LAST:"Current:%8.2lf %s"
        GPRINT:edenut:AVERAGE:"Avg:%8.2lf %s"
        GPRINT:edenut:MAX:"Max:%8.2lf %sn"

        LINE1:oldutgrap#002f8f:"old util "
        GPRINT:oldutval:LAST:"Current:%8.2lf %s"
        GPRINT:oldutval:AVERAGE:"Avg:%8.2lf %s"
        GPRINT:oldutval:MAX:"Max:%8.2lf %sn"

        LINE1:permutgraph#008fea:"perm util"
        GPRINT:permutval:LAST:"Current:%8.2lf %s"
        GPRINT:permutval:AVERAGE:"Avg:%8.2lf %s"
        GPRINT:permutval:MAX:"Max:%8.2lf %sn">
</body>
</html>

 

eHour 0.7 standalone test version is available for download here !

This is a standalone test version which contains quite some bugs. Don’t use it for anything else but previewing and bug reporting. As this uses an embedded database you cannot use your 0.6 data. The final version will consist of a MySQL/Tomcat combination, compatible with previous versions, and this standalone flavour.

To report bugs please use the comments or this bugtracker. Don’t use the Sourceforge’s one, thanks!

24/10: fixed the download link, whoops ;)

 

When you distribute your application a common task is to create the database tables and prefill them with some static data. You could do this using an SQL script but this complicates the installation process, especially if you use an embedded database such as Derby. By utilizing DdlUtils and a Jetty life cycle you can automate this whole process, creating the database just before your application starts.

DdlUtils reads the table definitions and static data from XML files you distribute along with your application. The easiest way to create those XML’s is to generate them from an existing database with an Ant task.
The build.xml example below generates two XML files (ddl.xml & dml.xml) out of a MySQL database.

Take note that the Maven 2 Ant tasks are used to construct the classpath. If you don’t use Maven 2, make sure ddlutils and your jdbc driver are on the classpath.

<project name="eHourDdlExport"
		default="database-dump"
		basedir="."
		xmlns:artifact="antlib:org.apache.maven.artifact.ant">

	<artifact:dependencies pathId="dependency.classpath">
		<dependency groupId="org.apache.ddlutils" artifactId="ddlutils" version="1.0"/>
		<dependency groupId="log4j" artifactId="log4j" version="1.2.13"/>
		<dependency groupId="mysql" artifactId="mysql-connector-java" version="5.0.5"/>
	</artifact:dependencies>

	<target name="database-dump" description="Dumps the database structure">
		<taskdef name="databaseToDdl" classname="org.apache.ddlutils.task.DatabaseToDdlTask">
			<classpath refid="dependency.classpath"/>
		</taskdef>

		<databaseToDdl modelName="MyModel">
			<database url="jdbc:mysql://127.0.0.1/db" driverClassName="com.mysql.jdbc.Driver" />
			<writeSchemaToFile outputFile="src/main/resources/db/ddl.xml"/>
			<writeDataToFile outputFile="src/main/resources/db/dml.xml"/>
		</databaseToDdl>
	</target>
</project>

Now that the data you want to preload is exported into a portable XML format it’s time to configure Jetty.

First register a JDNI datasource in the jetty.xml. In this example I’m using Derby with the create flag set since DdlUtils
can’t auto-create the database itself.

<New id="Datasource"
	class="org.mortbay.jetty.plus.naming.Resource">
	<Arg>jdbc/eHourDS</Arg>
	<Arg>
		<New class="org.apache.derby.jdbc.EmbeddedDataSource">
			<Set name="DatabaseName">ehourDb</Set>
			<Set name="createDatabase">create</Set>
		</New>
	</Arg>
</New>

Next thing is adding a new Jetty life cycle. Since Jetty executes the life cycles in the order they’re listed in
the jetty.xml, add it before the WebAppDeployer lifecycle (or whichever you use to boot up your main application)
as you want the database to be prepared before the actual application boots.

	<Call name="addLifeCycle">
		<Arg>
			<New class="net.rrm.ehour.db.DbValidatorLifeCycle" />
		</Arg>
	</Call>

Creating the DbValidatorLifeCycle is easy, extend Jetty’s AbstractLifeCycle and override the doStart method. The JNDI
tree is already available so the datasource can be plucked from there.

public class DbValidatorLifeCycle extends AbstractLifeCycle
{
    public void doStart() throws Exception
    {
		DataSource dataSource = getDataSource();

		checkDatabaseState(dataSource);
    }

    private DataSource getDataSource() throws NamingException
    {
		InitialContext initialContext = new InitialContext();
		DataSource datasource = (DataSource)initialContext.lookup("jdbc/eHourDS");

		return datasource;
    }

	private void checkDatabaseState(DataSource datasource)
	{
		..
	}
}

The final part is using DdlUtils to create the tables and load the data into the database.
The ddl.xml and dml.xml files were created by the ant task so make sure you distribute them along
with your application.

private checkDatabaseState(DataSource dataSource)
{
	Connection connection = dataSource.getConnection();
	Platform platform = PlatformFactory.createNewPlatformInstance(dataSource);

	// create the tables
	Database ddlModel = readModelFromXML("db/ddl.xml");
	platform.createTables(ddlModel, false, false);

	// insert the static data
	DatabaseDataIO	dataIO = new DatabaseDataIO();
	DataReader dataReader = dataIO.getConfiguredDataReader(platform, ddlModel);
	dataReader.getSink().start();
    dataIO.writeDataToDatabase(dataReader, new File(""db/dml.xml").getAbsolutePath());
}

In this example the current state of the datamodel is not checked so if there’s any existing data
already in the model you might end up with some conflicts when you try to recreate the tables.
One solution is of course to drop everything beforehand but this is probably not preferred when you want to
retain existing data during an upgrade.
A possible solution is to version your datamodel. Create a separate ‘configuration’ table which holds a
record containing the version of the existing datamodel. Now based on the existence of the version you
can decide whether you have to do a platform.createTables or a platform.alterTables when the version doesn’t match.

 

The end is in sight, all functionality is done and only browser compatibility and fixing bugs is left.
A new version is deployed showcasing the print month sheet and PM reports. Also all the admin pages are Firefox/IE6/IE7 compatible, just have to redo that ugly save icon :) Please test a bit and let me know if you ran into any problems.

Login details for the preview version are in the previous post

 

eHour 0.7 preview is available at http://dev.ehour.nl/ehour-test/.

You can login with one of the following accounts:
username: thies, password test for a normal user
username: report, password test for a reporting user
username: admin, password test for an admin user

The application is running in demo mode, saving/deleting in the admin section is disabled.

There are still a lot of bugs to fix and the print functionality and project mgmt section isn’t added yet. The UI isn’t finished either and was only tested in Firefox so far. Why the preview then if there’s so much unfinished stuff? Mostly because I want to know how Wicket behaves and especially the memory load.

So please click away and if you run into really odd stuff please register the bug.

 

Unfortunately the 0.7 release date of mid September went by without having a release. Daily life annoyances took way more time than I anticipated.

The good thing is that the release is nearing completion. All the complex stuff such as reporting, assignment admin, etc. is refactored. The only open issues left are:

  • refactor timesheet printing
  • refactor project management reports
  • better error handling
  • fix a couple of TODO’s and FIXME’s
  • make the css’s work in IE6/IE7
  • testing

When it’s done, it’s done. Please be patient :)

 

Now for some it may come as a disappointment that the new project assignment admin functionality is a replica of the existing one. However it’s a lot less buggy and better validating than the original one.

Assignment admin

© 2011 Thies' blog Suffusion theme by Sayontan Sinha