UniVerse and XML

Parsing XML documents in UniVerse just sucks really. I don't know how to describe it any better - it really just sucks!

Oh but validating them against XML schemas works really well! 1-1!

JDBC Connection Pooling

For JDC database connection pooling you can use this cool datasource called the BasicDataSource which comes out of the commons-dbcp library (and also requires commons-collections and commons-pool). You can set parameters like the initial pool size, the maximum pool size, whether abandoned connections are removed etc.

They key thing to remember while using the connection pooling is that the datasource object is the scope of the connection pool. ie. if you create a new datasource object then you create a new connection pool. So you need to create one datasource and then either pass it to all your child classes (which something like SPring does for you pretty well), or you refer to a static instance of it from your child classes.

This may sound pretty obvious but I've been working on some code today that had this flaw and and was pretty hard to tell it was there. If you have other users/apps hitting the same DB then it is worth creating a new users for this test. The key is to set the maximum pool size to a low number eg. 10), set the initial pool size to 5, and then run up a 20 or 30 threads. You should not see more than 10 connections to the database (use 'show processlist' in MySQL). If you do, then the pooling isn't working and your code is probably creating datasources as it goes. Nasty!

Enabling multiple localhost sites in Tomcat

I'm developing code on a Mac under Tomcat for a multi-application website which looks at the domain name to determine which app to run. It was getting damn annoying changing the database tables whenever I wanted to login to a different app. "There must be a better way to do this" he cried! ... and there is!

Rather than just having localhost pointing to your machine, you can have any name you want. eg. dev.localhost, qa.localhost, test.localhost, etc.

Step 1.
Setup the new host names in your hosts file. For me this is under /etc/hosts and I have to sudo to edit it. In there you will already have localhost setup. Add your new ones underneath and save it.

127.0.0.1 localhost
127.0.0.1 qa.localhost
127.0.0.1 dev.localhost
127.0.0.1 test.localhost

Now you should now be able to ping these and see the 127.0.0.1 IP address. In fact, you can try your URL with this server-name instead of localhost and get to your site!

Step 2.
Setup Tomcat to route these hosts to your app. This will run a seperate Tomcat instance for each application. Modify the server.xml file (in the conf folder) by adding the following XML:

<Host name="qa.localhost" appBase="webapps">
<Context path="" docBase=".">
<Host>

<Host name="dev.localhost" appBase="webapps">
<Context path="" docBase=".">
<Host>

<Host name="demo.localhost" appBase="webapps">
<Context path="" docBase=".">
<Host>

I guess if you have other web applications that sit elsewhere you can just modify the appBase path. I didn't need to do this. Note that I was using Eclipse with this setup and when I tried to debug, it started a host for each server and took ages to start; and then died with an out of memory error (this was with 7 aliases setup).

Step 3.
Try it out. Fire up Tomcat, and browse to http:/qa.localhost:8080/{blah blah} and you're away! Nice one brother!

Apache Tomcat URL Limit?

So how long can your URL be before Tomcat spits the dummy and gives you an HTTP 414 error?

On the site I'm currently working on it is 8208 characters. 8207 works fine, 8208 gives the 414 (Request URI too large). And that is one looooonnnnnnnnnnng URL!

That's using Tomcat 5.5.17 and Java 1.5.0_06-b05

Software dev can be so frustrating...

This has got to be a typical scenario. I'm working on a system that has been half-written by another team and never quite made it into production. I started work yesterday on a chunk of software dev work that I expected to take about 1/2 a day...so I'd allowed myself a full day to do because we all know to expect the unexpected.

There are four bits to this job, all of them similar. So I'm expecting the first one to take up about half of the time (identify the approach, solve it, test it), and then the other three would flow on pretty easily from there taking the other half of the time. This is one of those cases when 1/4 = 1/2!

So I'm making good progress on the first bit - I'm just testing it out and I'm on target...maybe even ahead of schedule...could be time to nip out for a coffee or a paddle? But then the test results start looking all wrong...further investigation reveals that there is already a bug in the system here and it isn't doing what it should.

Bugger! What do I do? This is a one in ten bug so I really have to fix it. The paddle is definitely off but the coffee could still happen (might have to be a short black though!). So I fix it and find that this is a bit of a b$#%h to test - it requires an exact setup and you've got to do things in the right order. Time ticks on - a good thng I allowed a full day!

But then these tests start to look a bit screwy and lo, it turns out that there is another bug in a fundamental area that this depends on. SH%$#T!

You get the idea, before long I've forgotten what the original job was. By the end of the day (a rather LONG day) all I've succeeded in doing is the first 1/4 of the job, and now I'm faced with writing up all these other little bugs I've found to justify how I've spent the day. And by the time I've done that I'll have lost the flow on the original job and that will end up taking another day.

C'est la vie!

(and it is a good thing I get paid by the hour!) :-)

XSD validation

I spent some time yesterday faffing around with XML and XSD files. In particular, I wanted to use an XML file to send data between systems. The idea being that I could give a sample XML document and and XSD file to someone and they would know enough to interface into the system.

To start with, creating the initial XML document was a pain in the butt! Using my favourite windows text editor TextPad proved to be tedious, so I use Microsoft's XML Notepad 2007. The UI was a bit clunky but it was nice and simple to use.

So XML created, the next step was to generate the XSD. Again, this proved to be a pain in the butt, particularly checking if the XSD syntax was OK. In the end I used Visual Studio 2005 to edit the XMLSchema a it gives the nice red squiggly line under the bits with errors.

So the final step was to validate the XML against the XSD. You think this would be easy right? Well, no.

The browsers (both Firefox2 and IE7) only performed limited validation. They would validate the tags, but not perform the xsd:restrictions. To get this going I had to edit the XML document I wanted to validate and put the name of the XSD file as an attribute of the first element, and then open the XML document via the browser.

eg. <myfirstelement xsi:noNamespaceSchemaLocation="file://c:/xmlschemas/myschema.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> ...

Good ole XML Notepad seemed the best option. You can choose View-Schemas and then add in your XSD and it will validate against it. Not as simple as I wanted but it worked.

On the Mac I used EditiX (on a 30 day trial) which seemed pretty good, and included the ability to create an XSD from a chunk of XML which can get you started faster. VS2005 also has this feature.

Post number one...

...and there it was!

Another fascinating post joins the blogosphere and provides thousands of lonely web-crawlers a moment of joy - discovering and indexing another post so that millions of people have something pointless to read.