Wednesday, July 14, 2004

Arbitrary Complex Data in CFMX Web Services

I've been banging my head against a brick wall with this for a couple of days now, and it's finally dawned upon me that I'm trying to achieve something that the existing mechanism just isn't designed to handle:

I want to put a web-service interface layer on my CFCs.
I originally wanted to have my web-service interface CFCs in a sub-directory under the webroot, so they were publicly accessible but had their own Application.cfm; I also wanted to keep my back-end CFCs outside the webroot.
(maybe it's my background at an IT Security company, but the thought of putting my back-end business logic and data components in an environment where anyone can get at them just makes me feel queasy)
But, after spending an age battling with inter-component references and JRUN mappings , all to no avail, I eventually condeded defeat.
Looks like you HAVE to have your WS interface CFCs in the same directory. Mappings just get ignored.
Oh yeah, and don't use underscores in your CFC names. They won't work.
Oh, and don't use mixed-case either.

Nice one MM...

Anyway, I decided that I'd deal with security later (my old colleagues at the security company would be horrified, but sometimes you just have to suck it and see, y'know?)

Onto the main issue:

I wanted to create a CFC type to be returned from all web service methods, which wouldhave these fields:

  • intReturnCode

    an integer constant, indicating whether the request succeeded, and if not, what went wrong.

  • vcErrorDescription

    a description of the error, if any (e.g. "Could not find the person you asked for" )

  • objReturnData

    This would hold the actual data to be returned, whatever that may be - a query, an object, an array....whatever.

All sounds fine and dandy, and I had a nice object model designed to achieve this - then I started trying to get it work.

Over the last two days, I've seen more hair-pulling, teeth-clenching, blood-pressure-heightening, eye-popping, chair-kickingly frustrating error messages than I have done for years. There's been some great ones, from the old favourite

"Could not generate stub objects for web service invocation"


"Web service operation 'listPersonsByInitial' with parameters {VCEMAIL={[whatever]},VCINITIAL={[whatever]},VCPASSWORD={[whatever]},} could not be found."
...just after adding the web service 'listPersonsByInitial'

but this is my personal favourite:

"java.lang.IncompatibleClassChangeError : Dependent CFC type(s) have been modified. Please refresh your web service client."
When did I first get this error? When I was refreshing my web service client page, because I'd modified the dependent CFCs. GRRRRRRRRRRRRR! I seemed to get this every five minutes or so, even when I hadn't even changed the code!

Anyway, after swearing a lot, spending ages googling for other people with the same errors ( try googling for 'coldfusion "java.lang.IncompatibleClassChangeError"' ) and coming up with a few people with similar problems but NO applicable solutions, I posted some messages to the CF-Talk mailing list. After a few replies along the lines of "er....have you tried (one of the first things I tried) ?" I ended up saying:

"If anyone can come out with a 'I've done this and it worked first time, and I've never had any problems with it...' I'll be eternally grateful if they can walk me through it. Otherwise, I'm giving up - this is way too much of a headache."

That was yesterday. To date, no one has....

But it dawned on me just now that I'm actually trying to get the engine to do something which it fundamentally wasn't designed to do.

It seems like the WSDL which describes the web service gets cached on the client. If you want to change the web service, every client has to update their WSDL. Which is fine, I guess it makes sense that way, and I could live with that in development if it worked consistently.

The problem is that in order for a web service to be able to understand your data, the WSDL needs to describe any non-native type (pretty much anything more complicated than a string or basic number) right down to the last detail. What fields are in your complex data, what type they are... so if you're trying to return arbitrary complex data, you're not enforcing a rigid type in your data, you're leaving it to run-time. So the WSDL can't possibly describe your data ahead of time.

Bottom line: it's my own fault. I'm trying to push the SOAP/web services mechanism way outside of its scope. What I actually need is a whole host of facade classes, one for every bit of copmlex data, even if it's only slightly different.


Back to the drawing board then....

Tuesday, July 13, 2004

General ANT build and release scripts

While pottering about with SVN and ANT, I discovered - yes, you guessed it - SVNANT, a plug in custom task for ANT which provides integration with SVN repositories.

So, I had a bit of a play, and came up with two ANT scripts that I thought I'd share, to save other innocents the buttock-clenching frustration that ANT development on Windows can be.

The first script is to be run on your development machine, and it's called build.xml

This build script:

  • Connects to a given repository repository

  • Exports all files in a given revision from the given folder to a temporary directory

  • Zips up all files within that temporary folder which have been modified since a given date

  • Clears up the temporary folder

…producing a timestamped zip file containing all files changed since a given date.

You can then upload the zip file to the live server and extract it over the live code using the second script, release.xml.

This second script:

  • Scans the build/ directory for build_*.zip files

  • Asks you to choose the build file for release

  • Backs up all files contained in the zip file to the build/ directory

  • Extracts the build_*.zip over the top of the existing files

giving you a backup file called BACKUP_(whatever your build file was called).zip, which you can easily re-apply if something goes wrong with your release.

So, to run the build.xml file on your dev machine, you will need :

You can download all the above in one package here :

I'd recommend that you put the three files in the root of your project - most projects I've worked on tend to have a structure along the lines of:

Project Root
- wwwroot
- components (or custom tags, or whatever)

so i've written the scripts on the basis of them being in the project root.

One thing to note : you'll need to set the repoURL value in to the default url of your repository in SVN, something like svn://yourserver/yourrepository/trunk

To run the script, just open up a command prompt, cd to the project root directory, type:

and follow the on-screen prompts.

This will produce a build file in the Project Root/build/ subdirectory, which will be created if necessary.

On your live server, you'll need a working installation of ANT, plus this file : release.xml

Again, put this in your project root directory on the live server.

To run it, again open a command prompt and cd to the project root directory, but this time type:
ant -f release.xml

to make ant execute the file release.xml (by default it looks for build.xml)

It will look for zip files in the Project Root/build/ subdirectory, so make sure your build .zip file is in there. Again, it prompts you for everything it needs.

Standard legal disclaimer bumph: these files are provided as is, use them at your own risk, if it makes a mess of your server, it's entirely your own fault and I'm not accepting any responsibility whatsoever. Nope, none. Sorry...

Happy scripting!

Thursday, July 08, 2004

Version Control with Subversion and TortoiseSVN

Been looking at version control systems for use with our CF development. After a lot of investigation, I decided to go with Subversion (SVN) for the server, and TortoiseSVN for the client.

These two nifty little tools are free (as in beer) and open source, and they're self-hosting : i.e. the source code and version control for Subversion is done with... Subversion. SVN is intended to be "a new version control system that is very similar to CVS, but fixes many things that are broken", and I'd say it does that job very well.

I've also found the documentation to be pretty good, which is something you don't always get with open-source projects. The TortoiseSVN docs in particular are excellent, I actually found myself looking through the TortoiseSVN docs for info about Subversion rather than the Subversion docs themselves! It's probably the fact that it runs as a Windows Explorer plug-in that makes it feel very intuitive and easy to get to grips with.

Mind you, that's not to say either of them are perfect. There's a few niggles and things that you have to work around - for instance, SVN doesn't support a way to show/export ONLY the files that have changed since a particular revision or date, and setting up an initial repository can be a bit confusing until you've figured out what's actually going on - but they're a damn good place to start. There's also a Subversion API for integration with custom programs and developing addons. I found the svnant package, which gives you a custom ANT task for use in automatic build files to be excellent.

At my last job we used StarTeam, which again took some getting used to, and was also good, but far from perfect -he lack of an explicit branching method was particularly frustrating. But StarTeam costs $699 for just the Standard version, and if you're looking for something free, open-source, widely used, well-documented and fairly easy to get to grips with, you'd have a hard time finding a better system than SVN.

I'll probably blog more details about my issues, solutions and workarounds here in future - stay tuned...