Arbitrary Complex Data in CFMX Web Services

I've been banging my head against a brick wall with this for a couple of days now, and it's finally dawned upon me that I'm trying to achieve something that the existing mechanism just isn't designed to handle:

I want to put a web-service interface layer on my CFCs.
I originally wanted to have my web-service interface CFCs in a sub-directory under the webroot, so they were publicly accessible but had their own Application.cfm; I also wanted to keep my back-end CFCs outside the webroot.
(maybe it's my background at an IT Security company, but the thought of putting my back-end business logic and data components in an environment where anyone can get at them just makes me feel queasy)
But, after spending an age battling with inter-component references and JRUN mappings , all to no avail, I eventually condeded defeat.
Looks like you HAVE to have your WS interface CFCs in the same directory. Mappings just get ignored.
Oh yeah, and don't use underscores in your CFC names. They won't work.
Oh, and don't use mixed-case either.

Nice one MM...

Anyway, I decided that I'd deal with security later (my old colleagues at the security company would be horrified, but sometimes you just have to suck it and see, y'know?)

Onto the main issue:

I wanted to create a CFC type to be returned from all web service methods, which wouldhave these fields:

• intReturnCode
  
  an integer constant, indicating whether the request succeeded, and if not, what went wrong.


• vcErrorDescription
  
  a description of the error, if any (e.g. "Could not find the person you asked for" )


• objReturnData
  
  This would hold the actual data to be returned, whatever that may be - a query, an object, an array....whatever.

All sounds fine and dandy, and I had a nice object model designed to achieve this - then I started trying to get it work.

Over the last two days, I've seen more hair-pulling, teeth-clenching, blood-pressure-heightening, eye-popping, chair-kickingly frustrating error messages than I have done for years. There's been some great ones, from the old favourite

"Could not generate stub objects for web service invocation"

through

"Web service operation 'listPersonsByInitial' with parameters {VCEMAIL={[whatever]},VCINITIAL={[whatever]},VCPASSWORD={[whatever]},} could not be found."
...just after adding the web service 'listPersonsByInitial'

but this is my personal favourite:

"java.lang.IncompatibleClassChangeError : Dependent CFC type(s) have been modified. Please refresh your web service client."
When did I first get this error? When I was refreshing my web service client page, because I'd modified the dependent CFCs. GRRRRRRRRRRRRR! I seemed to get this every five minutes or so, even when I hadn't even changed the code!

Anyway, after swearing a lot, spending ages googling for other people with the same errors ( try googling for 'coldfusion "java.lang.IncompatibleClassChangeError"' ) and coming up with a few people with similar problems but NO applicable solutions, I posted some messages to the CF-Talk mailing list. After a few replies along the lines of "er....have you tried (one of the first things I tried) ?" I ended up saying:

"If anyone can come out with a 'I've done this and it worked first time, and I've never had any problems with it...' I'll be eternally grateful if they can walk me through it. Otherwise, I'm giving up - this is way too much of a headache."

That was yesterday. To date, no one has....

But it dawned on me just now that I'm actually trying to get the engine to do something which it fundamentally wasn't designed to do.

It seems like the WSDL which describes the web service gets cached on the client. If you want to change the web service, every client has to update their WSDL. Which is fine, I guess it makes sense that way, and I could live with that in development if it worked consistently.

The problem is that in order for a web service to be able to understand your data, the WSDL needs to describe any non-native type (pretty much anything more complicated than a string or basic number) right down to the last detail. What fields are in your complex data, what type they are... so if you're trying to return arbitrary complex data, you're not enforcing a rigid type in your data, you're leaving it to run-time. So the WSDL can't possibly describe your data ahead of time.

Bottom line: it's my own fault. I'm trying to push the SOAP/web services mechanism way outside of its scope. What I actually need is a whole host of facade classes, one for every bit of copmlex data, even if it's only slightly different.

Bugger.

Back to the drawing board then....