The Enron Explorer!

Phew - at long last we've gone live, and I can finally blog about what we've been working on here!

( yeah, yeah, skip all that, what about Enron? )

Our next major product is aimed squarely at the enterprise market, and it's a tool to analyse and extract meaning from corporate data stores and email traffic. Based on this information extraction, we can then map social networks and analyse information flow throughout the organisation, and use this information to give a shot in the arm to communications effectiveness - forwarding emails to you that are especially relevant to you but that you might otherwise have missed, and letting you set "volume level" for each of your interests.

There's no end of applications for this kind of tech - everything from expertise analysis to Sarbannes-Oxley compliance. The hardest question of all, though, was what to call it. After a tortuous voting and lobbying process second only to the IOC in labyrinthine complexity, we finally settled on the name SONAR, as it kind of implies what the product does (scans things and identifies things that you need to know about) and it's also a merely-mildly-icky acronym - SOcial Networks And Relevance.

Anyway, while we were working on the early stages of SONAR, we needed a large set of plausible test data to test the NLP algorithms - sadly, my usual plethora of furry animals and pitiful puns just wasn't enough in this case - and Jan came up with a stroke of genius - the Enron email archive.

In Oct. 2003, the FERC released ~200,000 Enron emails into the public domain, as part of the inquiry into the Enron fraud. So we grabbed the database, imported it into a very early version of SONAR, and set it chugging away. The results were so absorbing that we can't stop fiddling with it! We decided it was a great way to demo our system, even though it's a very early version, so laydeeeez an' gennulmen, (drum roll) allow me to present....

THE ENRON EXPLORER!

Just click on a name or a theme to get in, and off you go. There's some real gems in the archive, ranging from the hilariously obscene :

Jeff Skilling to Andy Zipper:
Fuck you, you piece of shit. I can't wait to see you go down with the ship like all the other vermin. Smug, paranoid, unhappy mother fucker. Eat shit.
...to which I think Andy Zipper responded with admirable restraint!

- to the heartbreaking:

Sato (Enron Japan) to Andy Fastow:
Please don't fire Enron Japan staff, we do nothing wrong! Please!!! [....] We know how serious is the situation but please don't fire us now. Our families are waiting for a happy chrismas and new year!!

We can (and often do) lose ourselves in this for ages - have an explore and enjoy finding out what they were saying to each other as the house of cards crumbled down around them.

On a technical note, the interface is written in RubyOnRails, with a Java backend. It's AJAX'ed up to the eyeballs - hey, we even have rounded corners and gradient fills on the AJAX loading indicators! - and although there are still a couple of issues to sort out, we've worked hard on maintaining the usual expectations of browser behaviour.

The back button works (with a couple of minor niggles), and you can bookmark and email the url of your current view, and it should (nearly!) always bring you back to the same point. There's some smooth integration between the Java applet visualiser and the AJAX calls too, although again there's still a couple of niggles to deal with.

There's a whole load of other niftiness going on behind the scenes, but I'll blog about that later. In the meantime, have fun with it, and if you find anything particularly juicy, leave a comment (either here or on the app itself) and share it with the world!