The Sentimeter, Part 011

This week I wrap up the Sentimeter, my almost real-time Twitter topical sentiment analysis tool. Go here to catch up.

But first of all this week I must correct something I wrote last week: that SAP, the company that publishes Xcelsius, hadn't responded to a support issue I raised in a developer's forum on June 18. It turns the senior Xcelsius product manager did respond but I hadn't seen the posting, which isn't too surprising as the response was posted July 29, some 42 days later.

In my last Gearhead I ended having just fired up Microsoft's Fiddler, a free Web debugging tool, to try to figure out why the Sentimeter was generating errors.

To continue reading, register here and become an Insider. You'll get free access to premium content from CIO, Computerworld, CSO, InfoWorld, and Network World. See more Insider content or sign in.

This week I wrap up the Sentimeter, my almost real-time Twitter topical sentiment analysis tool. Go here to catch up.

But first of all this week I must correct something I wrote last week: that SAP, the company that publishes Xcelsius, hadn't responded to a support issue I raised in a developer's forum on June 18. It turns the senior Xcelsius product manager did respond but I hadn't seen the posting, which isn't too surprising as the response was posted July 29, some 42 days later.

In my last Gearhead I ended having just fired up Microsoft's Fiddler, a free Web debugging tool, to try to figure out why the Sentimeter was generating errors.

While I think about it, there's one thing to watch out for when you're using this tool: If you leave Fiddler running, as I did, and forget it is handling all your Web traffic, you'll suddenly start to see loads of security certificate failures and your browser will start to bug you about whether you want to create exceptions. Switch Fiddler off and all will be well (yes, it was one of those "du'oh" moments – I did it so you won't have to).

Using Fiddler I was able to figure out that Xcelsius-generated Flash movies display "Error #2032" for crossdomain policy violations AND all sorts of other problems that range from no response to a request through to HTTP status responses such as 404, "ain't no such content", 500, "the server application has barfed", and 504, "gateway timeout" (which OpenAmplify occasionally generates when the Twitter Search API is too slow). Is the fact that "Error #2032" is generated for all these conditions documented anywhere? Of course not!

This discovery necessitated another round of hacking my proxy code so I could trap all of these conditions and return something meaningful to the Sentimeter. Now when there is a problem, my proxy sends the Sentimeter a response with the expected XML structure of a successful request, but with a particular field set to a "magic" (i.e. unique) string to indicate an error. I also set another field to the condition code. That way I can do a simple test for my magic string and then warn the user what happened. I also create null entries in the list of stored values that are graphed by the Sentimeter rather than display misleading results.

There's just one condition my proxy can't help with and that is, of course, when the proxy itself is unavailable for whatever reason (connection down, server down, gods of communications not adequately appeased, etc.). As there's no user definable error handler in Xcelsius, there's no way to detect you are in deep trouble, so you just have to warn your users up front when the application starts. This is business intelligence? Sigh.

Anyway, after all sorts of trials, tribulations and ugly, ugly hacks, I have the Sentimeter at an alpha release stage. The current release, version 0.5.21, which you can try out at http://gibbsuniversal/guii/sentimeter.html, is set up to track the sentiment of Twitter tweets that match a simple search term over a rolling period of one hour (that's going to be extended to 24 hours). As of this writing, only single word searches are supported (that should also change in the next few days). Check it out and let me know what you think.