Add another to my bounty of reasons for avoiding Javascript. This is not a problem with Javascript as such, more-so a problem with web functional testing tools. Still if you don’t use Javascript, you won’t have these and many other problems the most notably being cross-browser compliance issues.

So, to the problem; In my efforts to grow a virtually non-existent set of acceptance tests, I was after a way to disable Javascript in JWebUnit. Google told me this:

HttpUnitOptions.setScriptingEnabled(false);

Great, that was easy.

It seemed to work. Then enter ludicrously complex Javascript date selection pop-up. It appeared as if scripts were running again, so I investigated and discovered on each of the following WebResponse methods HttpUnit reloads all page elements, including attempting to load and execute scripts *Deep Breath*:

getForms, getLinks, getApplets, getImages, getTextBlocks, getTables, getElementsWithName, getElementsWithAttributes, getElementNames, getElementsByTagName, getElementsWithID, getFrames

Phew! That’s plenty of opportunity for a page to get reloaded. Given it’s possible to invoke these many times interrogating the one web response, how can this strategy be a good thing? Why re-assemble the entire page just because I want to retrieve elements by id for instance? If the answer is ‘Just in-case Javascript has done it’s dirty-work in the meantime’, damn, I hope there is a better way. Surely the performance effects are very significant. Significant enough to at least make you think twice when tempted to use these methods.

So add another to my pile of reasons for hating Javascript. Oh, and does anyone have any idea why disabling Javascript in HttpUnit does not disable this reloading feature? Any answers much appreciated.

Plenty of people fail to see the value in unit testing. As it discourages testing the real code working together they can’t see the value.

What if a method returns a value, for arguments sake the string “A”, and that value is passed to a method that has never been unit tested to expect “A” - how can you guarantee what the system will do?

My answer is this - if you haven’t tested a behavior the outcome is non-deterministic.

But there is a problem with the question - your unit tests should be testing all the vastly different inputs and outputs expected by methods. If an input hasn’t been tested, you can’t determine the methods behavior. Essentially you have an incomplete test.

So what if the application behavior changes in some way, and it’s possible for methods to pass around different values? Say you write an isolated test for the core new behavior, codify it, run your unit tests - you see green lights. Fine. You then run your acceptance tests and you see tests exploding everywhere - nothing but red.

My point is this - unit tests are not intended to test the integration of methods - they are intended to ensure the functionality of a method, based on expected inputs and outputs, is correct. They are of high value in isolating the root cause of problems. Theoretically, with excellent unit test coverage, if a method output is altered to be vastly different, the unit tests should fail for that method only.

What if the inputs are dynamic, for instance database driven - how do you test the application works with this data? The short answer is unit tests combined with making every effort to ensure the data is legitimate data the application expects. There are masses of tools that provide data cleansing facilities should the state of your data warrant it. Simple things, like guaranteeing types, length, null states etc., can be easily controlled through a nice schema. If you’re schema’s crufty, it will lead to all sorts of compromises in the application - I rarely, if ever, justify the effort insulating the codebase from a layer of cruft. Fixing the cruft is always the by far the easiest course of action in the long-run.

So what about integration tests? Do I find them useful? Absolutely. I still write integration tests for code integrating with external services, such as persistent stores - but I still test a single method, just as I do in unit tests. Any integration test invoking a number of source code methods reeks of performing the application logic - essentially you’re venturing into the realm of testing the test.

So what about testing to make sure the application hangs together? I think the best candidate for this is functional/acceptance tests, simulating user flows through the application. Integration tests exorcising a number of application layers are confusing, extremely difficult to maintain and generally not worth the effort in comparison to the effort involved in writing simple acceptance tests.