Test First UI experiences

For the last year on my own and for the last six months or so with my
development team at the office we have been learning, experimenting, and
otherwise playing with or extending our test driven development ideas
to UI code, primarily web based UI’s. We settled on our own variation
of the Model View Presenter pattern that went through many iterations
on both pure sample code and eventually on released systems. This
effort was not a small one. There is very little consensus on how to
test UI code in general, even less on how to do TDD with UI code and
far fewer real world examples to draw on for both samples and
confidence. In addition, my team wasn’t as sold on the idea as I was
though they definitely appreciated many of the benefits we were trying
to get from the effort. The deciding point had less to do with
developers than with the rest of the company who were responsible for
manual testing of those things that couldn’t be automated and are now
more free to concentrate on new development more than on worrying about or executing regression

Now, everybody has done TDD development of UI code here. We are still
struggling with choices between dynamic mock frameworks vs hand coded
mocks because each represents it’s own pain level and that pain doesn’t
always arrive until you’ve already committed to one direction or the
other. I need to experiment with the NMock2 and Rhino frameworks more
to see if the pain is lessoned somewhat, but with NMock we often seem
to have paint ourselves into corners that the framework doesn’t

But the real reason for this post is that after having gone through the
first and second implementations, I would have thought that with the
heaviest part of the learning curve behind us, the benefits would feel
good enough to make us forget about the awkward parts. The opposite is
happening and now I need to reflect on the experience and figure out
why. Here is some of the feedback I’m getting on this that I didn’t get
when we started this process on our domain objects.

  • UI code really doesn’t get reused that much so designing and
    testing for reuse is not as easy to justify as it is for domain
    objects. In that case, the final acceptance tests are much more useful
    because the same people will request changes and won’t mind redoing
    their tests after the change
  • The
    tests seem to take much longer to write than domain object tests and
    developing our skills hasn’t helped as much as we would have expected
    to at this point.
  • The actual code needed to make the UI testable is much more
    complicated to write, even though it’s much simpler to test. There is
    not a sense that this code is actually better except in cases where
    complicated interactions in the UI are difficult to test manually. In
    other words, on simple UI’s the result is an anti-benefit, but for some
    the design really does help implement the code
  • Domain objects evolve and become richer and more powerful over time. With UI code, the objects just increase in number
  • Test suites in the UI seem to make us less agile because
    you have to drag not only the domain code, but now interfaces, mocks,
    views and presenters along with you on every change of requirements

The last bullet is the salient point here. I have long felt that
the most valuable business reason for pursuing TDD and automated
testing is general is the impact that trail of tests has on the whole
process, simplifying things dramatically for those doing the acceptance
testing and shortening the whole process. The extra time spent unit
testing has worked as advertised, but with UI based testing, this
doesn’t seem to hold true. The benefits of regressions are far less if
you don’t reuse the code. An acceptance test by the project leader can
often verify most and in some case more that what the unit test would
verify. That statement falls apart in multiple screen wizards or
certain types of rich UI scenarios, but that leads to my conclusion
from all this.

Why not take take the stance that you should design so your UI *can* be
tested, certain key things are tested, and the “hard parts”, whatever
they are, are tested? The rest is left for acceptance tests, which have
to be done anyway. Here are the benefits

  • less coding time up front
  • Better agile story for changing the UI. There is less time fixing
    tests that only demonstrate the design, something that’s obvious
    already to the acceptance tester
  • When there are complicated UI design issues, application context
    specific strategies (for company X do Y. For company Z do Y and Z) you
    still can apply your large arsenal of testing experience and tools to
    the problem because the design is testable

This point of view seems especially appropriate in the early stages of
a design when both the developer and the users are discovering what the
application will be and changes are just part of the dialog. I hate the
idea of restricting that dialog, but if you insist on building a
regression safe barrage of tests for every iteration of your design, it
seems to be exactly opposite of the agile goals we’re trying to
achieve. But even if you have agreed to put off some of the test
writing until the design stabilizes, you have also implicitly said you
are not going to do TDD. The approach I’m suggesting is still TDD, but
statements like “the code is fully tested” is not satisfied by the unit
test suite. It would be satisfied by the process however: the project
sponsors would not sign off until it’s been accepted.

So, let’s hear it, we can’t be the first people to notice flaws in the TDD story for UI design.


Comments are closed.