I was recently on site with a client where I was sharing my experiences with Test Driven Development (TDD), and how it can help with different types of problems. Note that this client has nothing but sharp, enthusiastic, and friendly employees. Unfortunately their development process has severely limited how quickly they can release new products to the market. A large new infusion of capital was called for to modernize their systems and their practices. My little software consultant shop was hired to help with both endeavors: help improve the software and help improve the process.
This particular week, I met with the head of the QA department and had a good one hour discussion about the merits of TDD and specifically my experiences in different types of organizations: teams with technologically-savvy QA staff and those without.
This positive first meeting lead this manager to ask me to demonstrate to a larger audience a very specific technique: how do you test something that isn't built yet?
Developers practicing disciplined TDD do this often, many times a day, with our unit and integration tests. We write tests that at first don't even compile, much less pass and “go green”. Our IDE's feedback mechanisms are practically screaming at us to fix these problems. We get to the point where we don't even consider this practice odd (or "backwards") at all. Many of us come to like it so much that we never want to go back. But from a product QA perspective, where they're (mostly) thinking about acceptance testing, I realized that as a community- perhaps we Agilistos have done a poor job communicating how this can happen. For example, I was asked, "How in the world can I test a web page that hasn't been built yet?".
Having the most experience solving this particular problem using Selenium, I rehearsed for about 20 minutes before the meeting to make sure I could pull off something useful. I made sure I coded a few simple, but non-trivial acceptance tests that failed (because the web page didn't exist yet), and then incrementally developed the (dreamed up) feature so that within 20 minutes or so all the tests passed.
Then the meeting: I proceeded to do the same demonstration for the larger audience, which took about two hours. Why so much longer? Well, first of all- it was less a demonstration and more of a discussion: we discussed the techniques; the technology (Java, Spring, Groovy, Selenium, MVC); switching roles between product owner, tester, and developer; and encouraging them to do the same. They also came up with a much better idea for a feature to build in this demonstration, which happened to be slightly more complex but still feasible. I resisted the urge to fall back on exactly what I had rehearsed, but felt a little like I was living without a net.
I'll share the feature because I think it's important to illustrate just how small of scope we aimed for: the beginnings of a Help section of a web site. We wanted a Help-Menu page with links to other Help sections: help with cookies and help with signing in. They wanted the page accessible when the user was signed in and when they weren't yet. As we continued in product owner role, we realized we didn't want the "Trouble signing in?" link on the page in the case when you've already signed in. And lastly, we needed a link, from the top navigation, to this new Help-Menu page, visible in both user states (signed in and not). That was it. Pretty simple.
The real meat of the "How to test a web page that doesn't exist yet?" was mostly explaining how we can use Selenium's API and locators to expect certain text or behavior on any web page, even one that hasn't been built. We had healthy discussion about how some locators (such as overly specified XPath expressions) end up influencing and restricting how the developers build the page. In our case, I recommended using the link text as the locator, so that the developer would be free to develop the markup however she saw fit. So after writing just a few Selenium/JUnit test cases, we now had 4 tests that failed. 4 red lines. Perfect.
Switching now to the developer role, I designed and implemented the smallest increment of work I could think of. And of course- it was the smallest amount of work I could think of to make just one test pass. Boom- it did. 3 red lines. One green.
I really wanted to hammer the importance of this home to this team: this was, on a very small scale, the very essence of TDD. Product owners, testers and developers working very closely together early in the life-cycle of the feature. The feature isn't releasable on some arbitrary subjective metrics: it's ready when the tests pass. Who wrote the tests? The product owner, tester, and developer- together. And also- visible progress.
Then within another 20 minutes we finished the entire story and made all 4 tests go green.
We then attempted some extrapolation: imagine what our daily standup meetings would be like if we agreed to always have them around a dedicated projector that was showing two screens: yesterday's red and green bars, and today's red and green bars. I realized we had happened onto something important that isn't expressed often at all in Agile development: a Continuous Integration build that we intend to be broken. Most of the mindshare around Continuous Integration is having a build that is always green- and when the build does break (and it will rarely), it is addressed immediately. But that's a different sort of build: that one is extremely valuable from a development integration perspective, but arguably worthless from a "delivering business value" perspective. And our "epiphany" was that most executive sponsors and stakeholders don't give the smallest damn about development integration- they care about progress. They shared how their executives are often frustrated because they don't see the results of all their hard work until just before the team releases to production. Which, at the moment, is infrequent.
Now imagine having this projector always on, centrally located in an open and informative workspace, where stakeholders can look at it on their way to get coffee. They'd see red bars with items like "Signed in users do not see the 'Trouble signing in?' section on the Help Menu." - but they see green bars for items like "Both signed in and non-signed in users can navigate to the Help Menu easily from the top navigation area". The next day (or an hour later) they walk by and see all green for the tests for the Help Menu feature.
How could there not be a huge increase in trust between the stakeholders and the team if this happened daily or hourly? How could this not improve communication between all roles on the team? How could this not shift the end-of-iteration burden off QA's shoulders (where it is in every single waterfall process). And wouldn't it be fun too?
Of course, these practices have been written about- Steve Freeman and Nat Pryce write eloquently and practically about it in their excellent "Growing Object-Oriented Software, Guided by Tests" book- perhaps my favorite of 2009.
And some thought leaders like Jeff Patton are starting to talk less specifically about Agile when trying to help teams. Instead they argue that visibility is everything: just make all these unintentionally hidden things visible, and most good software teams will invent their own improvements without needing to know anything about Agile.
In conclusion, TDD is practiced at several levels in the life-cycle of shipping new business value: unit testing, integration testing, acceptance testing, even deployment testing. Let's start sharing more about the other levels- the other builds, not just the development integration builds. The ones that fail most of the time, but get greener and greener as we move through the iteration: illustrating progress, quality and commitment.
What do you think?
Thursday, February 25, 2010
Tuesday, August 11, 2009
Quality Goverance goes down a little easier when automated
It's been a while since my last post. I've recently started a new job contracting for a company in the health care industry. As one of the earliest developers on the project, I was tasked with setting up some continuous integration (we decided on Hudson) and the beginnings of some software quality governance (PMD for static analysis and Cobertura for test coverage).
Our client has some fairly strict requirements in terms of static analysis and test coverage, but fortunately they have a PMD ruleset already defined- and their test coverage metrics amount to 95% coverage.
As we're early in the project, we're ramping up the team and still doing lots of 'infrastructure' work like defining ANT tasks to automate certain build tasks (although realistically that sort of thing never ends on an evolutionary project). I've never worked on a team where there was much "formal" governance- the aspirations of the team, in terms of code quality, were stated and agreed upon, but never governed with strict rules. We tended to pair program occasionally and scrum regularly so that any wide discrepancies in practices were ferreted out.
But having automated our static analysis rules by using the Hudson PMD plugin, and automating our test coverage using the Cobertura plugin, I must confess I'm liking the strict governance. Now, the build fails not just when all the tests don't pass, but also when there aren't enough tests (coverage metrics failed), or the static analysis metrics fail.
I've been a big fan of Test Driven Development for a quite a few years now, and although I've been fortunate to work on teams where pair programming wasn't out of the ordinary, it's always been a struggle to maintain high code coverage as the project evolves. Having a client that mandates challenging goals and strict governance will actually be a blessing.
As new developers join the team, it won't seem like such an artificial or arbitrary goal to have 95% coverage and strict analysis metrics. For one thing, it's being mandated by the client. But more importantly, it's now (and forever more) baked into the build- these developers will quickly understand that their work will not be allowed unless it passes the build- and in this case that means governance as well as functionality. I'm really hoping it makes the overall quality goals consistent, maintainable and achievable.
It also self reinforces the Test Driven approach. For example, say up to this point we have an initial domain model, some services, some controllers and some stubbed out objects. The moment we add a persistence layer, we'll have to test it. The code coverage rules will fail if we don't. It might be easy (and very tempting) to just whip up some simple CRUD operations on a DAO without any tests (and let's face it, with frameworks like Hibernate, we know pretty much that they'll work). But our build process won't allow us. This means we have to think about how we're going to test our persistence layer very early- do we go with DBUnit? Do we grow something ourselves? Those are nice questions to have answered (and working) before we actually have any persistence code working. Not to say that we must have it perfect out of the gate- but we must have it covered.
Having the governance automated makes swallowing those sort of pills just a little bit easier.
Our client has some fairly strict requirements in terms of static analysis and test coverage, but fortunately they have a PMD ruleset already defined- and their test coverage metrics amount to 95% coverage.
As we're early in the project, we're ramping up the team and still doing lots of 'infrastructure' work like defining ANT tasks to automate certain build tasks (although realistically that sort of thing never ends on an evolutionary project). I've never worked on a team where there was much "formal" governance- the aspirations of the team, in terms of code quality, were stated and agreed upon, but never governed with strict rules. We tended to pair program occasionally and scrum regularly so that any wide discrepancies in practices were ferreted out.
But having automated our static analysis rules by using the Hudson PMD plugin, and automating our test coverage using the Cobertura plugin, I must confess I'm liking the strict governance. Now, the build fails not just when all the tests don't pass, but also when there aren't enough tests (coverage metrics failed), or the static analysis metrics fail.
I've been a big fan of Test Driven Development for a quite a few years now, and although I've been fortunate to work on teams where pair programming wasn't out of the ordinary, it's always been a struggle to maintain high code coverage as the project evolves. Having a client that mandates challenging goals and strict governance will actually be a blessing.
As new developers join the team, it won't seem like such an artificial or arbitrary goal to have 95% coverage and strict analysis metrics. For one thing, it's being mandated by the client. But more importantly, it's now (and forever more) baked into the build- these developers will quickly understand that their work will not be allowed unless it passes the build- and in this case that means governance as well as functionality. I'm really hoping it makes the overall quality goals consistent, maintainable and achievable.
It also self reinforces the Test Driven approach. For example, say up to this point we have an initial domain model, some services, some controllers and some stubbed out objects. The moment we add a persistence layer, we'll have to test it. The code coverage rules will fail if we don't. It might be easy (and very tempting) to just whip up some simple CRUD operations on a DAO without any tests (and let's face it, with frameworks like Hibernate, we know pretty much that they'll work). But our build process won't allow us. This means we have to think about how we're going to test our persistence layer very early- do we go with DBUnit? Do we grow something ourselves? Those are nice questions to have answered (and working) before we actually have any persistence code working. Not to say that we must have it perfect out of the gate- but we must have it covered.
Having the governance automated makes swallowing those sort of pills just a little bit easier.
Wednesday, October 8, 2008
New release of SocialVino
Today I released the first revision to www.socialvino.com since the launch.
The three major new features are:
The three major new features are:
- changing your password
- pagination of browsing members
- pagination of browsing wines
Saturday, September 27, 2008
SocialVino launches!
Well, many months in the making, SocialVino is finally live!
About a year ago I set out to learn Groovy, a new(ish) programming language. For me, there's no better way to learn a new language or technology than to actually engineer something with it. Most engineers agree that just reading books or Internet articles doesn't cut it. I think that even running canonical examples like the PetStore isn't much better either.
I've found that trying to solve new problems with new software is the best way to learn. I already had a ton of experience in the digital music industry, most proudly working on Rhapsody and for Liquid Digital Media. The last thing I wanted to do was to build yet another music download site.
But I did want to work on something I was passionate about that had some interesting software challenges and the potential for fun, too. So I decided on wine.
Along the way, I changed jobs and found myself in the social networking industry. SocialVino then morphed into a social wine site, where the members, and the user generated content they provide, completely power the content.
As my ideas and inspiration shifted often, so did the technologies I was most interested in learning.
I was fascinated with Grails, a Ruby-on-rails inspired web application framework powered by Groovy but that would run on the JVM. SocialVino is a Grails application. And I'm more fascinated with Grails than ever.
I also became intrigued by the possibilities of cloud computing, as Amazon was releasing (and improving) their cloud computing platform, the Elastic Compute Cloud (EC2). And so... SocialVino runs on EC2.
On the front end side of things, the Yahoo User Interface (YUI) library was taking off, and I thought (and still think) that it has just the right combination of simplicity and under-the-hood power. SocialVino relies heavily on YUI for css and javascript.
In future posts I'll talk about more of the technology stack and the Agile development methods I used and in which I believe strongly.
About a year ago I set out to learn Groovy, a new(ish) programming language. For me, there's no better way to learn a new language or technology than to actually engineer something with it. Most engineers agree that just reading books or Internet articles doesn't cut it. I think that even running canonical examples like the PetStore isn't much better either.
I've found that trying to solve new problems with new software is the best way to learn. I already had a ton of experience in the digital music industry, most proudly working on Rhapsody and for Liquid Digital Media. The last thing I wanted to do was to build yet another music download site.
But I did want to work on something I was passionate about that had some interesting software challenges and the potential for fun, too. So I decided on wine.
Along the way, I changed jobs and found myself in the social networking industry. SocialVino then morphed into a social wine site, where the members, and the user generated content they provide, completely power the content.
As my ideas and inspiration shifted often, so did the technologies I was most interested in learning.
I was fascinated with Grails, a Ruby-on-rails inspired web application framework powered by Groovy but that would run on the JVM. SocialVino is a Grails application. And I'm more fascinated with Grails than ever.
I also became intrigued by the possibilities of cloud computing, as Amazon was releasing (and improving) their cloud computing platform, the Elastic Compute Cloud (EC2). And so... SocialVino runs on EC2.
On the front end side of things, the Yahoo User Interface (YUI) library was taking off, and I thought (and still think) that it has just the right combination of simplicity and under-the-hood power. SocialVino relies heavily on YUI for css and javascript.
In future posts I'll talk about more of the technology stack and the Agile development methods I used and in which I believe strongly.
Subscribe to:
Posts (Atom)
Posts
