Principles of Agile Testing

A few weeks ago my company had an internal conference called “The Fiserv Leaders Conference on Testing” and I, along with some others, was tapped to do a talk on Agile and testing. We decided to write down what we feel are the principles of Agile testing (similar to the principles laid down in the Agile Manifesto). This isn’t what we ended up with, but this is my list and thus I will record it here.

Before I do, I should acknowledge this isn’t a new endeavor. Others have done this before. I particularly like this one from Karen Greaves and Sam Laing.  And I’ve certainly borrowed from these ideas in coming up with this stuff below.  I don’t think it breaks any ground, but this is where my head is currently at, and how I frame testing in my current organization.

Testing is a whole team activity:

Everyone is responsible for quality. When a story, or work item, is marked done, that means the team is agreeing that the work is done — the whole team, not just the testers. Everyone is accountable for it, thus everyone should be involved in testing. Anyone on the team can take on the “role” of tester, just as anyone on the team can program, or write a story. Working together is the best way to get the best quality work done.

Continuous feedback is essential:

Agile works primarily because of its built-in feedback loops that happen much more often than in waterfall or even in earlier iterative frameworks (Spiral, RUP). Scrum, for instance, has daily feedback via the daily Scrum. Then there’s feedback at every sprint boundary with the Review, Retrospective and Sprint Planning. And beyond that Release planning. When you include XP, and other common agile testing practices you create even more feedback loops on top of these. The Agile Testing Quadrants model expresses that quite well (and perhaps this does it even better). One thing to note is that “checkpoints”, “milestones” – things like that are NOT feedback. Those checks come too late (by definition). So, for example, organization mandated Control gates like sign offs, Pen testing, or Performance testing that must occur before a product is released are not valuable sources of feedback. They occur too late for us to react. Failures at these points almost always result in delays and overruns which decreases value. Check out this prezi for illustration.

Agile testing demands flexibility and the ability to respond to change:

Working on an agile team should not be like working on an assembly line. Products change, people change, environments change, organizations change, to expect one’s daily role or work to stay the same through all of that is unrealistic at best. A core principle of Agile is the ability to not only respond to change, but to be open to it. At a practical level, that means things like:

  • Changing the way you write tests to suit the needs of the team.
  • Adopting a new tool to solve a problem, or removing the use of a tool that’s no longer provides enough value to the team
  • Taking time from the sprint to learn and practice new techniques
  • Giving up some responsibilities or picking up new ones to improve the throughput of the team

Be a source of information:

Testers are not the assurers of quality (and honestly they never were). Testers provide information about how the product does, or doesn’t behave. And that is valuable! So provide that. Update stories with information about system behaviour. Be proactive with updates to the product owner. Continuously work with programmers to match up expectations with reality. Communicate with other teams and stakeholders who have questions about the system. And above all, have the courage to provide that information even if you think it won’t be well received!

Simplicity is a virtue:

When providing information, or writing tests, or executing tests, make it as simple as possible. Complexity is the enemy of information and should be avoided. When communicating about the system, don’t hide behind misleading metrics or test documentation that nobody reads. CYA is not a key tenet of Agile! When writing a test, remove useless or obfuscating information. For example, you shouldn’t need exhaustive steps – you are part of a team that knows your software. You’re no longer throwing it over a wall to a test team that has no experience or familiarity with the product. Execution should be as simple as possible. Most testing does not need to be integrated across services, products and platforms! You are usually testing a single behaviour within the confines of your system. If so, then make sure that’s exactly what you test – use test doubles (mocks, shims, stubs, etc.) as often as possible. Use tools or automation to make execution easier and faster if warranted. If automating, make it obvious what you are checking – name checks properly and make it easy to see the result matches the expectation. Design systems with testability in mind – make it easy (or at least possible!) to test the individual parts of the system.

  1. #1 by James Willett on June 1, 2016 - 9:16 am

    Hi Alex,

    Good post. I wanted to ask you more about “Testing is a Whole Team Activity”. I certainly agree with the concept, and I have seen it adopted in my own organisation, but I think that it is easy to underestimate the time it takes to shift the mindset of the team to this way of thinking….

    You are correct that anyone on the team can do “testing”, but organisations (even very agile ones) still tend to recruit “testers” or at least SDETs. The problem / resistance I encounter is that while “developers” can do the testing, the testers typically don’t have the skills to do the developing, so it doesn’t work both ways. While the devs are helping out with the testing, there is other work (that only they can do) that sits in the backlog.

    Isn’t it better to have the roles clearly designated in the team? Otherwise you could just have a whole team of developers that occasionally do testing on an ad-hoc basis? (i.e. whoever draws the short straw…)

    Interested in your thoughts on this!

    James Willett

    • #2 by Alex Kell on June 1, 2016 - 9:42 am

      It can be difficult. I find that if the team is committed to undergo the change to Agile, they’ve already come most of the way.
      And please don’t misunderstand me. I’m not suggesting there is no need for the role of Tester, or of people that identify as Testers, or for us to discount the discipline entirely. Quite the contrary, in fact.
      Put it this way. If you had a team of all programmers, who’s going to do the testing? Well, the programmers, of course. And would they do a good job? Would they be efficient? Would it be fast? Would it be effective? What might get missed? What would they need to learn? What skills would they lack?
      I also disagree with your other premise. If you had a team of all testers, who’s going to do the programming? Someone has to! The testers would have to do it. Add the same questions on the end.
      It’s great to have people to fill roles on a team – in fact often it is essential – these roles must exist. WHO does them and when is the part that’s fuzzy.
      You want your team to inspect and adapt, to learn. With the goal of getting better, producing more working, tested code. This principle will help you get there.

  1. Testing Bits – 5/29/16 – 6/4/16 | Testing Curator Blog

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: