Archive for November, 2011
Eric Jacobson: I have a question for your blog.
EJ: I just got my ass kicked trying to convince a project team to stop calling features done without the automation done.
ManageToTest: That’s illegal! Call HR!
EJ: I had like 10 people telling me I was wrong.
MtT: That’s interesting. So why is it wrong? Does it not need to be done? What will you do in the meantime? When will there be time to do it if not now? Why is time then “less valuable” than it is now?
EJ: Their bottom-line excuse is that automation is important, but not part of the “critical path.”
MtT: Oh really? Critical path for what? This particular feature?
EJ: They say, “Yeah, it would be nice to have it, but we can go to prod without it.”
MtT: Ah, yes, of course.
MtT: Sure you can put it into prod, but what about the next feature? When will you do automation for that? Oh, well, I guess you can put that off too.
MtT: So when will you do the automation from that first feature? Is there ever a time there aren’t any features?
MtT: What is the value of having this automation? If there is none, then they are correct. Don’t do it.
MtT: But perhaps we aren’t understanding what the value is.
MtT: The value should be the safety net of knowing that new functionality hasn’t broken existing functionality. Right now the only way to do that is by doing manual regression, and the time it takes to do that will increase over time.
MtT: So if you don’t automate, you’ll have to cut some of that manual regression out, creating risk.
MtT: That amount of risk is what you need to determine. That is the “value” of the automation.
EJ: …great stuff. I wish I would have said it in my meeting.
MtT: So you’re saying I should blog about this?
Hey Mr. ManageToTest, do you happen to have any standards or guides around Quality Control Test Plans? We have a few different QC groups doing things slightly different ways….no formal documentation on which way is the “best” way, or best practices around how to write a test plan.
First off, there’s no “best” way to write a test plan. Depending on the context, there will be more effective methods. Also, you probably need to define what you mean by a test plan, and what it is you want to accomplish with it as the definitions and needs vary between contexts.
That being said, I can provide some examples of test planning based on my experience:
I find this method to be wasteful and often redundant. Such plans are written and then never referred to again, and as projects change, these documents are rarely updated so they present an inaccurate record of what was actually done.
A search on your company’s document repository will probably yield several examples of such a document or template, so if you really want to use this style you probably already have it in practice somewhere in your organization.
In many cases, a checklist of capabilities and areas to test are appropriate. This is an excellent way to lay the groundwork for a testing session or the creation of a set of test cases. It is complementary to discussions of user stories when eliciting acceptance tests and during delivery story discussion when eliciting tasks and estimation of work. Here’s a great article from James Whitaker (Director of Testing at Google) where he describes a method of creating and organizing such a plan.
Another method is Session Based Test Management (SBTM) created by James Bach. A Charter is written that describes the goal of the testing session. A list of Charters defines the overall plan for the testing effort (and some predictive value based on the number of charters). Note that this method is a hybrid of a test plan and test cases. The act of engaging in the testing session produces the test cases on-the-fly resulting in a record of what was actually tested. The Charters completed represents your test coverage.
Any one of these ways may be effective in a particular context, or perhaps a combination of things from each of them would suit better. And there are certainly other ways of generating and presenting test plans. Again, I would stress that no method is the “best” across all contexts.