Monday, September 10, 2012

JIT testing, or what to test when you think there is nothing to test

More than once it happened that you have a goal to test a new build of a product but it turns out the build is not available in time; you could:
   a) Test a product that has lower priority in your current Sprint
   b) Write some obscure automated test scripts for a product that isn't even ready yet and you haven't seen it at all but you did have 1 training session where you understood squat about the product
   c) Play Windows Solitaire until the build gets ready

Choosing any of the above is not the right answer and will quickly get you off the train where we're headed as a company. What to do?

Enter JIT testing
Your goal says you need to find bugs score in minimum of 250 and verify resolved bugs for a new product build. Ok - you cannot verify the resolved bugs as you still don't have the new product build, this part of the goal is externally blocked. However you can find bugs score of minimum 250 by testing previous product build you have available:

   1) Take the latest product build you have available; this can be a public build approved for production or it can be previous testing round build that was rejected due to a few found Critical bugs; it can also be a build recently created internally by the automated nightly build script - check the FTP yourselves and check the build version - don't wait for explicit notifications about such builds

   2) Test the hell out of the build and create new bugs that are worth 275 score points (10% over your goal); chances are that all the new bugs you find will still be there in the official new build when it is eventually submitted to testing

Once the new product build is ready, verify the existence all the bugs you found during the testing of the previous available build; I bet that most of the issues will still be there and that in the worst case scenario you may only need to find a few bugs only to exceed the score goal of 250

Other things to do while "waiting"
Don't forget that you'll still get ad-hoc requests to handle new installer testing, new website content to be checked - these are all higher priority and can be handled in parallel with the active Sprint

Developer teams will also create Patch testing PBIs. If a patch testing PBI contains Critical severity bug fixes, test it now and don't let it linger; if a patch testing PBI contains only High severity bug fixes, also test it now - there are only a few bug fixes per patch to test anyway so it won't take too much of your ad-hoc testing/support time

Things NOT to do while "waiting"
What I don't want to see is QA mailing excessively developer teams asking "When will we get the new build ready for testing" - such questions should be directed to QA Product owner (CTO) or not asked at all unless you are planning your next Sprint

No comments:

Post a Comment