We must be able to find a perfect balance between administration and efficiency in order to achieve the goal - deliver customer needed fixes as soon as possible. Let's address specific use cases that were all encountered in under 14 days' time
Support
1) Each week Support check status of customer requested bug fixes, gets patch ETAs from developer teams - this is good and will result in good hard info that will be part of developer team SMART goals. However not all patch and bug fix requests are backed up by Product backlog items (PBIs) - this is not good and is something we must have in order to:
a) Ensure all needed customer bug fixes are properly prioritized in developer team Product backlogs; remember that dev teams are mainly Scrum oriented
b) Have hard evidence when and what bug fixes were requested right there in the TFS, easy for all teams to access and see, not scattered around in dozens of emails and forgotten about
2) If a specific PBI hasn't yet been committed
to, add new bug fix request there; this may be a change to how we did it so far (add bug fix requests only if the goal is not yet approved, otherwise create new PBI), but this change will increase efficiency and reduce your dependencies on development Product owner to approve new PBIs
3) Don't exaggerate - ask for bug fixes for only the most troubling issues for our customers, or if it can directly help our revenue; other bugs will be fixed eventually. The more bug fixes in a patch you request, the longer it will take to be done
4) Provide development Product owner with weekly product patch priorities; NOT individual bugs but actual product names and their TFS PBIs; this will help Product owner understand customer priorities better and correctly prioritize developer team goals to meet customer needs
5) Reproduce a bug and put it into TFS before you ask for a patch; if you cannot reproduce it, QA are there to help
Developers
1) You're self-managing spec-ops now - don't rely on a single person to guide each and every step of the way to deliver the requested patch to customers. There is no central synchronous checklist to follow and wait until someone else completes their action item
2) Some of the bugs requested in the patch cannot be reproduced, require too many changes and time to do, or look as though they were by design? Contact Support now and clarify them ASAP, discuss each and every special case as these bugs are customer favorites and cannot be pushed aside
3) Release notes not yet updated and reviewed? Don't wait for this step as it is an asynchronous operation - instead get the patch to the customer now
4) Always cut a label before sending it to testing; this ensures the code base is pristine after the patch is quick-tested
5) QA found a newly broken feature bug in the tested patch build? Fix it now and create a new label to send to testing, don't wait for explicit approval as the patch is not done until the customer starts smiling
6) As soon as the label is quick-tested by QA, send the build link to Support; you don't need release notes, website content or COO, CTO, CEO to approve this - just get the patch over to the customer
7) Once release notes are updated, update and rebuild the label, then it can also be promoted publically on the website. We determined that there are dozens of "silent downloaders" that do get these patches manually after they are posted on the website so we should help them as well
Showing posts with label patching. Show all posts
Showing posts with label patching. Show all posts
Thursday, October 11, 2012
Monday, September 10, 2012
JIT testing, or what to test when you think there is nothing to test
More than once it happened that you have a goal to test a new build of a product but it turns out the build is not available in time; you could:
a) Test a product that has lower priority in your current Sprint
b) Write some obscure automated test scripts for a product that isn't even ready yet and you haven't seen it at all but you did have 1 training session where you understood squat about the product
c) Play Windows Solitaire until the build gets ready
Choosing any of the above is not the right answer and will quickly get you off the train where we're headed as a company. What to do?
Enter JIT testing
Your goal says you need to find bugs score in minimum of 250 and verify resolved bugs for a new product build. Ok - you cannot verify the resolved bugs as you still don't have the new product build, this part of the goal is externally blocked. However you can find bugs score of minimum 250 by testing previous product build you have available:
1) Take the latest product build you have available; this can be a public build approved for production or it can be previous testing round build that was rejected due to a few found Critical bugs; it can also be a build recently created internally by the automated nightly build script - check the FTP yourselves and check the build version - don't wait for explicit notifications about such builds
2) Test the hell out of the build and create new bugs that are worth 275 score points (10% over your goal); chances are that all the new bugs you find will still be there in the official new build when it is eventually submitted to testing
Once the new product build is ready, verify the existence all the bugs you found during the testing of the previous available build; I bet that most of the issues will still be there and that in the worst case scenario you may only need to find a few bugs only to exceed the score goal of 250
Other things to do while "waiting"
Don't forget that you'll still get ad-hoc requests to handle new installer testing, new website content to be checked - these are all higher priority and can be handled in parallel with the active Sprint
Developer teams will also create Patch testing PBIs. If a patch testing PBI contains Critical severity bug fixes, test it now and don't let it linger; if a patch testing PBI contains only High severity bug fixes, also test it now - there are only a few bug fixes per patch to test anyway so it won't take too much of your ad-hoc testing/support time
Things NOT to do while "waiting"
What I don't want to see is QA mailing excessively developer teams asking "When will we get the new build ready for testing" - such questions should be directed to QA Product owner (CTO) or not asked at all unless you are planning your next Sprint
a) Test a product that has lower priority in your current Sprint
b) Write some obscure automated test scripts for a product that isn't even ready yet and you haven't seen it at all but you did have 1 training session where you understood squat about the product
c) Play Windows Solitaire until the build gets ready
Choosing any of the above is not the right answer and will quickly get you off the train where we're headed as a company. What to do?
Enter JIT testing
Your goal says you need to find bugs score in minimum of 250 and verify resolved bugs for a new product build. Ok - you cannot verify the resolved bugs as you still don't have the new product build, this part of the goal is externally blocked. However you can find bugs score of minimum 250 by testing previous product build you have available:
1) Take the latest product build you have available; this can be a public build approved for production or it can be previous testing round build that was rejected due to a few found Critical bugs; it can also be a build recently created internally by the automated nightly build script - check the FTP yourselves and check the build version - don't wait for explicit notifications about such builds
2) Test the hell out of the build and create new bugs that are worth 275 score points (10% over your goal); chances are that all the new bugs you find will still be there in the official new build when it is eventually submitted to testing
Once the new product build is ready, verify the existence all the bugs you found during the testing of the previous available build; I bet that most of the issues will still be there and that in the worst case scenario you may only need to find a few bugs only to exceed the score goal of 250
Other things to do while "waiting"
Don't forget that you'll still get ad-hoc requests to handle new installer testing, new website content to be checked - these are all higher priority and can be handled in parallel with the active Sprint
Developer teams will also create Patch testing PBIs. If a patch testing PBI contains Critical severity bug fixes, test it now and don't let it linger; if a patch testing PBI contains only High severity bug fixes, also test it now - there are only a few bug fixes per patch to test anyway so it won't take too much of your ad-hoc testing/support time
Things NOT to do while "waiting"
What I don't want to see is QA mailing excessively developer teams asking "When will we get the new build ready for testing" - such questions should be directed to QA Product owner (CTO) or not asked at all unless you are planning your next Sprint
Friday, September 7, 2012
Scrum tales - part 5
Patch requests have started piling up in the form of Product backlog items (PBIs) in developer team Product backlogs. Some developer teams have a new sprint starting out soon so incorporating these new PBIs into new Sprint is easy. However some developer teams have just started sprinting; what do we do now:
a) Make customers wait to get their patches until the next Sprint in 2+ weeks so we can follow our Scrum development process rules
b) Close one eye and make a small exception within the Scrum rules, add the new PBIs to existing sprint
The answer is c) neither of the above - we need to both get patches to the customers in a timely manner and not violate the Scrum rules. Here's how...
Case 1: A new developer team Sprint is about to start in under a week's time
The answer is simple - just get the new patch request PBIs approved and prioritized on time by your Product owner and incorporate it in the new Sprint
Case 2: A new Sprint has just started and it will be another 2 weeks until it is finished; approved PBI requires a patch containing High severity bug fixes only
This means that the patch will take up to 3 weeks to be finished but it will be ok as none of the defects are Critical severity in the first place
Also use common sense - if you have 2 simple High severity defects to patch which would take a few hours of ad-hoc time, do so now and don't let it linger on for weeks
Case 3: A new Sprint has just started; approved patch requesting PBI contains Critical severity bug fixes that can't wait for the next Sprint
Pause the work on your Sprint and get this patch done ASAP; you already have allocated ad-hoc time for such work outside of the Sprint time
Ad-hoc working Q/A
Q: I've spent my 2 hours of allocated ad-hoc dev time for today; should I go back to my Sprint tasks even though Critical severity bugs patch isn't finished?
A: No - focus on completing one goal at a time; this means you should work whole day if needed to get that ad-hoc patch PBI wrapped up before going back to the Sprint tasks
Q: We're starting work on ad-hoc PBI - do we need to define individual tasks in the team and possibly make a separate Sprint for this, then sprint two Sprints in parallel?
A: Don't introduce complexity when there is none. The reality is you are working on 2-3 ad-hoc bug fixes to get the patch ready; these should show up as Bug fixing tasks in your Daily status report, i.e. "PBI #12345 - Create patch for Refactor 2012 R1 with 2 bug fixes; TFS #8943 - High - Bug name"
Q: I'm going to work on a Saturday, but should I focus on my ad-hoc tasks since weekends aren't calculated in the Sprint time?
A: Assume that working weekends or holidays are just like any other working day and take your priorities in order. Sprint comes first unless there are higher priority ad-hoc tasks such as a patch requesting PBI for Critical severity bugs
a) Make customers wait to get their patches until the next Sprint in 2+ weeks so we can follow our Scrum development process rules
b) Close one eye and make a small exception within the Scrum rules, add the new PBIs to existing sprint
The answer is c) neither of the above - we need to both get patches to the customers in a timely manner and not violate the Scrum rules. Here's how...
Case 1: A new developer team Sprint is about to start in under a week's time
The answer is simple - just get the new patch request PBIs approved and prioritized on time by your Product owner and incorporate it in the new Sprint
Case 2: A new Sprint has just started and it will be another 2 weeks until it is finished; approved PBI requires a patch containing High severity bug fixes only
This means that the patch will take up to 3 weeks to be finished but it will be ok as none of the defects are Critical severity in the first place
Also use common sense - if you have 2 simple High severity defects to patch which would take a few hours of ad-hoc time, do so now and don't let it linger on for weeks
Case 3: A new Sprint has just started; approved patch requesting PBI contains Critical severity bug fixes that can't wait for the next Sprint
Pause the work on your Sprint and get this patch done ASAP; you already have allocated ad-hoc time for such work outside of the Sprint time
Ad-hoc working Q/A
Q: I've spent my 2 hours of allocated ad-hoc dev time for today; should I go back to my Sprint tasks even though Critical severity bugs patch isn't finished?
A: No - focus on completing one goal at a time; this means you should work whole day if needed to get that ad-hoc patch PBI wrapped up before going back to the Sprint tasks
Q: We're starting work on ad-hoc PBI - do we need to define individual tasks in the team and possibly make a separate Sprint for this, then sprint two Sprints in parallel?
A: Don't introduce complexity when there is none. The reality is you are working on 2-3 ad-hoc bug fixes to get the patch ready; these should show up as Bug fixing tasks in your Daily status report, i.e. "PBI #12345 - Create patch for Refactor 2012 R1 with 2 bug fixes; TFS #8943 - High - Bug name"
Q: I'm going to work on a Saturday, but should I focus on my ad-hoc tasks since weekends aren't calculated in the Sprint time?
A: Assume that working weekends or holidays are just like any other working day and take your priorities in order. Sprint comes first unless there are higher priority ad-hoc tasks such as a patch requesting PBI for Critical severity bugs
Wednesday, September 5, 2012
Patch obstacles course
Before clarifying a few use cases below, make sure to (re)read the clear set of Patching rules we already have defined
1) Support must make sure to clarify newly created PBIs which represent goals/requests for patches:
a) Specify product name and version that must be patched; this doesn't mean to specify product version where bugs were found but to specify actual version of the product that is currently available to customers and must be patched
1) Support must make sure to clarify newly created PBIs which represent goals/requests for patches:
a) Specify product name and version that must be patched; this doesn't mean to specify product version where bugs were found but to specify actual version of the product that is currently available to customers and must be patched
b) List all defects individually that need to be patched - these can be High or Critical severity bugs only - each defect's number, severity and name must be specified in PBI description; OR you can simply link TFS bugs to this new TFS PBI
c) Send FYI email about created/updated PBI to developer team AND to Product owner; if you send to developer team only then they will have to pull Product owner's sleeve to get this approved which introduces more emails into the otherwise direct process
2) Developers must review the new PBI immediately; then:
a) Know your Scrum rules – you cannot put a new and especially unapproved PBI in ongoing Sprint – do so and you will violate core Scrum rules which is primarily bad for your team as your sprint may fail to achieve original goals
2) Developers must review the new PBI immediately; then:
a) Know your Scrum rules – you cannot put a new and especially unapproved PBI in ongoing Sprint – do so and you will violate core Scrum rules which is primarily bad for your team as your sprint may fail to achieve original goals
b) If PBI contains Critical severity defects to patch, estimate if you will be able to get it done in week's time per rules by waiting for the next sprint; if not, you have ad-hoc time allocated for fixing such Critical issues outside of the regular sprint
c) If PBI contains High severity defects only, work with Product owner to get it approved in time for inclusion in the immediate next Sprint
d) Support team doesn't care what code branch you will work on, how to implement the same fixes in multiple branches, etc., they just need the patch build delivered per rules
Friday, August 31, 2012
Learn how to patch in 3 easy steps
If you're wondering who owns the product patches, who determines when a patch will be done, what will the patch cover and how it will be delivered, the answer to all these questions is summed up in one word - Support
You don't depend on CTO Development or Operations manager to forward the requests and make developers do the actual work of creating patches. Simply use the Scrum process we have set in place and get your patches delivered just the way you like them - medium rare or well done
Below you will find 3 easy steps to help you help customers
1. Identify the need
As you are directly in contact with customers you can feel their pain and determine what issues trouble them the most; no one else can do this for you, not CTO, not QA, and definitely not developers who created the issues (although inadvertently) in the first place
Ask yourself the following questions:
a) How many customers have been affected?
b) How serious is the bug at hand? (see How to determine your own bug severity)
c) Does the identified bug have a workaround you can offer now to the customer?
d) How fast does the customer need the fix?
e) How many "invisible" customers who didn't contact you could have been affected by the bug?
f) How brand-damaging is the bug?
g) Will patching the bug now help the company to keep existing customer or gain a new one?
Compile and maintain the list of bugs-for-patching candidates daily; make sure you include only bugs that customers actually need. Remember: not all bugs are created equal - a High severity bug that has a workaround isn't necessarily instant candidate for patching
2. Make the request
Now that you have a neat list of one or more bugs that obviously need to be patched ASAP, create a patch request for corresponding product: just add/update a Product backlog item directly in the product-owning developer team's Product backlog
If you are requesting a new patch, create a new Product backlog item and describe in details what exactly you need delivered - specify all bugs individually, their ID, severity and description
If you have found additional bugs for the same product that need patching, append them to previously created Product backlog item if it is still open
Send an FYI to Product owner and the developer team about added / changed Product backlog item in their Product backlog
3. Deliver
You've done your job - don't worry if or when will you get the actual patch sent to you, this is now up to the developer team (they also have a system and a set of rules to follow). However for reporting needs you should still maintain a list of all patches you requested and when you requested them
As soon as you receive the patch with requested bug fixes, directly contact the customers who reported the issues and make them happy
Q/A
Q: I need to get a patch for this Medium severity bug as customers desperately need the fix. However rules say I can only request a patch for Critical and High severity bugs?
A: Why do customers desperately need the fix? If this is so obvious, are you absolutely sure your bug should be Medium severity in the first place?
Q: I requested Critical severity bugs patched and the customer needs this yesterday. Developer team just started a new two-week sprint without the patch goal included so this patch won't be done in another two weeks - can I yell at developers to speed this up?
A: No - don't yell and don't go pulling on CTO or Ops manager skirt. Developers also have a set of rules to follow and one of the rules is that Critical bug fixes must be delivered in a week's time even if it means working outside of the Sprint during allocated ad-hoc time. You'll get your patch sooner than you think
Q: I always request new patches each Friday once I build up a list of obvious patch candidates but customers don't want to wait 3 weeks to get the patch. What can I do?
A: Start by requesting new patches as soon as you identify bug candidates for patching, not once per week - there's nothing limiting you to do this daily if needed
You don't depend on CTO Development or Operations manager to forward the requests and make developers do the actual work of creating patches. Simply use the Scrum process we have set in place and get your patches delivered just the way you like them - medium rare or well done
Below you will find 3 easy steps to help you help customers
1. Identify the need
As you are directly in contact with customers you can feel their pain and determine what issues trouble them the most; no one else can do this for you, not CTO, not QA, and definitely not developers who created the issues (although inadvertently) in the first place
Ask yourself the following questions:
a) How many customers have been affected?
b) How serious is the bug at hand? (see How to determine your own bug severity)
c) Does the identified bug have a workaround you can offer now to the customer?
d) How fast does the customer need the fix?
e) How many "invisible" customers who didn't contact you could have been affected by the bug?
f) How brand-damaging is the bug?
g) Will patching the bug now help the company to keep existing customer or gain a new one?
Compile and maintain the list of bugs-for-patching candidates daily; make sure you include only bugs that customers actually need. Remember: not all bugs are created equal - a High severity bug that has a workaround isn't necessarily instant candidate for patching
2. Make the request
Now that you have a neat list of one or more bugs that obviously need to be patched ASAP, create a patch request for corresponding product: just add/update a Product backlog item directly in the product-owning developer team's Product backlog
If you are requesting a new patch, create a new Product backlog item and describe in details what exactly you need delivered - specify all bugs individually, their ID, severity and description
If you have found additional bugs for the same product that need patching, append them to previously created Product backlog item if it is still open
Send an FYI to Product owner and the developer team about added / changed Product backlog item in their Product backlog
3. Deliver
You've done your job - don't worry if or when will you get the actual patch sent to you, this is now up to the developer team (they also have a system and a set of rules to follow). However for reporting needs you should still maintain a list of all patches you requested and when you requested them
As soon as you receive the patch with requested bug fixes, directly contact the customers who reported the issues and make them happy
Q/A
Q: I need to get a patch for this Medium severity bug as customers desperately need the fix. However rules say I can only request a patch for Critical and High severity bugs?
A: Why do customers desperately need the fix? If this is so obvious, are you absolutely sure your bug should be Medium severity in the first place?
Q: I requested Critical severity bugs patched and the customer needs this yesterday. Developer team just started a new two-week sprint without the patch goal included so this patch won't be done in another two weeks - can I yell at developers to speed this up?
A: No - don't yell and don't go pulling on CTO or Ops manager skirt. Developers also have a set of rules to follow and one of the rules is that Critical bug fixes must be delivered in a week's time even if it means working outside of the Sprint during allocated ad-hoc time. You'll get your patch sooner than you think
Q: I always request new patches each Friday once I build up a list of obvious patch candidates but customers don't want to wait 3 weeks to get the patch. What can I do?
A: Start by requesting new patches as soon as you identify bug candidates for patching, not once per week - there's nothing limiting you to do this daily if needed
Subscribe to:
Posts (Atom)