Intermediary sprints

The main goal for a Release sprint is to have a product release in the end, but when we are not yet ready to release then we also have an option to initiate an Interim sprint that will not end in the product release but will instead produce one or more building blocks necessary for the eventual product release.

Interim sprints are used when the team has unknowns ahead and needs to perform analysis, research, learning in order to be able to know how to plan a future Release sprint in details.

Even if the work ahead for the product release is well known but there is too much of it for the team to finish everything in a single max-length 4-week sprint, then having one or more connected Interim sprints is the solution.

Keep in mind that the main expectation for any kind of sprint is to have finite and measurable deliverables in the end which applies to Interim sprints as well. For the Interim sprint the deliverable will not be a product released to production but something smaller - some examples are:

• When looking for the best way to find out how to develop something then do the hands-on approach: develop a prototype or attempt to build a proof of concept
• It is still OK if this Interim sprint deliverable might not be used in the final product in production as long as we learned from the exercise and can now plan enough time to do it right
• It is also OK if the proof of concept turns out to be impossible to build, i.e. due to technical limitations or external constraints that we have now clearly identified and have removed all unknowns from our path
• When fixing a complex issue or optimizing performance we need to plan out enough time to attempt several or all potential fixes or experiments that will lead to resolution or improvements
• It is OK to have limited incremental success here without committing to specific outcome or specific numbers in the end as long as we have systematic progress and can keep stakeholders updated what worked / didn't work
• When researching and need to figure out how comps are doing things, or to learn more about a subject that is new but without possibility to build a tangible prototype even if it is thrown away in the end: it is OK to just do a detailed summary on the subject or ideally a plan on how we'll approach the next Intermediary or Release sprint

How much time can be allocated to an Intermediary sprint?

• Max-length applies to all types of sprints and it can never be longer than 4-weeks
• If multiple Intermediary sprints are needed then it is OK to connect them one after another but with explicitly approved plan on how many such sprints will exist before the next Release sprint
• Sprint planning time for an Intermediary sprint cannot be longer than 1 day
• Remember that if you have unknowns that are to be clarified during the Intermediary sprint then the sprint plan can be somewhat looser, i.e. Team member 1 Day 1, Team member 2 Day 2, etc. but this is an edge case applicable to time-boxed research only - everything else 

Product release sprints

1) Sprint planning

• Team has two days max to plan out the new sprint
• This is more than enough time for a quick research of goals ahead and a detailed sprint planning
• Take your time for sprint planning, up to several hours for the entire team is sometimes needed and the entire team needs to be present to take part in sprint planning, this is not something a single person (i.e. ScrumMaster) should do
• In case we have more unknowns then the team should not recommend a Release sprint but rather an Intermediary sprint where the primary goals will be to clarify the unknowns through hands-on proof of concept / prototyping or documented plan / deliverable, or to build necessary prerequisites for the eventual release

2) Sprint

• Maximum sprint duration is still 4 weeks or 20 workdays
• Each Release sprint will be divided into two structured wholes: Work phase and Test plan execution phase
• Maximum time allocated for Work phase shouldn't exceed 3 weeks or 15 workdays
• During this time the team should fix bugs, implement features, and complete other general Product Ownership related goals agreed on sprint start
• Work goals should be made in such a way that we can drop off some of the goals during the sprint and still achieve the release: i.e. drop off some of the planned new features, fix less bugs than planned
• If the team doesn't have a Test plan for the product that is up-to-date then one of the work goals needs to be to update the Test plan: this work is owned by the team SA but needs to get a buy-in from the entire team
• Maximum time allocated for Test plan execution shouldn't exceed 1 week or 5 workdays, and should match Test plan execution time estimate in team days
• During this time the entire team will focus on testing the product build in development by following previously prepared Test plan and by splitting test areas among all team members, both developers and SAs
• Test plan execution is "All Hands on Deck" time meaning that no team member is allowed to work on anything other than executing regression test to figure out quality of the current build
• There is no more "inter-team cross-testing" meaning you shouldn't send your build to other teams or other SAs to test it for you
• This doesn't mean that inter-team cooperation isn't allowed - actually it is still a good thing to do cross-testing with other teams especially when other teams are relying on your products and may be affected with the new release
• Devs must still execute "internal cross-testing" and verify each other's bug fixes (i.e. Dev1 tests bug fixes for Dev2, Dev2 for Dev3, Dev3 for Dev1 and final report is sent to SA)
• When the sprint ends the team will figure out how many bugs were found that SA will report in the bug tracker, and will estimate how much actual work remains to release the product
• Not all bugs found are blocking bugs (more on this soon)
• We just need blocking bugs fixed in order to release the product
• At this point the team will know if blocking bugs can be fixed within the Release phase time (see below) or not and will decide one of two things: we are going to release or we need to start a new release sprint in order to fix remaining blocking bugs that will take more than one week's time
• We need to watch out not to begin Release phase if we are uncertain that the remaining blocking bugs can be fixed within the allotted time: it is of no consequence to begin a new Release sprint, but there will be many affected stakeholders if we decide to enter the Release phase and fail to release on time

3) Release phase

• This is instead of what was previously known as "Test phase" but with main difference being that we are no longer spending time to test the product build for the first time but are focusing entire team efforts to release the product
• Release phase can take up to 6 days including the release day which is also considered the first day of the new sprint planning phase
• SAs no longer invest time into test plan or bug prospecting efforts as this is supposed to be already finished during the sprint
• Devs fix only blocking bugs and nothing else
• SAs verify fixed bugs only until we ensure there are no more blocking bugs
• Remember: the ultimate goal is to release a build that is equal or better in quality than the current one in production, nothing more, nothing less

The art of project management

Ok, so you've figured out what work to subcontract and have found a potential freelancer who will consider doing the work - congrats on learning the art of subcontracting basics, now comes the actual project management

As we already determined that every member of our product team is not just a PO but also a PM, below are a few PM operational algorithms that will help you to ensure your project is successful


Project takeoff

  1) When you have project specs vetted by the team (SSE, devs, others as needed), send the specs to the selected tried out subcontractor, or send the specs to Ops for assistance in finding a subcontractor with the best bid

  2) Subcontractor will usually have some questions about the project specs before providing work estimates
    a) If the answers to the questions are already covered in the specs that means the subcontractors didn't bother to read through the specs and you can dismiss them
    b) If there are no questions at all or no questions you hoped to hear then make sure to ask a few "control" questions of your own, i.e.: how do you plan to implement something, what do you suggest for a solution for this problem, which technology will you use to accomplish this, did you work on a similar project before, etc.
    c) If you get some very basic questions about the project that should've been covered by the specs already that means that the specs aren't good enough and my suggestion is that you don’t waste time in ad-hoc discussion with the subcontractor but go back to step 1) to rewrite/update the specs to cover all important information and then send the new specs to the subcontractor

  3) After the subcontractor provides work estimates and project ETA, perform a sanity check
    a) If you don't agree with some of the estimates don't challenge them directly but simply ask for a more detailed breakdown and explanation how did they come up with such an estimate
    b) Be constructive when talking with subcontractors, it isn't your goal to reduce the estimates but to make sure both sides are fair and professional: you are an employer but you need to treat your subcontractor as your partner

  4) Once the project estimates are good define project milestones as these are critical steps along the way that will allow you to track the progress and to avoid situation where you reach the project deadline and figure out it isn't done right
    a) Although we don't pay out $ when a project isn't done or done right, we still have growing opportunity cost caused by the delays which can be greater than the actual project cost
    b) Each project can be broken down into several milestones with no exceptions (like a sprint PBI can always be broken down into smaller tasks)

  5) Get in touch with Ops for assistance with securing project finances

  6) As soon as you have a formal approval for the project, agree on a hard date with the subcontractor by when the project will be completed, and by when the first milestone will be delivered


Project cruise

  A) If the subcontractors have any additional questions about the project at any point along the way, make yourself available to help out as soon as possible
    a) Insist on written communication whenever possible, especially if you discuss changes in the project requirements; use Skype only when it will save you time but have all project agreements written down in case we need them later on
    b) #EveryoneThinks applies to the subcontractors as well - don't solve problems for them but instead always ask for them to suggest and explain the best solution that you'll review and approve or suggest differently

  B) If you uncover additional work is needed for the project along the way that wasn't covered by our original requirements go ahead and immediately discuss project appendix estimates with the subcontractor and let Ops know to assist you with financial approval
    a) Be fair - if you didn't ask for that work in the original specs then don't insist for subcontractors to do it for free

  C) Make sure you know the upcoming milestone ETA as well as the project completion ETA at all times
    a) If the subcontractor misses an ETA you need to send a follow up immediately
    b) If you don't hear back from the subcontractor send another follow up and add Ops to CC to help out
    c) If someone wakes you up in the middle of the night you need to be able to immediately provide project ETAs and status

  D) #EveryoneShares - be transparent towards all stakeholders (i.e. Facilitators): if you are asked about project status then you're already late
    a) Push information when a milestone is approved
    b) Push information when a defined ETA is changed for whatever reason
    c) Push information when additional project requirements are uncovered

  E) As soon as you receive a deliverable from the subcontractors prioritize reviewing it and providing feedback
    a) If you need some time to review the deliverable provide the subcontractor with an ETA - what you expect from the subcontractor about giving and honoring ETAs applies to you as well
    b) Check deliverable conformance to specs, functionality, quality, performance, code where applicable
    c) If something isn't good enough send it back with sufficient feedback
    d) If a deliverable meets all of the milestone acceptance criteria inform Ops and the subcontractors about it to cover the financial side of the milestone


Project touchdown

  1) Project is officially complete as soon as all milestones are approved and all project requirements are fulfilled - you determine when this happens by giving away final project approval
    a) Don't give an approval if you haven't allowed all team members (devs, SSEs) to review the deliverables and to report all found issues, provide feedback
    b) Don't give an approval just to end the project sooner if there are still remaining issues - be persistent for the subcontractor to resolve all of the known issues
    c) When in conflict with the subcontractor whether an issue is a bug or an additional request feel free to consult with Ops, but also remember that we need the project 100% completed as soon as possible with the highest possible PUQ so if a small additional project appendix will get us there don't think twice about it and err on the subcontractor side
    d) Always remember: the more things subcontractor didn't finish the more work for you

  2) Never stop thinking about additional work you can subcontract, especially if you're happy with the subcontractor
    a) Spin off a new project immediately if you already found additional work needed for an R2, R3, Rn down the way
    b) When applicable ask the subcontractors to do some comp research on their own and to suggest any missing features, areas where we can improve the project - senior guys who are looking for additional work will be happy to help you out with this


Key things to remember

  1- You are the project manager and you own the project from start to finish
  2- Ops is always there to help you out with administration and PM advice whenever in doubt
  3- Don't be afraid: Failure is ok, fear isn't ;)

The art of subcontracting

When a fully staffed product (development) team is unable to achieve all of the team goals within an acceptable time frame, the team is faced with two options:
  a) Keep developing on your own and work long hours until the goals are achieved or at least until new challenging goals show up so you can work more long hours again
  b) Develop smarter by temporarily increasing team velocity through subcontracting: outsource and manage development projects instead of developing everything on your own

Each team is by definition a Project management team and each team member is a PM as soon as you start thinking outside of the confines of your own 10 typing fingers

What does "subcontracting" mean in practice?
  1) Identify potential projects by defining the needs in your team where you need help
  2) Prepare functional/technical requirements and/or specification for the project
  3) Leverage help from the Ops team to find the best external freelancer/team to complete the job
  4) Communicate with the selected freelancer/team, review their progress, provide work feedback and set your expectations
  5) Give your approval (or not) for parts of the project as it progresses, and for the entire project once it is complete
  6) Implement deliverables in production

What all can be subcontracted?
  a) Reusable UI components, shared between teams or products
  b) Backend libraries / engines / frameworks, either for internal team use or for multiple teams
  c) Modular features for owned products, i.e. an externally developed functionality that can be easily "plugged" into the product
  d) Fully integrated features for owned products, i.e. subcontractor gets full access to a separate product code branch and adds additional functionality directly
  e) Entire new products including backend functionality and frontend UI based on our standards
  f) Test automation helper apps that are unaware of the product, i.e. data generators
  g) Product-specific test automation helper apps specific to your product's interaction design

We can subcontract everything that we find complex?
Not just complex but even if you know how to do it internally and don't have enough time to do it (remember: success = results / time)

What's the difference between functional and technical requirements?
Functional requirements are your concise expectations of what you need completed

Technical requirements are your general technological expectations / tech limitations of how you want the project done

Requirements are short, generalized documents that provide freedom to outsourcers to figure out the tiny details along the way

At a minimum each project needs to have at least functional requirements defined

What's the difference between functional and technical specification?
Functional specification is a more detailed document providing more thorough expectations, i.e. interaction workflows, UI mockups, etc.

Technical specification is a technical minutia document which can define technical workflows or even entire development project UML

Detailed specifications are optional - they can either help our or make a project unmanageably complex depending on the type of the project

How should SSEs be involved into subcontracting?
Everyone in the team needs to spec-test functional requirements/specification, especially team SSE prior to starting a new project

Once a subcontracted project deliverable comes in such as an UI component, new product feature, new product, test automation helper, etc. SSE should review and scrutinize the deliverable prior to acceptance as if it was developed internally

How much work can we subcontract?
This is up to you mostly; as long as we have resources available to dedicate to outsourcing work you'll be able to suggest new projects for subcontracting and will get assistance from Operations to get started

Release process and test phases

Release process covers 3 distinct consecutive test phases for which product teams (devs and SSEs) are in charge:
  1) Design and usability testing
  2) Continuous testing
  3) Static/release testing

Each team must have a clear test plan that will allow for systematic advancement between subsequent test phases culminating in a product release

In a good product release scenario a team should never regress from one test phase to a previous one - some guidelines follow to explain each test phase


1. Design and usability testing
During this phase the team is testing new features, changes in a product, and is working towards locking down the product interaction design

Usability testing helps make sure the interaction design is optimal and customer friendly - the team should leverage feedback from as many external resources as possible: company stakeholders, power-users, other product teams, etc.

It is ok to update interaction design, make functional UI changes during this phase, but not after this phase is locked down


2. Continuous testing
Once interaction design is locked down, the team must focus on finding as many bugs as possible and fixing them all as soon as they are found and reported

Usually this phase will help to clean up product functional bugs, regression bugs, and make sure the build is stable enough to go on to the next and final phase of the release process (for more details about Continuous testing vs. static testing read this article)

At this time we should have no usability or interaction design bugs reported, however if it happens that such a bug is found then the team must stop continuous testing and immediately regress back into #1 Design and usability testing phase


3. Static/release testing
At this time the team cannot make multiple product builds meaning the code must be fully locked down, and that only one build can be tested

If the previous two testing phases were covered well then the static testing will only reveal limited number of functional bugs

For static testing round #2 the team should fix only the most critical bugs:
  a) Regression/broken bugs
  b) Bugs that will clearly make the new build considered worse than the public one

If done correctly there should be no static testing round #3, but if it does happen there can be no round #4 so product release is mandatory after that

If it happens that there are design bugs or multiple "critical" bugs found during static testing then the team must stop static testing and immediately regress back into corresponding previous test phase


At any time when a process regression happens this only means that the previous testing phases weren't done well and the team must update/improve owned testing plan in order to prevent process regression from happening in the future

Remember that each team gives its own product release recommendation at the end - good litmus test question for a release: "Is this product build better than the previous one?"

The only thing that a team must follow at all times is this clear release process and must be able to provide feedback to all stakeholders about:
  a) What is the current product testing phase
  b) What is the ETA to go into the next phase / release

Blitzkrieg testing

(Wikipedia) Blitzkrieg (German, "lightning war") is an anglicized term describing a method of warfare whereby an attacking force spearheaded by a dense concentration of armored and motorized or mechanized infantry formations, and heavily backed up by close air support, forces a breakthrough into the enemy's line of defense through a series of short, fast, powerful attacks

Although we're striving to automate as much development processes as we can, in some cases of complex product releases we still need to ensure products pass human / manual testing so there are no broken functionality issues we missed

What happens when we have limited number of testers covering multiple products from multiple development teams? As we never test in parallel, one or more products inevitably gets delayed, we pile up technical waste and lose agility

Consider a typical development scenario:
  1) On day 1 Dev team A works for 3 weeks and builds Product A for testing
  2) On day 2 Dev team B works for 3 weeks and builds Product B for testing
  3) The same software testers cover both Dev team A and Dev team B
  4) Software testers receive Product A and Product B for testing on week 4
  5) We prioritize products based on their ROI, date of last release, customer requests - one product still gets delayed and there is always at least one customer that we've let down

Instead of losing momentum, we can now add air support (developers) to help mechanized infantry (software testers) and blitzkrieg the product for one day

Years ago in early days of ApexSQL each developer was owning exactly one product and was also responsible to self-test the product before sending it to testers. Without this mandatory self-testing we've regressed to not doing any self-testing prior to sending products to testing

Consider different step #5 above:
  5) Dev team A works together with software testers to blitzkrieg-test Product A for one day and release it as Interim build or even production build if software testers agree
  6) Dev team B works together with software testers the next day to blitzkrieg-test Product B for one day and release it as Interim/production build

Critical note: developers cannot report bugs "internally" or fix them on-the-fly: all bugs must be reported to software testers who will then create bugs in the system and prioritize them accordingly

Also to avoid confusion - some online blogs say that Blitzkrieg testing = spot-testing; there is a major difference: spot-testing is quick testing by a lone scout, while Blitzkrieg is a short but major quality offensive

Developer Vs. Programmer

At ApexSQL we're making killer tools for SQL Server, and to do so we're proud to have strong product and website teams currently consisting of developers and SQAs. We have no programmers in our teams and we don't need any.

What is the difference between a programmer and a developer?


Definition

Programmer is a grunt receiving orders from a team leader and/or project manager working on a piece of code outsourced for someone else's solution and based on someone else's plan.

Developer is a spec-ops, lean-mean-development-machine part of a small band of heroes owning and building top-notch solutions, come hell or high water.


Examples

Programmer has many bosses: a senior, a team leader, a project manager, etc.
Developer has only one boss: the customer.

Programmer wants to write a program or some code.
Developer wants to create the best solution for the customer.

Programmer comes to work and works for 8h.
Developer has incremental results each day regardless of hours worked.

Programmer asks questions.
Developer makes suggestions.

Programmer speaks for himself as one from the team.
Developer speaks to the team and for the team.

Programmer reads roadmaps and writes code to accommodate roadmaps.
Developer makes roadmaps and works to accommodate customers.

Programmer doesn't know the competitors and tends to be surprised.
Developer knows what competitors had for breakfast and what they plan for dinner.

Programmer schedules meetings and gives updates to stakeholders when asked to do so.
Developer has an active com-link with battle buddies and keeps stakeholders constantly informed of mission status.

Programmer learns when new skills are needed for work and/or is trained by others.
Developer learns organically and shares the knowledge by training others.

Programmer works with other teams to share personal workload.
Developer works with other teams to lessen everyone's workload.

Programmer strives not to fail the sprint.
Developer strives to add more work on top of a successful sprint.

Programmer waits for SQAs to test code changes and report bugs.
Developer self-tests new code and makes it tough for SQAs to find bugs.

Programmer is unsure about performance of created tools.
Developer perf-tests regularly and knows that owned tools are top-performers in the market.

Programmer expects of SQAs to write test cases and to regression test everything.
Developer cooperates with SQAs to write unit tests, to implement detailed logging and to automate regression testing.

Programmer uses 3rd party tools to speed up work.
Developer uses our company tools to speed up work and to help improve the tools in the process.

Programmer "overcommits", "cannot reproduce", "cannot fix", "must ask someone else".
Developer gets the job done. Period.

Efficient customer support chain

To have an efficient chain of help from customer and all the way down to developers teams should make sure to not skip steps and seek shortcuts as they will prevent internal growth but also unwillingly consume more of team time on the long run making the support inefficient and time consuming

 

 

What does this mean in practice?

 

   1) Support - don't be a hero, you cannot isolate all bugs by yourselves as this will suck up your time and you won't be able to work on anything else that day; pass complex support cases down to QA to isolate and reproduce the issue when you define the problem itself; not passing cases down to QA will also deny them of the opportunity to grow hard skills and help you better/faster in the future

 

   2) Support - make sure you do define what the problem is about when the customer first contacts you; this means you shouldn't forward received customer email directly to QA if you don't know the answer immediately - try to at least define the problem before pushing it down the chain to QA to isolate and report a bug

 

   3) Support - some of you have tendencies to contact devs and discuss directly with them whether a customer report is a bug and whether or not devs can fix it easily; don't go to devs but go to QA instead because going to devs without a defined bug will only waste time for both teams

 

   4) QA - although you no longer have unattended assistance metric this doesn't mean you should contact devs for every single support case and definitely not repeatedly when similar issues show up - I can hear directly from devs which QA team is doing better job isolating technical support case bugs

 

   5) QA - you can and should contact customer directly when you need additional help to isolate a complex problem; contacting customer through Support will only waste time of both teams and prolong solving the case

   6) Devs - help QA grow by giving them pointers what to research, check and isolate; if you always roll down your sleeves, dig down into code and isolate a problem for them, you'll be stuck doing this indefinitely while we'll have consequential impact on our product releases

Clueless QA Q&A

Unless you're finding tons of bugs each day and feeling primeval hate toward high entropy and other teams' imperfect work, you'll want to check out some of the use-case based Q&A below

Q1: I found a bug but my team members say it is not actually a bug and that I shouldn't report it
A1: What would a customer say after encountering the same issue in production?
Just ignore your team members, report the bug and let product owner worry about resolving reported bug one way or another

Q2: Product issue I found obviously seems like High priority but I'm afraid that team members and/or management will think I'm just inflating my monthly found bugs score
A2: What would a customer say after encountering the same unfixed issue in production that you didn't elevate?
Set the priority guidelines and use common sense, but always err on the side of customers. Don't worry, if I see priority inflation you'll hear from me

Q3: I used our Product B to help me while we're testing Product A and I found a bug in Product B; I don't think I should report Product B bugs at this time
A3: What would a customer say after encountering the same unfixed issue in Product B?
I don't remember when I saw some product bugs reported outside of regular test rounds - remember that everything is always in testing: all products, entire website, keyboard you're typing on, chair you're sitting on, this blog post you're reading now

Q4: Support asked me to help out to isolate a customer reported bug but I don't think I should talk to customers as my English skills aren't so good
A4: What would the customer say if the bug isn't isolated and isn't fixed in the next product version?
Think about it - customer is giving you a free bug and they are usually High priority, all you need to do is ask for details

Q5: I don't like how our new product is designed because it is confusing and hard to use, however it must be how developers, support, management and God thought it would be best so I won't report this
A5: What would the customer say to the same confusing or hard to use design?
Raise such issues early as bugs, if you think something is wrong then it most certainly is

Q6: One of the product menu items doesn't seem to work, but this is less used part of the product so it doesn't deserve a Test Case because Test Cases should only result in High bugs
A6: What would the customer say after encountering unfixed issue in production because if was simply missed and not reported as there was no corresponding Test Case?
Note that customers don't give a damn about our internal Test Case rules
 
Q7: I'm concerned that the bug I found cannot be fixed by developers and it should be fixed by another team so I don't think I should report it
A7: What would a customer say after encountering the same issue in production?
If you're so concerned about developers, ask them if they'd like a coffee and a cake, or just a foot massage instead of a bug report; OR you can just forget entirely about who should fix a bug and focus on reporting the bug itself
 
If you didn't notice a pattern above how to resolve QA concerns, please contact me directly to organize a special training for you

Concise bugs for improved revenue

There's a whole chain of inefficiency spawned by each poorly qualified bug:
   1) QA reports a bug with a title not matching the actual issue close enough
   2) The bug slips through spot-checks with incorrect priority assigned due to inconclusive title
   3) Developers spend time to fix lower priority bug or miss fixing higher priority bug
   4) Support team cannot fix/define release note correctly and spends much time including getting help from QA
   5) Marketing team spends time copy-editing poor release note grammar or semantics
   6) Finally customers get lower quality release or delayed release with incomprehensive release notes

Solution - QA

Peer-reviews: if devs can peer-review code, QA can also peer-review new bugs
   a) Make pairs, i.e. team member 1-2 and team member 3-4
   b) Define review schedule - each day, just before sending test summary (this is up to you)
   c) Suggest corrections to your team pair about all bugs that don't look correct

Once we get the final summary, both the bug owner and the bug reviewer will be held responsible for poor bug title, incorrect priority, stds issues, etc.

Apply the same to new test cases

Solution - Support

As prime owners of release notes you shouldn't bow your head down and keep nagging while rewriting poor release notes
   a) For each release note you need to open a corresponding bug in order to rewrite it, write down the bug owner name
   b) Compile a simple rank list (leaderboard) 1, 2, 3, 4 - from most to least frequent bug owner who you needed to correct
   c) Include the leaderboard along with each corrected and approved release notes

Pick up the rifle v2.0

Not sure who owns specific task, system functionality, who has a ball now? The simplest and usually the correct answer to this is You do!

A couple of things that all Scrum teams encounter is a task blocked by an external team - i.e. waiting for question feedback, waiting for review, waiting for approval, waiting for... Does this mean that the external team owns the task now? No - You still do
How to unblock such tasks? There's always a way, here are a few:
   a) Talk about or at least mention all blocked tasks within your team every day on Daily Scrum meeting
   b) Help ScrumMaster to summarize and report all blocked tasks to Product Owner after each Daily Scrum
   c) Make sure you have an ETA for reply from the external team and remind them early and often
   d) If not waiting for approval then attempt to workaround, try to resolve the task yourself vs. waiting indefinitely

Sprint cannot fail because a task was externally blocked because You own the task and your team headed with ScrumMaster have the ultimate responsibility to unblock it on time


What about Scrum unrelated system ownership - who owns support forum, who owns product source code, who owns posted defects? One would think these are all straightforward and implicit but let me still clarify by taking the latest as an example - ownership of all posted defects

Simply think of a posted defect the same as a QA owned Sprint task:
   A) All defects are posted by QA ("tasks" are created by the Scrum team itself); others teams may in some cases create a few defects (Support for customers, developers for engines only) but defect priority ("task" estimate) is provided or modified by the QA team
   B) Defect creator is current owner of the "task" and must follow through to the end - provide additional details, retest, close the defect
   C) If a defect owner is not available to update the defect ("task" is blocked), QA team have the ultimate responsibility to unblock it by taking the ownership vs. waiting indefinitely on external teams to act
   D) If there are defects created by Support and unverified, posted by former colleagues, made obsolete by new product versions, who's "sprint" will fail if they remain blocked indefinitely?

Product release notes - why do they matter

Everyone involved in pre or post-production is writing product release notes: developers, QA and Support. In the end release notes for all products appear consistent and are published following carefully constructed set of internal writing standards. Here's an autopilot plan listing release notes creation process in top-down priority by owners


Support

You are the direct owners of product release notes; they serve as proactive help and guidance for our customers by letting everyone know what we improved, changed and fixed in the new product releases. Having good release notes has multiple benefits:
   a) Satisfied existing customers
   b) More new customers
   c) Less need for later reactive support

This means that you have the final and ultimate responsibility to polish the release notes for all products before the release, make them consistent and easy to understand for all and push to production. To do so, use all the help you can from other teams (QA, developers) to clarify required technical details and rewrite the release notes as needed

Release notes standards are there to make all release notes consistent, not to create more bureaucratic overhead; if you see a way to improve it by making release notes simpler and easier for customers to understand - act on it now


QA

You write the most of the release notes from scratch as soon as Bugs are reported which basically makes you the main authors of 90% of all our release notes. Writing good release notes has multiple benefits:
   a) Release note text can also be directly used as Bug title - two flies with one swat
   b) Bug title matching well written release note will make developers understand bugs more easily and will pull on your sleeve less often
   c) Save time for Support which in turn allows them to resolve more reactive support issues, to forward less issues to you and thus save your own time


Developers

You are the first and the last line of defense - you consolidate all release notes for a new product release into one file on start and incorporate them back into product after Support team reviews them

Yes, you still have to write all enhancements and changes as no one else knows what has improved and changed in the product better than you. Additionally you have the responsibility to incorporate final release notes back into your product and to make sure formatting is consistent across all products

As Everyone Serves, your main job is to provide prompt technical assistance to other teams when they need to write a good and simple release note


Common suggestions

   1) When in doubt how to write a release note just put yourself in customer's shoes and see if an outsider will understand it after a single read
   2) Make all release notes as concise as possible - they are not help content but new product version proactive one-sentence guidelines
   3) Describe anything and everything that has changed and especially improved in new product releases, even if you accidentally improved performance by a few percent
   4) If not sure how to write a release note contact other teams for assistance - don't write a poor and inconclusive release note as it will eventually come back to you

Tune in to correct bug frequency

Not all bugs are created equal which means that some are "more equal" than others, especially those with higher Probability / Frequency rating; but how to determine the right frequency?

Here's what we found recently:

1) One of the main product features doesn't work at all when used on a specific database / a specific SQL script; this is High severity as the core feature is broken and this is always reproducible on this specific database / script it will remain a High bug?
No, the frequency here is Sometimes or even Rarely depending on the probability that customers will use that specific database or script

2) Product functionality worked perfectly during the first few repeated tests, but then it just stopped working. After OS reboot, the same scenario repeated; this is High bug as the feature is not working but as not always reproducible it will be Medium?
In such cases you need to stick a bit more with the bug and focus on isolating the exact individual steps that led to feature no longer functioning correctly. In case you isolate the bug cause better to reproduce it more Often / Always, it may even be a High bug

3) After changing some default system settings / stopping product background service / installing 3rd party kernel mode driver, our product stops working correctly and throws errors = High severity; this is easily reproducible in 100% of cases when following the exact steps which means frequency is Always?
Ask yourself how many customers will do what you just did - hack or tweak a default system or software installation or install that specific 3rd party software; the answer is very few, including those power users who like to play with everything - this is the actual probability of someone repeating those steps and it matches Rarely / Sometimes frequency corresponding to a Low/Medium bug

4) There are several UI standard violations on the main Options dialog as standard buttons are missing - this is Low severity and it is obviously reproducible always so this is a Medium bug?
Yes, but your reasoning is not right - although a Low severity issue it will be encountered by majority of the customers as present on high profile product dialog which then makes the frequency Often / Always

5) Several icon inconsistencies on the main product window when compared to other products - Low severity but this will be seen by all customers as this is on the main product window so Medium bug?
Recheck this - icons are on the main window but are you that certain that the majority of the customers will actually notice slightly different icons and text in the main menu / ribbon bar compared to other products? I say this will be noticed Rarely if at all

To summarize: frequency/probability should be interpreted with some thought vs. literal assumption that if the icon is "always different" or the button is "always missing" it corresponds to 100% or Always frequency

Optimize for success

"With great power comes great responsibility" - Stan Lee

Self-management can be a double-edged sword without discipline and a plan. Many teams including QA now have SMART goals to guide you, however you must guide yourself on a daily basis in order to achieve monthly SMART goal expectancy

Let's focus on the latest real-life use case:
   1) Testing weekly plan was defined: use up to 2 hours per day to test patches and focus on testing our new enterprise product for the remainder of the day
   2) Testing weekly plan was refined and approved - by the end of this week we'll have 2nd testing round of enterprise product wrapped up
   3) Week is almost over, however enterprise product 2nd testing round has just started and will be postponed for 3-4 days

What happened? Here's what I heard:

   A) "We had too many patches to test and this took a lot of time"
These patches were preplanned in the weekly test plan - it is not an excuse to violate weekly test plan. Our goal isn't to fully regression test each patch and verify its readiness for production but to focus on verifying those few (usually 1-2) bug fixes, spot-test core functionality and get it out to the customers who requested it

   Solution: dedicate a fixed chunk of time for testing each patch build, up to 1 hour; first verify fixed bugs then spot-test core functionality and finally send the patch testing summary. If the whole QA team found no core functionality issues during this time, it is highly unlikely that the single customer who requested the patch will find any; if there are new issues, we'll quickly re-patch without wasting much time

   B) "We had too many Support team forwarded cases that we stuck with for a long time in order to not forward them to developers"
What I'm seeing is ~82% of Support forwarded cases handled within the QA team which is way above 50% SMART goal; although this will be measured more precisely soon and is also an important goal, you must optimize your time better - taking 4-5 hrs to stick with a single support issue is an overkill and will inevitably cause your other SMART goal (Zig score) to suffer and our products to be late to production due to testing delays

   Solution: dedicate a fixed chunk of time for sticking with a support issue, especially if you are over the 50% SMART goal expectation for the month. Discover your own diminishing return point and use it to balance out the SMART goals when you make no progress with the support case at hand

   C) "We had many ad-hoc issues that piled up and took more time than actual testing, developers needed help with specific bugs, new team member needed guidance, there were team planning meetings, bugs needed priority corrected as we updated severity guidelines, Skyfall is premiering in the movies this week"

   Solutions:
   a) Dedicate a fixed chunk of time for team planning meeting (learn from Daily scrum) - 30min max
   b) When creating bugs, explain them in more details so devs don't ask you to clarify them - this will save time for both you and the devs on the long run
   c) Don't go to see the new James Bond movie until you have achieved daily SMART goal of at least 50 Zigs
   d) Before finishing a workday stop for a minute and ask yourself: "Have I achieved all my daily SMART goal expectations and are we on track with the weekly plan?" If the answer is yes, go ahead and have a scary night watching Paranormal Activity 4

Comprehensive test plan - PACT

SMART goal provides much more internal team organization flexibility compared to SCRUM where Product owner defines clear individual goals and expectations; however increased flexibility without planning can also cause disorganization and make your life more complicated

There is no singular Product owner for SMART, but why not consider a weekly team agreement / weekly plan as the "Product owner" to clearly guide the "What's" (incremental goals, or "what needs to be done") in the team?

Define weekly Priorities, Allocation, Continuity and Thresholds plan / a PACT to guide you as a team

Note that below are guidelines and you need to define and send your own weekly PACT plan and table

Priorities
Every bug leads one step closer to accomplishing your SMART goal; however some bugs need to be found before others:
   1) What are the main test priorities you need to work on? Check Production schedule what products are expected to be delivered to testing; contact developer teams to get direct feedback if there are unplanned changes to the schedule you don't see; also remember JIT

   2) How to prioritize main product testing? Focus on product ROI: enterprise products first, then developer tools and finally community (free) tools

   3) Any ad-hoc test priorities? Test patches and engines before regular product releases as this can usually be completed quickly; engines are usually needed for a specific main product release

   4) Quick-testing: always break to check new installers and website content

Allocation
There are finite number of test engineers and so many products and features to test. Parallelism is our enemy as we don't need test summaries for 4 products at once but one test summary at a time as soon as possible
   1) Unless you have a strong reason how you can improve efficiency by splitting the team to work in parallel on different test deliveries, focus on everyone testing one deliverable at a time

   2) Make testing for a single deliverable feature set circular in order to reduce the number of false negatives (and increase the number of Zigs) - tester A tests feature A while tester B tests feature B; then tester A tests feature B while tester B tests feature A

Continuity
We can have between 1 and 5 product test rounds. The first test round is usually the one with the most low hanging Zigs to pick and to put in your Zig basket. However the fifth test round is as important as the first one even though "What" to test is different
   1) Plan for the longest first test round, especially if you have a completely new product to test; always specify how long will the testing last as there won't be second chances to [regression] test all features from scratch

   2) Push the testing into the next week as necessary but always specify why

   3) Subsequent test rounds should be short but long enough to cover all fixes and changes made by the developers since the last test summary

   4) Final round is always #3 (#5 for new products) - no matter what you find there, the product will be released so think twice before deciding how much time to spend on this one as you cannot extend the testing further

Thresholds
You have 3 new product builds to plan testing but how will you know when to stop testing one and move on to the next one?

Actually I'd like to hear some of your suggestions here and then I'll update this post; there are many ways to define thresholds in testing but also don't forget that you have a SMART goal that must be achieved each month as it is reset to 0

Testing the hell out of one product as it has easy to find Zigs and glancing through another one is not an option as we will easily detect false negatives for the latter one: bugs are always there since developers inadvertently ensure this is true, you just need to find them

Back to the test drawing board

We've been cutting labels and sending products to testing for years - both developers and QA know the rules implicitly without even looking down at written guidelines. What scared me the most this week is that when system rules are violated and there is a massive confusion on both developer and QA sides, one of the arguments was "we've been doing it like that since I joined the company"

Nobody likes reading the rules, operation manuals and workflow guidelines, but still they are there for a reason. If you forget about them, you'll be hearing from me soon

Let's get a few things straight:

   1) Devs, there's no more Continuous testing; you cannot just send in a new major build without some internal testing and expect to have QA determine if the product can start or not, then find/fix a few bugs each day until the build is stable for real testing; instead conduct a few days of internal testing (plan it in your Sprints) to pick the low hanging fruit, resolve critical issues immediately (product doesn't start, main features don't work) and write down all non-critical ones to forward to QA along with the first label to log down to bug tracking system

   2) Devs, make sure you always send modified product version to testing; we cannot have two builds with identical versions that are functionally different just because engine takes too long to rebuild. Consider updating the way engines are referenced and versioning strategy, suggest solutions proactively; Everyone leads - proactively identify problems and solve them, don't ask questions but make recommendations

   3) Devs, always send an official request for testing when a new build is ready; specify testing areas as detailed as possible even when it means you have to roll back your sleeves and write detailed guidelines for back-end high-tech stuff that QA might not understand on their own; Everyone shares - communicate widely and effectively, educate your customers

   4) QA, it IS your job to determine if the product can start or not; Everyone serves - treat everyone as a customers and take personal ownership of their issues vs. "it's not my job" attitude; if you got a version that doesn't start at all assume that it did work on devs' side and that it is your job to isolate on which systems product works and on which it doesn't, then report Bugs

   5) QA, report all found Bugs professionally following the established procedure without sending additional emails with built up emotions and no concrete analysis or actionable suggestions

   6) QA, use your right to cut testing short if there are too many obvious Critical issues found, however always follow the procedure and send in testing summary at the end and allow devs to fix the Bugs to surprise you in the next test round vs. hate-mailing about the build quality

Upside from this week's confusion related with our new product is that almost everyone managed to transition to constructive discussions with actionable suggestions lead by analysis and pros/cons; we need more of this in all aspects of our work

Think, simplify, consolidate

We must consolidate new Bugs and reduce administrative overhead and by doing so help all teams when dealing with Bugs. Having multiple Bugs that all relate to the same cause, result in the same violation, expose the same incorrect behavior will allow for them to be fixed independently and can cause inconsistencies or needlessly take more time to create, review and update

Some examples encountered recently:
   Issue 1: GUI standards issue: several menu / ribbon items have inconsistent casing and are named differently across different products
   Bug 1: Have only one Bug for each of the products and list all related UI issues and standard violations within; this way all issues will be fixed at one time vs. risking to introduce new inconsistencies

   Issue 2: Product UI has multiple issues when using 120 DPI system font resolution
   Bug 2: Have only one Bug for the product at one time vs. one Bug per dialog or even one Bug per visual issue; there is no reason why some should be fixed and some not - they must all be fixed in order to consider having support for specified system font resolution

   Issue 3: Product stops responding during a specific operation (any general perceived performance issue)
   Bug 3: Have only one Bug explaining how to reach the non-responsive state; don't create individual Bugs for everything that no longer works after the product is already in the non-responsive state as the core issue must be fixed to resolve this

   Issue 4: Multiple usability or obvious design issues
   Bug 4: Have one Bug per group of issues that should all be resolved together, i.e. per UI dialog or group of dialogs where they are discovered

Obviously it's not possible to list all cases encountered so far and all possible cases that will be found in the future. Note the following best practices and apply:
   1) If Bugs found should be resolved all together to ensure complete functionality, consolidate

   2) If resolving found Bugs individually (even if they are all reported with equal severity) can produce inconsistencies, consolidate

   3) If just by grouping similar issues in a single Bug you can save much time vs. explaining them individually, consolidate

   4) If there are multiple unwanted consequences resulting from a single cause, consolidate

   5) When in doubt, always think and try to simplify: "Think critically, objectively and with full information before you answer, act or deliver"

JIT testing, or what to test when you think there is nothing to test

More than once it happened that you have a goal to test a new build of a product but it turns out the build is not available in time; you could:
   a) Test a product that has lower priority in your current Sprint
   b) Write some obscure automated test scripts for a product that isn't even ready yet and you haven't seen it at all but you did have 1 training session where you understood squat about the product
   c) Play Windows Solitaire until the build gets ready

Choosing any of the above is not the right answer and will quickly get you off the train where we're headed as a company. What to do?

Enter JIT testing
Your goal says you need to find bugs score in minimum of 250 and verify resolved bugs for a new product build. Ok - you cannot verify the resolved bugs as you still don't have the new product build, this part of the goal is externally blocked. However you can find bugs score of minimum 250 by testing previous product build you have available:

   1) Take the latest product build you have available; this can be a public build approved for production or it can be previous testing round build that was rejected due to a few found Critical bugs; it can also be a build recently created internally by the automated nightly build script - check the FTP yourselves and check the build version - don't wait for explicit notifications about such builds

   2) Test the hell out of the build and create new bugs that are worth 275 score points (10% over your goal); chances are that all the new bugs you find will still be there in the official new build when it is eventually submitted to testing

Once the new product build is ready, verify the existence all the bugs you found during the testing of the previous available build; I bet that most of the issues will still be there and that in the worst case scenario you may only need to find a few bugs only to exceed the score goal of 250

Other things to do while "waiting"
Don't forget that you'll still get ad-hoc requests to handle new installer testing, new website content to be checked - these are all higher priority and can be handled in parallel with the active Sprint

Developer teams will also create Patch testing PBIs. If a patch testing PBI contains Critical severity bug fixes, test it now and don't let it linger; if a patch testing PBI contains only High severity bug fixes, also test it now - there are only a few bug fixes per patch to test anyway so it won't take too much of your ad-hoc testing/support time

Things NOT to do while "waiting"
What I don't want to see is QA mailing excessively developer teams asking "When will we get the new build ready for testing" - such questions should be directed to QA Product owner (CTO) or not asked at all unless you are planning your next Sprint

Self-help book: Define your own defect severity

You've found a new bug but you aren't sure what the correct severity for the defect is? Welcome to the "This is no exact science" club

Here are some general hints how to solve this question that I already shared with some of you over IM:
1) Treat Bug severity standards as guidelines - see the first line in the document, it clearly says that the document is written as guidelines and that ultimate responsibility for bugs you find is yours

2) Worried about how will Critical severity affect production, developers? Are you also worried about global warming and dying rainforests? If so, how does this help you define the severity of the bug you just found? The answer is: it doesn't

3) Wondering why your Low severity defects describing features not working still aren't fixed after two months but are considering to make this one Low severity as well? Rethink this again

4) Put yourself in the customer's shoes. What is your reaction to the newly found bug?
   a) I don't like the product and I won't use it
   b) This is very annoying, I won't use the product because of it... much
   c) I can't use this feature as I wanted to but I see the workaround exists - kind of annoying but I'll get over it
   d) I don't frequently click in the top right corner 3 pixels next to the Ok button and cause the mouse pointer to glow red - I'll forget about this by tomorrow

5) Make sure that you've put yourself into 80% of the customers' shoes - if you are thinking as someone who used to code Commodore 64 games in assembly back in the 80's and know how to get Linux distro kernel version, then a chance is this issue may only be visible to you

6) Contact your QA team members to see if the apparently Critical issue is easily reproducible by at least one more team member; maybe your VM OS stopped working after months of abuse

7) Devs insist on reducing the bug severity you've set and you're scared, best to comply? No - stand by your decision especially with facts you determined before defining the severity

8) Severity can go both ways, both you and devs have solid facts about the case. Contact analysis team for recommendation. However this should be an extremely rare case if you follow #7 above

9) Still not sure? Notice that I haven't answered any of the questions above with definite answers, when to use which bug severity. This is entirely up to YOU. As you have the ultimate responsibility for quality of our products and ApexSQL brand in general, you also have all the power needed to affect the quality with bugs you find

Note: if devs are giving you hard time, let me know directly ;)