Thursday, January 16, 2020

Release blocker bugs

Software has bugs. That is OK. We can't fix all bugs, otherwise, we will never release any product. Thus, when we commit to release a new product (version), we must be disciplined to introduce only enhancements and fixes agreed with the stakeholders. This will enforce predictability and more regular releasing. Predictable results make stakeholders happy. Regular releases that are equal or better in quality than the current one make customers happy

Release blocker bugs must be fixed prior to release 

Not all bugs are release blockers (showstopper) bugs. We can't strictly define each requirement to qualify blocker bugs. This is why we must review each found bug against conditions to decide if it is a blocker or not

Conditions

Important conditions to consider when reviewing bugs for potential release blockers:

  • Is this bug regression against the last publicly released version? If it's not, and customers haven't complained about it, then this defect is not really bothering them and the fix can wait for future releases
  • Does this bug make the new feature unusable for most customers? If it does, then a new version is not feature-ready. But if this bug only affects specific customers under specific conditions - then we shouldn't block the release and postpone to deliver the new feature
  • Will this bug impact most of the customers and there is no workaround? Again - we shouldn't block the release and postpone to deliver all other benefits that new release would bring to most customers due to an edge case when there is a workaround

Symptoms

All bug symptoms should be considered against important conditions. All bugs are important, but we must review them and understand their impact in order to define severity and declare blockers. For example:
  • the product cannot be installed,
  • the product does not start,
  • the product crashes or freezes,
  • the product could cause data loss,
  • the product has a security bug,
  • the product has a licensing problem,
  • the product has an unusable mission-critical (often used) function
These bugs are not necessarily blockers if they don't impact most of the customers and are met under specific conditions

If a bug is already present in a public version, and we won't make it worse with a new release if it remains unfixed - we mustn't block the release and deny the customers other prepared fixes or features

Examples
  • The product can't be installed on any Windows OS. Blocker for release - must be fixed before release
  • The product can't be installed on a Windows Server 2016, which is the most commonly used environment for product customers. Release blocker - must be fixed before release
  • The add-in can't be integrated into SSMS 2012. Not a blocker for release due to a workaround - customers can easily and for free use the latest SSMS
  • The "Options" window is not properly shown on 4K resolution. Not a blocker for release as long as there is a Compatibility based workaround

Again - we can't strictly define each requirement to qualify blocker bugs. Review each found bug against conditions to decide if it is a blocker or not.

Just remember that our goal is simple - release a build that is equal or better in quality than the current one in production, nothing more, nothing less