Release blocker bugs

Software has bugs. That is OK. We can't fix all bugs, otherwise, we will never release any product. Thus, when we commit to release a new product (version), we must be disciplined to introduce only enhancements and fixes agreed with the stakeholders. This will enforce predictability and more regular releasing. Predictable results make stakeholders happy. Regular releases that are equal or better in quality than the current one make customers happy

Release blocker bugs must be fixed prior to release 

Not all bugs are release blockers (showstopper) bugs. We can't strictly define each requirement to qualify blocker bugs. This is why we must review each found bug against conditions to decide if it is a blocker or not

Conditions

Important conditions to consider when reviewing bugs for potential release blockers:

• Is this bug regression against the last publicly released version? If it's not, and customers haven't complained about it, then this defect is not really bothering them and the fix can wait for future releases
• Does this bug make the new feature unusable for most customers? If it does, then a new version is not feature-ready. But if this bug only affects specific customers under specific conditions - then we shouldn't block the release and postpone to deliver the new feature
• Will this bug impact most of the customers and there is no workaround? Again - we shouldn't block the release and postpone to deliver all other benefits that new release would bring to most customers due to an edge case when there is a workaround

Symptoms

All bug symptoms should be considered against important conditions. All bugs are important, but we must review them and understand their impact in order to define severity and declare blockers. For example:

• the product cannot be installed,
• the product does not start,
• the product crashes or freezes,
• the product could cause data loss,
• the product has a security bug,
• the product has a licensing problem,
• the product has an unusable mission-critical (often used) function

These bugs are not necessarily blockers if they don't impact most of the customers and are met under specific conditions

If a bug is already present in a public version, and we won't make it worse with a new release if it remains unfixed - we mustn't block the release and deny the customers other prepared fixes or features

Examples

• The product can't be installed on any Windows OS. Blocker for release - must be fixed before release
• The product can't be installed on a Windows Server 2016, which is the most commonly used environment for product customers. Release blocker - must be fixed before release
• The add-in can't be integrated into SSMS 2012. Not a blocker for release due to a workaround - customers can easily and for free use the latest SSMS
• The "Options" window is not properly shown on 4K resolution. Not a blocker for release as long as there is a Compatibility based workaround

Again - we can't strictly define each requirement to qualify blocker bugs. Review each found bug against conditions to decide if it is a blocker or not.

Just remember that our goal is simple - release a build that is equal or better in quality than the current one in production, nothing more, nothing less

Intermediary sprints

The main goal for a Release sprint is to have a product release in the end, but when we are not yet ready to release then we also have an option to initiate an Interim sprint that will not end in the product release but will instead produce one or more building blocks necessary for the eventual product release.

Interim sprints are used when the team has unknowns ahead and needs to perform analysis, research, learning in order to be able to know how to plan a future Release sprint in details.

Even if the work ahead for the product release is well known but there is too much of it for the team to finish everything in a single max-length 4-week sprint, then having one or more connected Interim sprints is the solution.

Keep in mind that the main expectation for any kind of sprint is to have finite and measurable deliverables in the end which applies to Interim sprints as well. For the Interim sprint the deliverable will not be a product released to production but something smaller - some examples are:

• When looking for the best way to find out how to develop something then do the hands-on approach: develop a prototype or attempt to build a proof of concept
• It is still OK if this Interim sprint deliverable might not be used in the final product in production as long as we learned from the exercise and can now plan enough time to do it right
• It is also OK if the proof of concept turns out to be impossible to build, i.e. due to technical limitations or external constraints that we have now clearly identified and have removed all unknowns from our path
• When fixing a complex issue or optimizing performance we need to plan out enough time to attempt several or all potential fixes or experiments that will lead to resolution or improvements
• It is OK to have limited incremental success here without committing to specific outcome or specific numbers in the end as long as we have systematic progress and can keep stakeholders updated what worked / didn't work
• When researching and need to figure out how comps are doing things, or to learn more about a subject that is new but without possibility to build a tangible prototype even if it is thrown away in the end: it is OK to just do a detailed summary on the subject or ideally a plan on how we'll approach the next Intermediary or Release sprint

How much time can be allocated to an Intermediary sprint?

• Max-length applies to all types of sprints and it can never be longer than 4-weeks
• If multiple Intermediary sprints are needed then it is OK to connect them one after another but with explicitly approved plan on how many such sprints will exist before the next Release sprint
• Sprint planning time for an Intermediary sprint cannot be longer than 1 day
• Remember that if you have unknowns that are to be clarified during the Intermediary sprint then the sprint plan can be somewhat looser, i.e. Team member 1 Day 1, Team member 2 Day 2, etc. but this is an edge case applicable to time-boxed research only - everything else 

Product release sprints

1) Sprint planning

• Team has two days max to plan out the new sprint
• This is more than enough time for a quick research of goals ahead and a detailed sprint planning
• Take your time for sprint planning, up to several hours for the entire team is sometimes needed and the entire team needs to be present to take part in sprint planning, this is not something a single person (i.e. ScrumMaster) should do
• In case we have more unknowns then the team should not recommend a Release sprint but rather an Intermediary sprint where the primary goals will be to clarify the unknowns through hands-on proof of concept / prototyping or documented plan / deliverable, or to build necessary prerequisites for the eventual release

2) Sprint

• Maximum sprint duration is still 4 weeks or 20 workdays
• Each Release sprint will be divided into two structured wholes: Work phase and Test plan execution phase
• Maximum time allocated for Work phase shouldn't exceed 3 weeks or 15 workdays
• During this time the team should fix bugs, implement features, and complete other general Product Ownership related goals agreed on sprint start
• Work goals should be made in such a way that we can drop off some of the goals during the sprint and still achieve the release: i.e. drop off some of the planned new features, fix less bugs than planned
• If the team doesn't have a Test plan for the product that is up-to-date then one of the work goals needs to be to update the Test plan: this work is owned by the team SA but needs to get a buy-in from the entire team
• Maximum time allocated for Test plan execution shouldn't exceed 1 week or 5 workdays, and should match Test plan execution time estimate in team days
• During this time the entire team will focus on testing the product build in development by following previously prepared Test plan and by splitting test areas among all team members, both developers and SAs
• Test plan execution is "All Hands on Deck" time meaning that no team member is allowed to work on anything other than executing regression test to figure out quality of the current build
• There is no more "inter-team cross-testing" meaning you shouldn't send your build to other teams or other SAs to test it for you
• This doesn't mean that inter-team cooperation isn't allowed - actually it is still a good thing to do cross-testing with other teams especially when other teams are relying on your products and may be affected with the new release
• Devs must still execute "internal cross-testing" and verify each other's bug fixes (i.e. Dev1 tests bug fixes for Dev2, Dev2 for Dev3, Dev3 for Dev1 and final report is sent to SA)
• When the sprint ends the team will figure out how many bugs were found that SA will report in the bug tracker, and will estimate how much actual work remains to release the product
• Not all bugs found are blocking bugs (more on this soon)
• We just need blocking bugs fixed in order to release the product
• At this point the team will know if blocking bugs can be fixed within the Release phase time (see below) or not and will decide one of two things: we are going to release or we need to start a new release sprint in order to fix remaining blocking bugs that will take more than one week's time
• We need to watch out not to begin Release phase if we are uncertain that the remaining blocking bugs can be fixed within the allotted time: it is of no consequence to begin a new Release sprint, but there will be many affected stakeholders if we decide to enter the Release phase and fail to release on time

3) Release phase

• This is instead of what was previously known as "Test phase" but with main difference being that we are no longer spending time to test the product build for the first time but are focusing entire team efforts to release the product
• Release phase can take up to 6 days including the release day which is also considered the first day of the new sprint planning phase
• SAs no longer invest time into test plan or bug prospecting efforts as this is supposed to be already finished during the sprint
• Devs fix only blocking bugs and nothing else
• SAs verify fixed bugs only until we ensure there are no more blocking bugs
• Remember: the ultimate goal is to release a build that is equal or better in quality than the current one in production, nothing more, nothing less

Customer correspondence 101

Customers are generally satisfied with both our service coming from Sales consultants (SCs) and our technical support from Support engineers (SEs). However it still happens from time to time we drop the ball on some cases and run into issues between teams which shouldn't happen

Our goal must always be a satisfied customer and that's the only proof that shows we're handling our business correctly and are doing our job well

Below are listed a few important things that you should already know, but please make sure to read them carefully again and implement them in all future cases:

- SCs job and responsibility is to forward customer emails who have technical questions/issues to the support team and follow up on the progress early and often to make sure a customer got all the answers and the help he needed

- SEs job and responsibility is to include corresponding SC in all conversations they have with the specific customer and to also let SCs know when the support job is done; although SCs won't participate in this "technical part", they will see the progress of the case and will be able to plan their future activities 

- SC responsibility is to make sure a customer always gets what he needs (a technical support, a quote, respond to a specific question, etc.) 

- SEs - during the business hours (M-F 8-5 US ET), we should acknowledge all emails within an hour, provide actionable response within 24 hours and have a resolution within 72 hours
 -- If we get an email after business hours, SE on duty should ACK email, while the product owning SE should provide actionable response on the next business day
 -- Here it is very important to know that if the SE on the job doesn't know the answer to some question or he is not sure about something, ignoring an email until he gets an answer *is not an option*; SE must reply (ACK email) and provide an ETA when more details will be provided

- SCs – during the business hours, we should acknowledge all emails within 15 mins and provide actionable response within an hour; if the assigned SC finished the shift and a customer sends an email, a colleague from the team should respond and provide all necessary details

- Not responding to customer emails and not following the rules above is not an option; we need to be responsive to our clients and honor the things we put on our website such as our availability and resolution times
-- If we don't honor our commitment it creates dissatisfaction for our clients that has already in several cases directly or indirectly influenced client's decision to purchase our products

- After the customers purchase our products our job is not done – we still need to care about their satisfaction and to make sure we solve all the problems they run into
-- If we can't solve a problem immediately we can at least be responsive to provide all the info and all the help that we can to at least workaround an issue or provide an ETA for full resolution

- When replying to customer's email, it's desirable to always have following guidelines on your mind:
-- Read every email twice very carefully, and then once again
-- Make sure to provide an answer to every question a customer asks
-- Reply inline and use a different color (except red)
-- If you don't have an immediate resolution, make sure to provide an ETA to the customer for the question or issue he/she has reported
-- Always provide an ETA for the next step if the ball is in our court; if we miss that ETA for whatever reason then follow-up to provide a new ETA without forcing customers to follow-up first

Support engineer medior - Level 2

The path to Support engineer Level 1 or "SE certification" as we commonly call it is relatively straightforward:
  a) Get to 100% unified metrics in at least 3 consecutive months:
    -- ZIGS 100% - easy enough with majority of our products, just be as hard on them as possible
    -- Writing 100% - this is the most challenging part as it requires constant learning and systematic weekly writing ideally without ups and downs
    -- Support coverage divided between colleagues and following The Support Engineer’s Creed

  b) Contribute to your team by helping them own all of the content

  c) Contribute the team with product ownership - comp analysis

  d) Cover the operational work of getting the products through the testing and out to production

When you have the metrics and both your team and internal customers provide good values feedback you are all set for Level 1 certification - on average this takes 9-12 months but we did have some cases (one of them only a few months back) of certifying in 6 months flat

SE Level 2 shares all of the requirements with SE Level 1 except that metric requirements are higher

However simply achieving higher SE metrics won't be enough to get certified for SE Level 2 as the standards for contributing to the team and the company are also higher - meeting those standards is a requirement for a positive Level 2 values review by stakeholders and internal customers

Standards

Here are some specific examples of higher standards expected from SE Level 2:
  1) QA - as level description also says:
    -- Provide real-world testing vs. just some easy to catch surface bugs or edge cases by pounding your mouse and keyboard until something breaks
    -- Be an advocate for automation vs. allowing others to ask for test automation from your team
    -- Plan your work and be meticulous with regression testing, never allow the same bug to resurface in production
    -- Know your products performance, usability (UX especially) and quality and raise issues vigorously and frequently; don't allow customers or others within the company to do this for you

  2) TQM - keep owning the content including everything you previously wrote and is available in production as it must be kept up-to-date

  3) Earn the respect of your team:
    -- DON'T keep quiet and allow for known issues to go unchallenged
    -- DON'T be stubborn and insist on low-ROI fixes and changes
    -- DO stand your ground when you know and have the facts that the issue you raised is valid and important to the customers

  4) Lead or assist with company-wide initiatives
    -- Update and create standards that affect multiple products
    -- Raise issues with content and products owned by other teams

Time period

Expected time period to gain the necessary experience for SE Level 2 from day one on the job is at least 3 years

This depends on each person individually but if you started working without previous database administration / development industry experience then you can consider yourself a fast learner if you certify as soon as you hit this mark

No regression

More so with SEs than with other roles, regression is an automatic process based on metrics

This also means that you can't regress when it comes to values and contribution you had in order to achieve Level 1 and must keep hitting the high notes most of the time

Hard skills

This will come naturally with experience through the creation of new content, real-life use case testing, test automation, and other work

Being T-SQL guru or knowing secrets of SQL Server and cloud administration isn't the goal, but knowing your owned products well and related technologies is

Ultimate test

Success = results / time / stakeholder_time

Note the last variable in this equation: stakeholder time

Ask yourself how much time do you save to stakeholders?
  a) Are you being reminded to update owned content (roadmaps, articles, videos)
  b) Are others in the company raising endemics and red flags in your products
  c) Do stakeholders have to invest much time in order to review your content (articles, release notes, F matrices, etc)
  d) Are you being asked questions about the team status that catch you by surprise
  e) Are you unable to answer a functional question related with the owned product
  f) Are you unaware of a comp status or a melting iceberg affecting your team
  g) Is your view of the product PUQ different from reality
  h) Have you recently approached product development and design like a "coder" instead of a customer

If you've answered any of the above with a Yes then that is a potential blocker you need to remedy

Software developer on steroids - Level 2

Getting from a Jr. Software developer to Software developer Level 1 might have seemed like a long way to go but this is just the first step for a Software developer career in ApexSQL and all the future steps have even steeper learning and growth curve

If you've certified as a Level 1 dev then it means:
  a) You've been working in a fully functional team the company can rely on
  b) Have been a leader at least for a specific task or a goal within the team to carry your own weight
  c) Have gained enough experience with how we work and have already mentored other colleagues or can do so now
  d) You've grown and developed your hard skills just enough so that for any new challenges you encounter you already know how to approach resolving them even when it requires more learning
  e) You are successfully saving time to stakeholders and are working with them vs. working for them or thinking there is us-and-them

Time period

How long will it take to get from the day 1 in ApexSQL to a Software developer Level 2 will depend on each person individually - if you started as a Jr. Software developer (Level 0) then you'll need at least 3.5 to 4 years to be able to check off all of the items below

No regression

The first expectation is that you constantly confirm the level you're currently at which means you don't regress by no longer doing the work you were successfully doing before, or by doing it slower (i.e. you caught lazies)

If you had regression in your performance then it is critical to first make sure you get back on track and have consistent high performance of at least 3 or more months in a row - not doing so risks actual regression back to a Level 0

Soft skills + 1

When doing a Level 2 values review expectations will be one step higher than those for a Level 1 performance review

If you had a "Needs improvement" score for any of the values but have still managed to certify for Level 1 then those values if not improved in the meantime will read as "Unsatisfactory" during the Level 2 review which is a showstopper

If you had a "Meets expectations" score for a value then you need to have displayed consistent or improved values match in order to keep reading "Meets expectations" for the Level 2

Hard skills++;

Jr. Software developers can best testify the amount of new development knowledge and skills gained when going from Level 0 to Level 1 - the same goes for a Level 1 dev aiming to certify as Level 2

Each team has a bit different hard skills requirements depending on the owned product set so the only common metric here is how have those hard skills helped you be more efficient during work

Success = results / time;

The better hard skills you gained then you'll contribute to your team with more results over less time

Value added time saver

Can you confirm all of this:
  1) Are you self-sufficient enough so that others don't need to "manage" you by asking questions and giving directions / frequently correct you
  2) Do you communicate proactively and in an actionable manner with the stakeholders / no surprises caused by you and your team
  3) Do you educate stakeholders and provide suggestions and guidance yourself
  4) Can you be a role model for all Software developer Level 1 peers

Achievements

Additional information we should put on a table before starting the perf review for Level 2 is the list of achievements you can brag about since you've certified for Level 1

If you don't have anything exceptional to show for as a Level 1 dev then how can you justify you have earned the next level

This is also a great personal litmus test you can do for yourself before reaching out - are you able to provide a list of at least three achievements that you are proud of during your Level 1 tenure?

But don't say that i.e. you're proud of showing up for work every day or that you've managed to complete 3 sprints in a row as these are just normal everyday expectations so you won't be doing yourself a favor

For example:
  a) Took initiative to build a component / engine / feature which helped the team by ...
  b) Improved communication of the team with the stakeholders
  c) Helped the team achieve PUQ superiority by fixing X bugs in Y time / by improving performance by x% / by identifying and fixing usability issues
  d) Mentored new colleagues so that they have achieved ...
  e) Suggested [and helped implement] a feature / functionality that directly helped with Platinum goal
  f) Successfully managed X subcontracted projects
  g) Worked with team SEs to ensure uninterrupted team content flow
  h) Took ownership of a standard and helped everyone enforce it

Also such achievements are good to have but aren't the ones we're looking for:
  a) Learned to write thread safe code like a boss - good for you
  b) Read 3 books on design patterns - congrats
  c) Went to a .NET conference - nice
  d) Learned angular / js / css - great if it'll help you get the job done, but the focus is on the achievements that are "done" and have directly helped with your team and the company goals

The art of specs

In the initial The art of subcontracting article we've covered in short differences between functional/technical and requirements/specification

When preparing initial documents to describe a subcontracting project we have 4 distinct levels of documentation "depth" where each is more detailed than the previous one - and you don't necessarily need to document everything down to the lowest level in order to have a successful project

Level 1: User stories

User stories are a great way to abstract yourself from the technical minutia and to define top-level goals for the subcontracting project - they are the answers to "What?"

User story is a one-liner where you can describe your project goals to a person who is seeing the project for the first time and has no idea what you want to accomplish - never allow subcontractors to figure this out for themselves

Also a user story will make sure that the user (you) achieves clarity about the project goals before going down the rabbit hole

User story has a few elements:
a) "As a user" - this puts you in the proper set of shoes on start as you are the user
b) "of the <final deliverable>" - clarify what is the expected end deliverable of the project: an app, a component, an engine, documentation, specs, etc.
c) "I want to be able to <functional goal>" - one desired project use case

Some examples:
As a user of the application I want to be able to... (one goal what you want the application to cover, i.e. allow for browsing database encrypted objects, to be able to easily decrypt database objects)

As a user of the component I want to be able to... (visually view database objects in a form of a graph, manipulate database objects as a graph)

As a user of the engine I want to be able to... (access all of SQL Server instance properties in an object model form)

As a user of the functional specification I want to be able to... (have a detailed functional requirements breakdown for the engine mentioned above).
This specific user story is great for when creating specs-for-specs (see appendix)

As a user of the automated test I want to be able to...

All but the simplest projects will have more than one user story defined - i.e. one user story per needed application/engine feature


Level 2: Functional requirements breakdown

Once user stories are defined they can now be broken down into individual and more detailed functional requirements

Functional requirement is a free-form short description that further explains a user story

To breakdown a single user story "As a user of the visual component I want to be able to view database objects as a graph":
a) Each object type should be visually represented with a different geometrical shape
b) Related objects should be interconnected using lines
c) We should have several predefined presets / algorithms for object layouts
...

Functional requirements are short and don't go into details, i.e. what kind of geometrical shape should be applied to what type of object, how will object name be displayed, color of the shape, etc.

It is critical to invest as much time as needed to define all functional requirements that we need on start.

If we uncover that we've missed to define a functional requirement during the project then all such burden of additional requirements will fall down on us (as project appendixes that bear additional costs)


Level 3: Functional specification

Each of the individual functional requirements can be further broken down to create a functional specification / functional details

Consider each of the functional requirements as a new paragraph or a separate section with a freedom to go into all the tiny details

A section "Representing object types visually" can have:
a) A mockup / picture of a representative object type
b) List of default geometrical shapes, colors, text
c) List of configurable elements, and all required shapes to be available
...

Drawing up a good functional specification is very time-demanding and depending on the project scale and complexity it can be wise to consider subcontracting this work (see appendix)


Level 4: Technical specification

The lowest level of a subcontracting document is technical specs which can be more or less detailed, or can be skipped altogether

Almost every development subcontract will have some basic technical requirements in a form of required development technology, coding standards, etc.

Anything more than that is very risky as it effectively spoon-feeds the subcontractor and requires a lot of micromanagement time on your side thus making it questionable whether you should've worked on the project by yourself from the start

A good rule of thumb is to avoid technical specification whenever possible, or if really needed have subcontractors work on it prior to beginning any actual implementation/development work

For specific projects where it is important to go down to tech spec details level, try to keep it as short as possible


Appendix - Specs for specs

The main goal of subcontracting is to help you achieve team goals as quickly as possible (success = results / time)

Smart product teams will keep this in mind at all times and will not replace their development time with spec writing time

A smart product team will spin off subcontracting projects by focusing on writing up Level 1 (user stories) and Level 2 (functional requirements) subcontracting documentation

As such you can have subcontractors write Level 3 and Level 4 (functional and technical specification) documents themselves which you can later review and approve prior to starting the implementation

a) Know thy comps - research and figure out in details who we want to better / what we want to achieve
b) Know thy goals - write up good user stories by yourself what we want to achieve (Level 1)
c) Know thy needs - write up all functional requirements (Level 2)
d) Make it so - have subcontractors do the remaining work by writing the remainder of the project documentation prior to implementation