Why many software organizations still lack quality engineering leadership?

The year is 2022, and many businesses are still following the traditional software development cycle. Companies prioritize engineering over design and rarely consider application or product quality. But what exactly is a high-quality application? What distinguishes a quality-focused and product-focused organization?

The Quality Wave

We have moved away from traditional software development life cycles like Waterfall model and adapted Scrum and Agile Technologies. We have software or patches getting released every day. A sprint has moved from a several months to 2 weeks. Requirements change frequently and we adapt to it frequently.

But with so much going in terms of design & brand innovations, improved user experience, hundreds of A/B Tests setup, tons of user interviews and user observations lab, and continuous modification of stories; How do we make sure we are building the right thing? (validation) And How do we make sure we are building the thing right? (verification).

This wave of new SDLC triggered the need for the organizations to adapt a new Software testing life cycle (STLC). But what changed really?

The (big) shift left

Shift Left is about doing things earlier in the development cycle. Source: van der Cruijsen 2017.

Shift left targets to achieve following in compared to conventional testing models:

• Early involvement of QA Team in the complete SDLC, which means that QA gets a voice in major design and user experience modifications.
• The identification or I must say elimination of major bugs happens even before the software or the application is build.
• QA Team has more collaboration with product managers, engineering managers, scrum master, product designers, developers and sometimes even legal and sales team.
• QA Team gets to spend ample time designing and defining acceptance criteria’s before the development begin, promoting behaviour driven development (BDD).
• QA Team has a voice in various scrum meetings like spring grooming, spring planning as well a platform to provide feedback which happens in sprint retrospective.
• There are many sayings going around in tech field like “Own your module” or “Quality is everyone’s responsibility”. Shift Left promotes both the motto.

Behavior Driven Development (BDD)

BDD is an Agile software development methodology that documents and designs an application around the behavior that a user expects to see when interacting with it.

A developer, a test engineer, and a product manager are often involved in behavior-driven development (and potentially other stakeholders). The group gets together to brainstorm tangible instances of acceptance criteria in user stories. These examples are described in a feature file using a domain-specific language such as Gherkin. The feature file is transformed into an executable specification, from which developers can create an executable test.

BDD is designed to test an application’s behavior from the end user’s standpoint, whereas TDD is focused on testing smaller pieces of functionality in isolation

Benefits of BDD:

• Strong collaboration: Product owners, developers, and testers all have detailed visibility into the project’s development because to the shared language.
• High Visibility: Because it is a nontechnical process, behavior-driven development can reach a much larger audience.
• Rapid Iteration: BDD enables you to respond quickly to any and all user feedback so that you can make improvements to meet their needs.
• Focus on user needs: Users who are satisfied are excellent for business, and the BDD technique enables user demands to be met through software development.
• Meet business objectives: Each development may be tracked back to the actual business objectives using BDD.

How does BDD look in real life?

Feature: Google Searching
  As a web surfer, I want to search Google, so that I can learn new things.
  
  Scenario: Simple Google search
    Given a web browser is on the Google page
    When the search phrase "panda" is entered
    Then results for "panda" are shown

So… what went wrong? Why aren’t organization ready to adapt to modern quality engineering practices?

I have tried to address a few solid points that I have noticed first hand working with multiple clients and organizations.

Lack of a Quality Leader in the organization

There are times when the organization don’t want to invest in an independent quality team.

An independent QA team, with a strong QA Leader bring many benefits to table:

1. QA has their own voice.
2. QA Team gets to create global quality practices over multiple products or offerings, along with different modes (android, iOS, Web etc).
3. QA Team can develop generic tools and technologies to leverage automation and CI/CD pipelines configuration.
4. QA Team can generate generic reporting and static quality testing requirements.

This list goes on and on. But these are some of the most important tasks QA Team need to work on, independently from the developers, product managers and design teams.

The mentality that Developers are responsible for end to end QA

The old traditional belief that developers i.e. people who build the app are also responsible for testing it. This has many problems:

1. Developers have a development mindset. It’s hard for them to break things, hence to develop a “breaking thing” mindset.
2. Development along with testing sometimes create overload among your development teams, because they already are involved in unit testing, UI Design & Development, DevOps, Security and sometimes making sure legal and third-party contract verifications.
3. I believe developers are responsible for unit tests. We need to bring QA Team for integration, manual and automation end-to-end testing.

Leaders don’t want to invest in Quality

You often hear in big organizations, QA is everyone’s responsibility. I totally agree. It is. But only after your app (after every feature development, and minor/major release) has gone through multiple regressions and dry run plans.

A dedicated QA Team with solid domain knowledge, and highly qualified in test management, test scenario development, and automation technologies goes a long way.

Investing and caring about QA from the start, would lead your organization and product a long way instead of thinking of Quality when your product crashes heavily.

There is not enough time to invest in automation technologies

Change is invetable. We want to make our products better by enforcing a better user experience and that we make our stakeholders happy. But sometimes this come at a cost of establishing solid processes that will go a long way.

Avoiding quality is no more an option. A dedicated team or group of people specifically to verify and validate the build feature during every sprint is utmost necessary.

If the enterprise hasn’t invested in automation, and the CI/CD approach of building, testing and releasing things, it’s not too late. The first perfect time was years ago, the next perfect time is right now. Start small, Start with little things, but start.

Too much manual testing — Too much automation testing

Many organizations don’t want to spend in automation testing. They only realize the need of automation when product grows, and it’s hard to verify all the user requirements manually, but by then it’s too late.

Many organizations believe that 100% automation is the key and that everything can be automated. But that’s not true, manual and other integration tests are as important as automation testing, and that there should a proper balance between the requirements that fall under manual vs automation testing.

Quality Team (efforts) lacking Visibility

There are times when companies have a quality process set up. They have a good coverage of test scenarios through matrix and have process setup in CI/CD pipeline. But what they lacks is the communication the other team members i.e visibility among the feature team.

Lack of proper reporting and Feedback: QA Team have regression setup every night, and perform manual regression over multiple days to test any new feature launch. It is best to report the daily status through automatic reports (emails) to required people (like senior managers, product managers, core developers) etc.

Lack of proper CI/CD pipeline: The most important part of QA is of course testing, but more important setting up solid processes for a smooth feature release. There are times when the automation framework is not in sync with CI/CD pipelines for triggering tests after something gets merged to main branch or similar checks if necessary. Most of the platforms (like Jenkins) provide reports send to people via emails or slack messages too, which help improve visibility and alert if any breaking change happen.

Lack of Integration of test cases with Jira: It is important to link your test suites and test cases with acceptance criteria or stories or epics. This way, we can always relate a test case to a specific story/screen and the other way around too.

Focusing only on one aspect of Quality

For a typical enterprise product, which has launched platform on various platforms, following coverage is necessary:

• UI Testing (Web, Mobile, TV etc.)
• API Testing
• Performance Testing
• Security Testing
• DB Testing

Many companies tend to focus too much on any one of the aspect, and tend to ignore all the other domains. Some companies try to focus on all the mentioned aspect of testing, which makes tests hard to maintain. Hence a quality-focused organization adjusts their testing strategy based on their product and client-base.

Improvements come in small pieces — what an organization needs is a leader who pushes quality product over anything else. A good user experience, a high quality product makes a huge difference.

References

Behavior-Driven Development: Pros and Cons, Implementation, and Best Practices

BDD 101: Gherkin By Example