There is a surprising statistic: in 2020, the poor quality of software products cost companies more than $2 trillion in the US alone. The COVID-19 pandemic also played its part; in response to the pandemic challenges, enterprises struggled to offer value at the necessary pace without jeopardizing the quality of their software products.
You don’t have to put much effort into finding examples – just take a look at Bó, the digital bank developed by the high street lender RBS to compete with banks like Monzo and Starling. It was supposed to become a major mobile banking player, providing budgeting and security features. However, six months after the release, the Bó team faced serious issues – bugs. As a result, the team spent the three months after the launch fixing bugs. One of the main reasons for early tech failures was the haste to go live to avoid project cancelation. There were other circumstances that contributed to the collapse of Bó, but it is a well-known reality that critical bugs don’t build credibility for a company. Errors are not only expensive to fix, they also have a great impact on client perception.
Here at MindK, we are passionate about developing quality software products for our customers, so quality assurance (QA) is an integral part of each project we take on. However, we not only test the software we deliver, but we have also developed an effective QA management process that helps us develop high-quality products.
Today, we will explain exactly what software quality management is and which best practices in software testing we use to put product quality on autopilot. Let’s start with some basics.
What is software quality management?
Software quality management is a process aimed at developing and managing software quality so that the product fulfills the quality standards required by the customer, along with any other essential regulatory and development criteria, if any. Quality management activities include three main components:
- Quality planning. It involves choosing appropriate processes and standards for a specific project and adapting as needed to create a quality plan. We can call it a more project-oriented part. It may formalize such aspects of quality management as the roles and responsibilities of stakeholders, project and testing documentation, software requirements specification, types of relevant testing, a rough testing timeline, cost, and resources needed, reporting procedures, risk-mitigation strategy, and so on.
- Quality assurance. The goal of quality assurance is to build organizational procedures and quality standards. This is called the process-oriented quality management and may include identifying any standards that may be used in the development process, carrying out traditional processes such as quality reviews, encouraging process documentation standards, and similar. In short, setting up the QA process focuses primarily on preventing bugs and errors. Statistics show that it’s 6 times more expensive to fix a bug found during system testing.
- Quality control. This part is more product-oriented and is focused on ensuring that the software development team follows all the best practices and standards in order to develop high-quality products. Quality control involves software testing operations that are prescribed by quality planning and include manual and automated testing, performance, integration, usability, security, regression, and other types of testing.
With such a complex approach to software quality management, the company has a much higher chance of achieving a number of benefits, among which are:
- better productivity of the development team;
- enhanced product quality, since test statistics and defect monitoring are more precise and up to date;
- reduced rework and saved costs, as flaws are discovered sooner in the software project development life cycle; and
- increased credibility as well as client confidence in the product.
You may guess from the above, as soon as software quality becomes a priority, it cannot be left only to the QA team. All parties engaged in the development process must work together. QA process improvement by means of a well-planned quality management process aims to identify and organize these activities.
Now, it’s time to discuss what quality management practices we apply at MindK to milk it for all it’s worth.
Top 5 QA best practices used to enhance the quality of software testing
#1: Plan properly to avoid wasted time
No matter which software development methodology you use, testing should be an important element of any project plan. Waterfall as a traditional software development model considers testing as a separate phase that takes place after development. Any Lean approach to development implies the constant release of new code, with testing occurring throughout the development process. In any case, it is critical to plan the testing properly.
In order to make a holistic approach to planning, we have three main instruments: test plan, testing strategy, and test cases.
Testing strategy is a product-oriented document that is more of a collection of instructions, rules, or principles that govern the test design and the QA process. The test strategy’s objective is to define the testing objectives, industry standards, time and budget constraints, and the like. This document is rarely reviewed and is considered like a guiding star of the testing project. We’ve already provided an example of the software testing strategy used at MindK when talking about test automation strategy, so check it out.
Test plan is a detailed document that specifies the activities that should be carried out during the testing process, like test approach, testing timeline, estimates, deadlines, and resources needed to complete that specific project, risks, and much more.
You can consider it a roadmap for running the tests required to guarantee the software solution will work as needed. The test plan is created for each release or sprint, and is constantly reviewed by the team and stakeholders. A test plan is critical to planning since it serves several purposes:
- It establishes explicit objectives, expectations, and requirements for the testing process. Everyone understands what to expect from the process and what the end purpose is.
- It serves as a means of communication. First, it ensures that the client, stakeholders, and the testing team are always on the same page, which eliminates a number of problems. Second, it allows the project team to influence the test plan and vice versa, which means everyone understands the testing procedure and can contribute to obtaining better results.
- It allows better control over the testing process. Writing a test plan forces us to tackle the challenges and potential risks ahead and focus on eliminating their negative influence on the project.
- It assists us in dealing with change. We at MindK tend to change our plans during the early stages of the project as we gain more knowledge and information. Thus, we also adjust the testing plan as the project progresses and circumstances change. By changing the plan at important milestones, we can keep testing on track with the project demands.
And finally, a test case is far more specific than the test strategy and test plan. It is a series of operations carried out to assess whether the solution meets the requirements and performs properly. The main goal of the test case is to assess if various elements of the system are operating as expected and to check that the system meets all relevant standards, guidelines, and customer requirements.
While test cases are not always considered “cool” in many circles, they are an essential component of our testing approach for several reasons. Writing test cases forces our QA team to think critically about what they’re testing, how they test, and the situations needed to thoroughly test the functionality. Test cases are a powerful tool for regression testing at the end of a project, and they are an important element of our project handoff.
MindK’s example of the test cases for regression testing
Using all these three important artifacts of the testing process, we manage to significantly avoid time-waste, as we analyze and prepare all the steps beforehand. When testing starts, we know exactly what the goals of testing of the project are, which testing approach is the most suitable, what are the testing types for better test coverage, how to test, and the results that should be obtained. Moreover, it allows us to maintain the transparency of the process with our clients, as they are always in the loop of how their product is tested and the expected end result.
#2: Clearly define QA roles and responsibilities
If you ask a typical client or even a Product Owner about the duties and responsibilities of the quality assurance team, you will likely hear that the QA team is a group of testers. They work on searching for software bugs and reporting them to developers. Some may say that there are also Team Leads, who oversee the QA engineers.
In reality, the QA team is much more diverse. Among the common roles in QA are:
- software test engineer;
- test analyst;
- test automation (TA) engineer;
- test architect;
- test manager (or QA lead) and others.
Each of them has their own skill set, responsibilities, and tools to work with. Software testing engineers test the overall system by means of relevant testing techniques and tools. Test analysts define the testing conditions, test coverage, and develop test scenarios and documentation. TA engineers develop test scripts to run automated testing. The test manager prepares the test strategy and controls the testing process.
In view of this, it’s more than important to determine all these roles and responsibilities on the project. The way to determine these roles depends on the project complexity, company workflows, experience, and much more.
Google, for example, ensures consistency in quality by blurring the distinction between QA and developers. They expect developers to test their code, and testers to supply the required infrastructure. The QA industry is increasingly changing toward a unified job of a testing expert who is able to automate, develop testing tools, and even write production code when required.
We at MindK share such an approach and are progressing in this direction. For now, we follow the Agile methodology for most of our projects, which presupposes cross-functional teams. Thus, we don’t focus on the multi-level gradation of QA specialists, but rather on so-called General QA specialists who are highly-competent specialists with knowledge to act as test architects, test analysts above all. We also have test automation engineers that are responsible for automating test scripts and a Head of QA who controls the process and approves testing strategies.
#3: Test early, test often
As we’ve already mentioned above, the cost of fixing bugs increases with each phase of the software development process. Bugs detected in the testing phase require reworking of the existing code. It results in additional time, delays, and higher costs. In order to minimize and avoid any of these negative outcomes, we integrate QA as early as possible in the SDLC process.
The practice focused on finding and preventing issues early in the software delivery process is called a shift-left approach. Shift-left testing refers to testing that occurs earlier in the software development process and highlights the need to focus on quality from the beginning of a software build, rather than waiting for errors to be discovered later on.
Early testing is the guiding spirit of the Agile methodology. At MindK, we develop the majority of our projects following Agile principles, which is why the shift-left approach goes through our projects and brings a number of benefits both for the clients and for the development team. We involve the QA team from the requirements gathering phase, so it allows them to gain product knowledge, essential for testing. With this background, QA can help analyze the stakeholders’ requirements, product documentation, UX wireframes, or similar. This knowledge helps the QA team further in the design phase, too. When the QA team has a clear view of the software requirements, design, and technology stack used on the project, it can without further ado use this information for developing a test plan, test strategies with relevant test architecture, and so on.
An example of test automation architecture created by the MindK team for one of our projects
Except for early involvement of QA, Agile presupposes also introducing testing in the software development process DNA. This means that, unlike the traditional approach to software development where testing only begins after completion of the development stage, in Agile all these activities (planning, development, testing) occur in parallel. This way, the result of each iteration (or sprint) is a working and tested piece of software.
The beauty of such an approach is the continuous feedback the project team receives from the client. At the end of each iteration, the team demonstrates the piece of working functionality to the stakeholders and receives feedback.
If you are interested in how to develop the solution following Agile and what is the role of the client during the whole process, then you should check out our guide on Agile product development.
#4: Continuous testing to streamline the process
Did you know that years ago even, Amazon deployed software every 11.6 seconds? The company explained that speed is achieved thanks to DevOps principles and effective test automation (TA) strategies. As a result, it’s not surprising that over 40% of the organizations polled in the State of Test Automation 2020-2021 report spend up to half of their total QA budget on automation.
If you truly want to increase the quality of your software and streamline the process, then definitely consider automated testing or employing automation tools to perform the tests.
At MindK, we place greater emphasis on automated testing, Continuous Integration (CI) and Continuous Delivery (CD) to streamline the process.
When the project grows, there is more and more functionality that requires testing. Manual testing is expensive and, most importantly, time-consuming. This is where the CI/CD comes into play. CI/CD pipeline relies heavily on testing automation and is all about getting new code releases as soon as possible.
As for the tools we use for test automation, we choose them depending on the client project specifics. It can be out-of-the-box test automation solutions or the creation of a custom testing automation framework to get the optimal mix based on goals, technologies, and infrastructure.
To sum up, continuous testing increases the overall speed of development thanks to fast test execution and the repeatable nature of automated testing. By speeding up testing, the client also receives:
- frequent releases;
- faster changes and updates;
- shorter software development cycles; and
- faster delivery time-to-market.
#5: Report bugs effectively
Reporting is a key part of an effective quality management strategy. It is a critical activity in software testing that is often carried out by QA engineers in an attempt to document the outcomes of one or more test activities, such as test plans, test cases, or test scripts, when it comes to test automation.
If test reporting is done extensively and on time, the test analysis report and comments gained may be quite beneficial to your development lifecycle. There are a few advantages QA reports provide:
- Ensure that the software product meets the predefined criteria and requirements.
- Сomprehend the overall quality of testing and test automation activities and reveal the causes of a failed/negative test report.
- Allow the project team to make more informed release decisions, as the report’s data is critical to the company’s decision-making process.
- Improve communication between team members. The reports ease the interaction between every team member involved, making sure everyone is on the same page.
As a result, testing reports not only help to increase product quality but also allow for monitoring testing operations and enable early product releases. There are a great variety of quality and testing reports that depend on the project and its requirements. Generally, among them are:
- Test incident report that informs about any defects discovered throughout the testing cycle.
- Test cycle report comprises a group of test cases needed to achieve certain testing objectives during a test cycle.
- Test summary report completes a cycle and can be used to assess whether a product is ready for release.
Other types of test reports include test case reports, test execution reports, bug status reports, severity/priority reports, failure and timing reports, and so on, depending on the aspects you wish to cover.
The important thing you need to remember about quality reporting is that your test reports should be complete and well-done. “What is this supposed to mean?” you may ask. A well-done and complete test report should be precise, clear, standard, and explicit.
The final aim of reporting should be to streamline communication among all the team members involved. Keeping this in mind, it must contain information that is concise yet comprehensive, and, above all, transparent to all stakeholders. It is preferable if descriptions of specific items are brief and summarize test findings.
There are a number of test reporting tools on the market today. At MindK, we use Jira for this purpose, as it provides a transparent workflow for all the stakeholders. Reporting in Jira allows connecting requirements, test cases, and bugs to Jira issues like epics, stories, and so on.
Example of MindK’s Jira dashboard dedicated to bug reporting
For test automation, in particular, reporting is a crucial component in an automation framework. After running the automated test suites, the findings will be the sole artifacts available for analysis of test failures.
The test automation reporting varies depending on the test automation tool you use. However, some metrics are required regardless of format, like total number of scripts, list of all test cases, test result (passed/failed status), duration, and environment name.
The test summary report we get for API testing in Postman
An effective QA management process is at the heart of every high-quality solution
If you want your company to be competitive and win in the IT market, you are not going to make it without high-quality solutions. Improving the quality management process inside the company will have the greatest overall influence on the quality of the product you develop and the company’s success in general. That’s why skipping or underestimating quality management is not the case now, as the cost of mistakes might be too high.
At MindK, we pay great attention to QA management workflow and automated tests that provide continuous testing of the solutions. Based on the experience we gained on our projects, our team now helps companies that lack experience in test automation get all the benefits that come with it. We can streamline your product releases with our end-to-end automation testing package, provide you with professional test automation consulting or allocate our best QA engineers to help you design and develop test scenarios, write automated test scripts, run different types of software tests, and much more. Take a look at our QA automation services for more details.