Software Testing Simplified

  Thursday, September 3rd, 2015

Software testing is so vital to determining quality of any software-based system that it accounts for 30 percent (and in some cases more) of the overall software development budget. No wonder many automation tools have been created to simplify this mundane aspect of software, although many companies end up wasting more time with automation tools than they bargained for, simply because they chose the wrong tools. More often than not, these distract the project manager and the team from what’s really important to the test project’s success. Studies have shown that best projects rely on management fundamentals (teamwork, communication, and controls), not on technology and state-of- the-art methodologies, not that these are not important.

 

After more than twenty-five years of playing most of the roles in the system development lifecycle and managing software test projects for the past fifteen years, the author has determined three core adaptable principles that are crucial for the success of software test projects. These are: (1) Agree on essential tasks for the test project, (2) Establish and maintain focus, and (3) Simplify the test process. To address these three principles fully, the author has decided to present them in a three part series over the next several months, beginning with the first core principle.

 

AGREE ON THE ESSENTIAL TASKS FOR THE TEST PROJECT

Take a moment or two with the program or project manager and follow these three steps to determine which tasks are essential for the project:

• Set measurable objectives for the test project,
• use those objectives to identify tasks and tools,
• draw up a schedule and assign tasks.

SET MEASURABLE OBJECTIVES

This is perhaps the most important step as everything else depends on it. It’s much harder to prove that you’re making progress on any project (including test projects) if there are no measures for it. For this reason, you need to express your objectives in specific measurable business terms. Preferably, they should have numeric goals associated with them, which are in turn expressed in terms of key performance indicators. For example:

 

Objective: Reduce the number of helpdesk calls attributed to software problems.

Key Performance Indicator (KPI): Problem Report Rate — the percentage of customer reported problems per week that are attributed to software error.

Goal: Reduce the current Problem Report Rate from 10 percent to 5 percent.

 

Measurable business objectives such as the example above define the critical factors for a successful test project in such a way that everyone involved — project sponsors, stakeholders, program managers, project managers, software development and test teams, etc — can understand. A clear set of business objectives helps teams stay focused and builds commitment among team members. Goals also reduce conflicts among team members and the departments they represent.

 

Over the years, the author has found that establishing test project goals like these early on can help with up front planning, close tracking and reporting of status on an on going basis as well as prioritizing competing goals. They enable test practitioners to show they have achieved what they set out to achieve, and thereby demonstrate accountability. Of course, developing realistic business goals for an organization is easiest when the project sponsors, stakeholders and test practitioners take part in setting the organization’s overall test objectives.

 

Another benefit of these objectives is that they help in successful management or monitoring of remote workers. For example, with weekly monitoring of deliverables, a project manager can adequately foresee warning signs and make mid-course corrections. A team in a specific location can be focused on a particular objective. If thoughtfully assigned, teams in remote locations can focus on discrete tasks, thus eliminating problems that can be attributed to sharing responsibility over a distance. With proper regular monitoring of the team’s objectives, work can be managed locally even though the actual work is done elsewhere.

 

IDENTIFY TASKS AND TOOLS

Having established the test project objectives, you’re now in a better position to determine what needs to be done for the test project and what tools (if any) would be helpful in meeting those objectives. Assuming that you have defined clear objectives like the example given earlier, creating the first list of what features of the system are in and what’s out for the test project is easy. Will the test project include stress testing, load testing, regression testing, user acceptance testing, and functional testing, etc? Selecting what’s in and what’s out can be done by simply answering the question, “How likely will this type of testing affect our ability to meet the business objectives of the project?” If the answer is “Very Likely”, then that type of testing is in for now (as other project priorities and constraints may still be a factor). If the answer is “Unlikely”, then it goes on the parking lot list.

 

This type of exercise is good for all stakeholders and the team to participate in. It’s very therapeutic for stakeholders and programmers to debate the likelihood that a given type of testing can affect a business key performance indicator. At the end of the day, everyone will understand exactly why a particular type of testing was chosen or not chosen. It goes without saying that the type of testing that was agreed to will determine the type of test tools (automation tools) that would be employed. How you determine the best tool that can meet your objectives is outside the scope of this paper.

 

While this discussion is going on, someone captures the decisions on a notepad. With the price of laptops continuing to go down these days, the preference is to use a notebook/laptop as it’s easier to capture the information once without having to re-type it later. At the end, the decisions are published for all involved for reference.

 

DRAW UP A SCHEDULE AND ASSIGN TASKS

With the tasks identified, now it’s time to decide which features will be tested in what phase. This almost always has dependencies – depends on what features the software development team provides and when. Ask the test team members to pencil in their estimates for each task. After each task has been estimated, lay them out on a calendar and assign resources to them. It’s suggested that you use a spreadsheet to do this (if you have scheduling tools such as Microsoft Project, it’s even better).

 

With estimates and resources identified, determine if the estimates are within the time constraint of the project. If not, revisit your estimates, and make any necessary changes. If after adjusting the estimates and you still cannot fit all the tasks within the allotted time, then it’s time to have another discussion with the program manager and the stakeholders about the schedule. You may have to re-negotiate the tasks and/or re-prioritize them. In the next discussion, we will see how establishing and maintaining focus are vital for the success of software test projects.