How to calculate Statement, Branch/Decision Coverage

How to calculate Statement, Branch/Decision Coverage

HOW TO CALCULATE STATEMENT, BRANCH/DECISION AND PATH COVERAGE FOR ISTQB EXAM PURPOSE

Statement Coverage: (Find shortest path such that all nodes are covered at-least once)

In this, the test case is executed in such a way that every node is traversed at-least once.

Branch/Decision Coverage: (Find shortest path such that all the Edges are covered at-least once)
Test coverage criteria requires enough test cases such that each condition in a decision
takes on all possible outcomes at least once, and each point of entry to a program or
subroutine is invoked at least once. That is, every branch (decision) taken each way,
true and false. It helps in validating all the branches in the code making sure that no
branch leads to abnormal behavior of the application.

Path Coverage:(Find all the paths from source to destination)

In this the test case is executed in such a way that every path is executed at least once.
All possible control paths taken, including all loop paths taken zero, once, and multiple
(ideally, maximum) items in path coverage technique, the test cases are prepared based
on the logical complexity measure of a procedural design. In this type of testing every
statement in the program is guaranteed to be executed at least one time. Flow Graph,
Cyclomatic Complexity and Graph Metrics are used to arrive at basis path
How to calculate Statement Coverage, Branch Coverage and Path Coverage?

Draw the flow in the following way-
 Nodes represent entries, exits, decisions and each statement of code.
 Edges represent non-branching and branching links between nodes.

Example:
Read P
Read Q
IF P+Q > 100 THEN
Print “Large”
ENDIF
If P > 50 THEN
Print “P Large”
ENDIF
Calculate statement coverage, branch coverage and path coverage.




The flow chart is-

Statement Coverage (SC):

To calculate Statement Coverage, find out the shortest number of paths following
which all the nodes will be covered. Here by traversing through path 1A-2C-3D-E-4G-5H all
the nodes are covered. So by traveling through only one path all the nodes 12345 are covered,
so the Statement coverage in this case is 1.

Branch Coverage (BC):

To calculate Branch Coverage, find out the minimum number of paths which will
ensure covering of all the edges. In this case there is no single path which will ensure coverage
of all the edges at one go. By following paths 1A-2C-3D-E-4G-5H, maximum numbers of
edges (A, C, D, E, G and H) are covered but edges B and F are left. To covers these edges we
can follow 1A-2B-E-4F. By the combining the above two paths we can ensure of traveling
through all the paths. Hence Branch Coverage is 2. The aim is to cover all possible true/false
decisions.
Path Coverage (PC):
Path Coverage ensures covering of all the paths from start to end.
All possible paths are-
1A-2B-E-4F
1A-2B-E-4G-5H
1A-2C-3D-E-4G-5H
1A-2C-3D-E-4F
So path coverage is 4.
Thus for the above example SC=1, BC=2 and PC=4.
Memorize these….
100% LCSAJ coverage will imply 100% Branch/Decision coverage
100% Path coverage will imply 100% Statement coverage
100% Branch/Decision coverage will imply 100% Statement coverage
100% Path coverage will imply 100% Branch/Decision coverage
Branch coverage and Decision coverage are same.

Thanks,

Mohit Bhatia

Mutation Testing

Mutation testing

Mutation Testing involves running slightly corrupted versions of your target program through your test suite to see if any test cases flag the variations as defects. It evaluates the quality of software tests. Mutation testing involves modifying a program's source code or byte code in small ways. A test suite that does not detect and reject the mutated code is considered defective. These so-called mutations are based on well-defined mutation operators that either mimic typical programming errors (such as using the wrong operator or variable name) or force the creation of valuable tests (such as driving each expression to zero). The purpose is to help the tester develop effective tests or locate weaknesses in the test data used for the program or in sections of the code that are seldom or never accessed during execution.

It also answers following questions:-

·         How do you know that you can trust your unit tests? 

·         How do you know that they’re really telling you the truth? 

·         If they don't find a bug, does that really mean that there aren't any? 

·         What if you could test your tests?

Mutation testing is one way that you can test those tests.  Mutation testing involves deliberately altering a program’s code, then re-running a suite of valid unit tests against the mutated program.  A good unit test will detect the change in the program and fail accordingly.

Aim

Tests can be created to verify the correctness of the implementation of a given software system, but the creation of tests still poses the question whether the tests are correct and sufficiently cover the requirements that have originated the implementation. Mutation testing was pioneered in the 1970s to locate and expose weaknesses in test suites. The theory was that if a mutation was introduced without the behavior (generally output) of the program being affected, this indicated either that the code that had been mutated was never executed (redundant code) or that the testing suite was unable to locate the injected fault. In order for this to function at any scale, a large number of mutations had to be introduced into a large program, leading to the compilation and execution of an extremely large number of copies of the program. This problem of the expense of mutation testing had reduced its practical use as a method of software testing, but the increased use of object oriented programming languages and unit testing frameworks has led to the creation of mutation testing tools for many programming languages as a means to test individual portions of an application.

Mutation testing overview

Mutation testing is done by selecting a set of mutation operators and then applying them to the source program one at a time for each applicable piece of the source code. The result of applying one mutation operator to the program is called a mutant. If the test suite is able to detect the change (i.e. one of the tests fails), then the mutant is said to be killed.

For example, consider the following C++ code fragment:

if (a && b) {

    c = 1;

} else {

    c = 0;

}

The condition mutation operator would replace && with || and produce the following mutant:

if (a || b) {

    c = 1;

} else {

    c = 0;

}

Now, for the test to kill this mutant, the following condition should be met:

§  Test input data should cause different program states for the mutant and the original program. For example, a test with a = 1 and b = 0 would do this.

§  The value of 'c' should be propagated to the program's output and checked by the test.

Weak mutation testing (or weak mutation coverage) requires that only the first condition is satisfied. Strong mutation testing requires that both conditions are satisfied. Strong mutation is more powerful, since it ensures that the test suite can really catch the problems. Weak mutation is closely related to code coverage methods. It requires much less computing power to ensure that the test suite satisfies weak mutation testing than strong mutation testing.

Equivalent mutants

Many mutation operators can produce equivalent mutants. For example, consider the following code fragment:

int index = 0;

while (…)

{

    …;

    index++;

    if (index == 10) {

        break;

    }

}

Boolean relation mutation operator will replace == with >= and produce the following mutant:

int index = 0;

while (…)

{

    …;

    index++;

    if (index >= 10) {

        break;

    }

}

However, it is not possible to find a test case that could kill this mutant. The resulting program is equivalent to the original one. Such mutants are called equivalent mutants.

Equivalent mutants detection is one of biggest obstacles for practical usage of mutation testing. The effort needed to check if mutants are equivalent or not, can be very high even for small programs.

Mutation operators

A variety of mutation operators were explored by researchers. Here are some examples of mutation operators for imperative languages:

§  Statement deletion.

§  Replace each boolean subexpression with true and false.

§  Replace each arithmetic operation with another one, e.g. + with *, - and /.

§  Replace each boolean relation with another one, e.g. > with >=, == and <=.

§  Replace each variable with another variable declared in the same scope (variable types should be the same).

These mutation operators are also called traditional mutation operators. Beside this, there are mutation operators for object-oriented languages, for concurrent constructions, complex objects like containers etc. They are called class-level mutation operators. For example the MuJava tool offers various class-level mutation operators such as: Access Modifier Change, Type Cast Operator Insertion, and Type Cast Operator Deletion. Moreover, mutation operators have been developed to perform security vulnerability testing of programs.

Thanks,

Mohit Bhatia