Wednesday, 22 October 2014

[Coverage] some terms and definitions

Terms

Terms

  • condition:
    • a condition is a leaf-level Boolean expression (it cannot be broken down into a simpler Boolean expression).
    • condition: 2 > 1, a == b
    • not condition: (2 > 1) || (a == b)
  • decision:
    • a Boolean expression composed of conditions and zero or more Boolean operators. a decision without a Boolean operator is a condition
    • condition is a decision
    • decision: (2 > 1) || (a == b)

Coverage

  • statement coverage:
    • each statement is executed at least once
  • function coverage:
    • each function is called at least once
  • branch coverage:
    • each branch of control structure is executed at least once
  • condition coverage:
    • every condition in a decision in the program has taken all possible outcomes at least once.
  • decision coverage
    • Every point of entry and exit in the program has been invoked at least once, and every decision in the program has taken all possible outcomes at least once
  • condition/decision coverage:
    • every point of entry and exit in the program has been invoked at least once, every condition in a decision in the program has taken all possible outcomes at least once, and every decision in the program has taken all possible outcomes at least once.
  • modified condition/decision coverage:
    • every point of entry and exit in the program has been invoked at least once, every condition in a decision in the program has taken on all possible outcomes at least once, and each condition has been shown to affect that decision outcome independently. A condition is shown to affect a decision's outcome independently by varying just that condition while holding fixed all other possible conditions. The condition/decision criterion does not guarantee the coverage of all conditions in the module because in many test cases, some conditions of a decision are masked by the other conditions. Using the modified condition/decision criterion, each condition must be shown to be able to act on the decision outcome by itself, everything else being held fixed. The MC/DC criterion is thus much stronger than the condition/decision coverage.
    • Independence of a condition is shown by proving that only one condition changes at a time.
Examples
  • if (a && b) {}
    • [(a=true, b=false), (a=false, b=true)] to satisfy condition coverage, but not for branch coverage and decision coverage, since a && b are both true
    • [(a=true, b=false), (a=false, b=false)] to satisfy branch coverage and decision coverage, but not for condition coverage since b=true is not covered
    • [(a=true, b=false), (a=false, b=true), (a=false, b=false) ] to satisfy condition/decision coverage but not mc/dc because each condition (a or b) is evaluated to true once and false once but do not have an effect on the outcome.
      • (a=true, b=false), (a=false, b=false) -- this is not a effective test case in mcdc since the outcome of both is the same: false, and the true and false of a don't have effect on final outcome
      • (a=true, b=true), (a=false, b=true) -- it is effective test case in mcdc since the true and false of a do have effect on final outcome: true or false
    • [(a=true, b=true), (a=false, b=true), (a=true, b=false) ]
      • condition/decision coverage
      • mcdc
        • (a=true, b=true), (a=false, b=true)
        • (a=true, b=true), (a=true, b=false)

branch vs. decision coverage:

As ISTQB Foundation book gives, branch coverage is closely related to decision coverage and at 100% coverage they give exactly the same results. Decision coverage measures the coverage of conditional branches; branch coverage measures the coverage of both conditional and unconditional branches. The Syllabus uses decision coverage, as it is the source of the branches. Some coverage measurement tools may talk about branch coverage when they actually mean decision coverage. (c) ISTQB foundation book. [From http://stackoverflow.com/questions/12398494/is-branch-coverage-the-same-as-decision-coverage]

References