SE401 · White Box Testing · Annotated SlidesWhite-Box Testing
Original
Overview

White-Box Testing

Static white-box testing (desk checking, walkthroughs, formal inspections and their roles) and dynamic white-box testing: statement, path, branch/decision, condition and decision/condition coverage, basis path testing with flow graphs and cyclomatic complexity.

How to read this page

Every original slide is shown unchanged. Yellow marks on a slide point to a numbered note: hover or tap a mark to jump to its note, and hover a note to see its marks. Questions tagged Past exam appeared on a real paper; Prediction questions are likely angles, not confirmed. Pick an answer to each question: a wrong pick tells you why it is wrong, so you can try again.

All 72 slides
  1. 1 Slide 1
  2. 2 • To focus on white box testing • To distinguish between different types of white box
  3. 3 The slide
  4. 4 V design the
  5. 5 Should i memorize
  6. 6 • Requires access to code • Looks at the program code and performs testing by
  7. 7 • The program code truly represents what the program actually does and not just what it is intended to do!
  8. 8 Types of White Box Testing
  9. 9 • Involves only the source code and not the executables or binaries
  10. 10 • Humans can find errors that computers can’t. • Multiple humans can provide multiple perspectives.
  11. 11 • Different types • Desk checking of the code
  12. 12 • Author informally checks the code against the specifications and corrects defects found.
  13. 13 • Advantages • The programmer knows the code and programming
  14. 14 • Group oriented (as against desk checking) • Brings in multiple perspectives
  15. 15 • Group-oriented activity • Highly formal and structured
  16. 16 • Author • Author of the work product • Makes available the required material to the reviewers
  17. 17 • Typical documents circulated: • Program code
  18. 18 • Preliminary meeting (optional) • Author explains his / her perspective
  19. 19 do
  20. 20 • Prioritizing various programs • High priority / important codes subject to formal
  21. 21 Help identify / are used:
  22. 22 • “Never leave home without it!” • The code review checklist aids in organizational
  23. 23 • Done by running the executable on the machine • Entails running the product against some predefined
  24. 24 • Initial quick checks by developer • Removes “obvious” errors
  25. 25 • Exercises different parts of code • Maps parts of code to required functionality
  26. 26 • Statement coverage • Path coverage
  27. 27 de
  28. 28 • Generate test data to make the program enter the sequential block, to make it go through the entire
  29. 29 • Have data to test the THEN part • Have data to test the ELSE part
  30. 30 • There should be test cases that:
  31. 31 • Even if we were to achieve a very high level of statement coverage, it does not mean that the program is defect-free.
  32. 32 • Statement Coverage may not indicate “true coverage” • Path Coverage provides better representation
  33. 33 Slide 33
  34. 34 • Further refinement of path coverage • Makes sure each constituent condition in a boolean
  35. 35 • Finds how many functions are covered by test cases • Higher level of abstraction; hence possible to achieve
  36. 36 • Performance analysis and optimization • Resource usage analysis
  37. 37 • Which of the paths are independent? (If paths are not independent, we may be able to minimize the number
  38. 38 • Start with a conventional flow chart. • Identify the “decision points.”
  39. 39 Slide 39
  40. 40 • Start with a conventional flow chart.
  41. 41 Flow Diagram for Conventional Flow Chart Calculating Complexity
  42. 42 • Formal definitions • CC = # of predicate nodes + 1
  43. 43 Case (i): A hypothetical program with no decision node
  44. 44 Case (ii): Adding one decision node
  45. 45 • An independent path is a path in the flow graph that has not been traversed before in other paths.
  46. 46 Meaning and Interpretation of Cyclomatic Complexity
  47. 47 • Developers “not finding time for testing” • They develop blind spots for their own defects
  48. 48 the
  49. 49 • It is concerned with the degree to which test cases exercise or cover the logic of source code of the program
  50. 50 void foo(int a, int b, int x) {
  51. 51 All possible Paths
  52. 52 • A set of test cases that traverses every statement at least once.
  53. 53 • A stronger logic-coverage criterion than the statement coverage.
  54. 54 Decision Coverage (Cont’d)
  55. 55 coverage
  56. 56 • Exceptions
  57. 57 Decision Coverage
  58. 58 Problems!
  59. 59 Condition Coverage
  60. 60 Condition Coverage
  61. 61 • Four conditions
  62. 62 • Condition coverage test cases must cover the following:
  63. 63 • Condition coverage test cases must cover the following:
  64. 64 • Another set of condition-coverage test cases A B X
  65. 65 Solution! Decision/Condition Coverage
  66. 66 decisions and conditions
  67. 67 • We write sufficient test cases so that all possible combinations of condition outcomes in each decision and all points of entry are exercised at least once
  68. 68 • In Figure the following, we need to write test cases that cover the following outcome
  69. 69 • The following test cases cover all eight possibilities:
  70. 70 NOTFOUND=TRUE; DO I=1 to TABSIZE WHILE (NOTFOUND); /*SEARCH TABLE*/ ...searching logic...; END
  71. 71 • You would need eight test cases for the following decision:
  72. 72 • For programs containing only one condition per decision, a minimum test criterion is a sufficient number of test cases to
Slide 1 / 72

Slide 1

Slide 1: Slide 1
Slide 2 / 72

• To focus on white box testing • To distinguish between different types of white box

Slide 2: • To focus on white box testing • To distinguish between different types of white box
Slide 3 / 72

The slide

Slide 3: The slide
Slide 4 / 72

V design the

Slide 4: V design the
Slide 5 / 72

Should i memorize

Slide 5: Should i memorize
Slide 6 / 72

• Requires access to code • Looks at the program code and performs testing by

Slide 6: • Requires access to code • Looks at the program code and performs testing by

Notes

  1. White-box (structural) testing needs access to the code; tests are derived from its structure.

Slide 7 / 72

• The program code truly represents what the program actually does and not just what it is intended to do!

Slide 7: • The program code truly represents what the program actually does and not just what it is intended to do!
Slide 8 / 72

Types of White Box Testing

Slide 8: Types of White Box Testing
Slide 9 / 72

• Involves only the source code and not the executables or binaries

Slide 9: • Involves only the source code and not the executables or binaries

Notes

  1. Static testing: examine code without running it (reviews, walkthroughs, inspections, static analysis).

Slide 10 / 72

• Humans can find errors that computers can’t. • Multiple humans can provide multiple perspectives.

Slide 10: • Humans can find errors that computers can’t. • Multiple humans can provide multiple perspectives.
Slide 11 / 72

• Different types • Desk checking of the code

Slide 11: • Different types • Desk checking of the code
Slide 12 / 72

• Author informally checks the code against the specifications and corrects defects found.

Slide 12: • Author informally checks the code against the specifications and corrects defects found.
Slide 13 / 72

• Advantages • The programmer knows the code and programming

Slide 13: • Advantages • The programmer knows the code and programming
Slide 14 / 72

• Group oriented (as against desk checking) • Brings in multiple perspectives

Slide 14: • Group oriented (as against desk checking) • Brings in multiple perspectives

Notes

  1. Formal inspection: author, moderator, reader, recorder/scribe, inspectors; preparation, meeting, rework, follow-up.

Exam angle

PredictionQ1

In a formal inspection, who leads the meeting and keeps it on track?

Slide 15 / 72

• Group-oriented activity • Highly formal and structured

Slide 15: • Group-oriented activity • Highly formal and structured
Slide 16 / 72

• Author • Author of the work product • Makes available the required material to the reviewers

Slide 16: • Author • Author of the work product • Makes available the required material to the reviewers
Slide 17 / 72

• Typical documents circulated: • Program code

Slide 17: • Typical documents circulated: • Program code
Slide 18 / 72

• Preliminary meeting (optional) • Author explains his / her perspective

Slide 18: • Preliminary meeting (optional) • Author explains his / her perspective
Slide 19 / 72

do

Slide 19: do
Slide 20 / 72

• Prioritizing various programs • High priority / important codes subject to formal

Slide 20: • Prioritizing various programs • High priority / important codes subject to formal
Slide 21 / 72

Help identify / are used:

Slide 21: Help identify / are used:
Slide 22 / 72

• “Never leave home without it!” • The code review checklist aids in organizational

Slide 22: • “Never leave home without it!” • The code review checklist aids in organizational
Slide 23 / 72

• Done by running the executable on the machine • Entails running the product against some predefined

Slide 23: • Done by running the executable on the machine • Entails running the product against some predefined
Slide 24 / 72

• Initial quick checks by developer • Removes “obvious” errors

Slide 24: • Initial quick checks by developer • Removes “obvious” errors
Slide 25 / 72

• Exercises different parts of code • Maps parts of code to required functionality

Slide 25: • Exercises different parts of code • Maps parts of code to required functionality
Slide 26 / 72

• Statement coverage • Path coverage

Slide 26: • Statement coverage • Path coverage

Notes

  1. Coverage criteria from weak to strong: statement → branch/decision → condition → decision/condition → multiple-condition → path.

Slide 27 / 72

de

Slide 27: de
Slide 28 / 72

• Generate test data to make the program enter the sequential block, to make it go through the entire

Slide 28: • Generate test data to make the program enter the sequential block, to make it go through the entire

Notes

  1. Statement coverage: every statement executed at least once. Weak: can miss the false branch of an if.

  2. Trap 100% statement coverage does NOT imply 100% branch coverage; 100% branch coverage DOES imply 100% statement coverage.

Exam angle

PredictionQ1

100% decision (branch) coverage guarantees 100% statement coverage.

PredictionQ2

100% statement coverage guarantees 100% decision coverage.

Slide 29 / 72

• Have data to test the THEN part • Have data to test the ELSE part

Slide 29: • Have data to test the THEN part • Have data to test the ELSE part
Slide 30 / 72

• There should be test cases that:

Slide 30: • There should be test cases that:
Slide 31 / 72

• Even if we were to achieve a very high level of statement coverage, it does not mean that the program is defect-free.

Slide 31: • Even if we were to achieve a very high level of statement coverage, it does not mean that the program is defect-free.
Slide 32 / 72

• Statement Coverage may not indicate “true coverage” • Path Coverage provides better representation

Slide 32: • Statement Coverage may not indicate “true coverage” • Path Coverage provides better representation
Slide 33 / 72

Slide 33

Slide 33: Slide 33
Slide 34 / 72

• Further refinement of path coverage • Makes sure each constituent condition in a boolean

Slide 34: • Further refinement of path coverage • Makes sure each constituent condition in a boolean
Slide 35 / 72

• Finds how many functions are covered by test cases • Higher level of abstraction; hence possible to achieve

Slide 35: • Finds how many functions are covered by test cases • Higher level of abstraction; hence possible to achieve
Slide 36 / 72

• Performance analysis and optimization • Resource usage analysis

Slide 36: • Performance analysis and optimization • Resource usage analysis
Slide 37 / 72

• Which of the paths are independent? (If paths are not independent, we may be able to minimize the number

Slide 37: • Which of the paths are independent? (If paths are not independent, we may be able to minimize the number
Slide 38 / 72

• Start with a conventional flow chart. • Identify the “decision points.”

Slide 38: • Start with a conventional flow chart. • Identify the “decision points.”
Slide 39 / 72

Slide 39

Slide 39: Slide 39
Slide 40 / 72

• Start with a conventional flow chart.

Slide 40: • Start with a conventional flow chart.
Slide 41 / 72

Flow Diagram for Conventional Flow Chart Calculating Complexity

Slide 41: Flow Diagram for Conventional Flow Chart Calculating Complexity
Slide 42 / 72

• Formal definitions • CC = # of predicate nodes + 1

Slide 42: • Formal definitions • CC = # of predicate nodes + 1

Notes

  1. Cyclomatic complexity V(G) = E − N + 2 = decision nodes + 1 = number of independent paths (minimum tests for basis path coverage).

Exam angle

PredictionQ1

A flow graph has 11 edges and 9 nodes. V(G) =

Slide 43 / 72

Case (i): A hypothetical program with no decision node

Slide 43: Case (i): A hypothetical program with no decision node
Slide 44 / 72

Case (ii): Adding one decision node

Slide 44: Case (ii): Adding one decision node
Slide 45 / 72

• An independent path is a path in the flow graph that has not been traversed before in other paths.

Slide 45: • An independent path is a path in the flow graph that has not been traversed before in other paths.
Slide 46 / 72

Meaning and Interpretation of Cyclomatic Complexity

Slide 46: Meaning and Interpretation of Cyclomatic Complexity
Slide 47 / 72

• Developers “not finding time for testing” • They develop blind spots for their own defects

Slide 47: • Developers “not finding time for testing” • They develop blind spots for their own defects
Slide 48 / 72

the

Slide 48: the
Slide 49 / 72

• It is concerned with the degree to which test cases exercise or cover the logic of source code of the program

Slide 49: • It is concerned with the degree to which test cases exercise or cover the logic of source code of the program
Slide 50 / 72

void foo(int a, int b, int x) {

Slide 50: void foo(int a, int b, int x) {
Slide 51 / 72

All possible Paths

Slide 51: All possible Paths
Slide 52 / 72

• A set of test cases that traverses every statement at least once.

Slide 52: • A set of test cases that traverses every statement at least once.
Slide 53 / 72

• A stronger logic-coverage criterion than the statement coverage.

Slide 53: • A stronger logic-coverage criterion than the statement coverage.
Slide 54 / 72

Decision Coverage (Cont’d)

Slide 54: Decision Coverage (Cont’d)
Slide 55 / 72

coverage

Slide 55: coverage
Slide 56 / 72

• Exceptions

Slide 56: • Exceptions
Slide 57 / 72

Decision Coverage

Slide 57: Decision Coverage
Slide 58 / 72

Problems!

Slide 58: Problems!
Slide 59 / 72

Condition Coverage

Slide 59: Condition Coverage
Slide 60 / 72

Condition Coverage

Slide 60: Condition Coverage
Slide 61 / 72

• Four conditions

Slide 61: • Four conditions

Notes

  1. Condition coverage: every individual condition in a decision takes both true and false. Doesn’t guarantee decision coverage.

Slide 62 / 72

• Condition coverage test cases must cover the following:

Slide 62: • Condition coverage test cases must cover the following:
Slide 63 / 72

• Condition coverage test cases must cover the following:

Slide 63: • Condition coverage test cases must cover the following:
Slide 64 / 72

• Another set of condition-coverage test cases A B X

Slide 64: • Another set of condition-coverage test cases A B X
Slide 65 / 72

Solution! Decision/Condition Coverage

Slide 65: Solution! Decision/Condition Coverage

Exam angle

PredictionQ1

For "if (A && B)", tests (A=T, B=F) and (A=F, B=T) achieve:

Slide 66 / 72

decisions and conditions

Slide 66: decisions and conditions
Slide 67 / 72

• We write sufficient test cases so that all possible combinations of condition outcomes in each decision and all points of entry are exercised at least once

Slide 67: • We write sufficient test cases so that all possible combinations of condition outcomes in each decision and all points of entry are exercised at least once
Slide 68 / 72

• In Figure the following, we need to write test cases that cover the following outcome

Slide 68: • In Figure the following, we need to write test cases that cover the following outcome
Slide 69 / 72

• The following test cases cover all eight possibilities:

Slide 69: • The following test cases cover all eight possibilities:
Slide 70 / 72

NOTFOUND=TRUE; DO I=1 to TABSIZE WHILE (NOTFOUND); /*SEARCH TABLE*/ ...searching logic...; END

Slide 70: NOTFOUND=TRUE; DO I=1 to TABSIZE WHILE (NOTFOUND); /*SEARCH TABLE*/ ...searching logic...; END
Slide 71 / 72

• You would need eight test cases for the following decision:

Slide 71: • You would need eight test cases for the following decision:
Slide 72 / 72

• For programs containing only one condition per decision, a minimum test criterion is a sufficient number of test cases to

Slide 72: • For programs containing only one condition per decision, a minimum test criterion is a sufficient number of test cases to