CTFL 4.4.3: Explain checklist-based testing

• a
  Identical testing on every release, because the checklist fixes the steps to be followedA high-level checklist gives guidelines rather than fixed steps, so the testing varies between runs.
• b
  Full automation of the checks, because checklist items are meant to be tool-checkableChecklists should not contain items that can be checked automatically.
• ✓
  Correct answer: Some variation between releases, giving potentially greater coverage but less repeatabilityHigh-level checklists trade repeatability for potentially greater coverage.
• d
  A stable checklist, because its entries keep their effectiveness as the developers learnEntries gradually lose effectiveness as developers learn to avoid the same errors, so updates are needed.

In the absence of detailed test cases, checklist-based testing can provide guidelines and some degree of consistency for the testing. If the checklists are high-level, some variability in the actual testing is likely to occur, resulting in potentially greater coverage but less repeatability. Checklists should not contain items that can be checked automatically, and they should be regularly updated based on defect analysis.

Syllabus section 4.4.3

• ✓
  Correct answer: A list of conditions or rules to verify, drawn from experienceTesting is guided by a checklist of conditions or rules, usually from experience.
• b
  The measured statement and branch coverage of the codeUsing measured code coverage to guide tests describes white-box, not checklist, testing.
• c
  A formal decision table covering every rule combinationA decision table is a separate black-box technique, not what defines checklist-based testing.
• d
  Randomly generated inputs sampled across the input domainRandom input sampling is not the basis of checklist-based testing, which follows a checklist.

In checklist-based testing the tester designs, implements, and executes tests to cover the items on a checklist of conditions, rules, or things to verify. Checklists are typically built from experience, knowledge of what is important to the user, and an understanding of how software fails.

Syllabus section 4.4.3

• a
  It can only be applied to the internal structure of the codeChecklist-based testing is experience-based and does not work from the code's structure.
• b
  It requires a formal model before any test can be designedChecklist-based testing needs only a checklist, not a formal model, to begin.
• c
  It always produces low-level test cases with exact input valuesChecklist items are typically high-level, not exact low-level cases with specific inputs.
• ✓
  Correct answer: High-level checklist entries can be tested in inconsistent waysHigh-level checklist items can be interpreted and tested inconsistently between runs.

Checklist entries are often high-level (for example, 'check usability'), so different testers, or the same tester at different times, may interpret and exercise them differently, leading to inconsistent coverage. Checklists also need periodic review and updating to stay effective.

Syllabus section 4.4.3