Glossary

Study as flashcards →

86 terms

Anomaly: A discrepancy between the actual results and the expected results of a test, revealed during dynamic testing when a defect causes a failure; a false-negative result occurs when the tester does not observe the anomaly, and an observed anomaly triggers further investigation, usually starting with a defect report.
Appraisal costs: The cost of all activities aimed at defect detection, such as performing static testing and dynamic testing and reviewing work products.
Average savings per defect: A cost-benefit measure calculated as the average external failure costs minus the sum of the average appraisal costs and the average internal failure costs, used to articulate the added value of finding a defect through testing before release.
Breadth-first: A risk-based test prioritization approach in which at least one test for each risk is assigned highest priority and all other tests are prioritized by their level of risk covered; appropriate when stakeholders want an overall view of product quality as early as possible.
Cost of quality: A means of quantifying the total cost of quality-related efforts and defects by classifying project and operational costs into four categories related to product defect costs: defect prevention, appraisal, internal failure, and external failure costs.
Defect: A flaw in a work product that can cause a failure; defect management covers the activities that begin after observing actual results differing from expected results, other standards using the terms incident management or anomaly management for essentially the same activities.
Defect management: The activities that begin after observing actual results that differ from expected results; an effective defect management process gives the test team and stakeholders insight into the state of a project throughout its SDLC and supports deciding which defects to fix.
Defect management committee: A cross-functional team, sometimes convened in a triage meeting, that is generally responsible for managing the defects for a project; it determines whether each report is a valid defect and whether it should be fixed, rejected, or deferred, weighing benefits, risks and costs.
Defect prevention costs: The cost of all activities that are planned and proactive to prevent poor quality, such as qualifying developers through training in maintainable or secure code, reviewing the test basis as early as possible, and appropriate communication within the team.
Defect report: The record created when an observed anomaly is investigated; in most environments it must include a title with a short summary, a detailed description preferably with steps to reproduce, the severity of the impact, and the priority to fix.
Defect workflow: The sequence of defect states a defect report moves through (e.g. OPEN/NEW, IN PROGRESS, REJECTED, RESOLVED, CLOSED); in most states one person owns the report and performs a task, and the workflow is adapted to a given context following good practices.
Depth-first: A risk-based test prioritization approach in which tests are executed in strict descending order of the levels of risk they cover, starting with the highest; appropriate when the highest-level risks must be mitigated as early as possible.
Experience-based testing: A testing approach whose management depends on the availability of testers with strong domain knowledge, since the technique relies on the tester's skill and experience to design and execute tests; the test manager must consider tester availability and expertise as a test resource constraint when planning this test type.
Expert-based estimation technique: A category of test estimation technique that relies on the opinions and judgments of experts or team members, such as the Delphi method and planning poker; suited to topics of high complexity.
External failure costs: The cost of all non-value-added and reactive activities arising when a defective product is delivered to the customer post-release, such as loss of revenue, assets, human health, human life or the environment, legal costs, and fixing field defects flagged by customers.
Failure: The event caused by a defect during dynamic testing that results in a discrepancy (an anomaly) between the actual results and the expected results of a test; a failed test does not always result in a defect report, for example in test-driven development where component tests must initially fail before development of the component is complete.
Functional testing: A test type whose management focuses on ensuring that all functionalities are thoroughly tested and meet the defined requirements, requiring a test strategy that aligns with functional requirements and project objectives and resource coordination to cover all functional aspects of the system.
Goal Question Metric (GQM): An analytical approach that defines measurement goals, refines them into questions that express the quality aspect from the stakeholders' viewpoint, and then selects metrics that provide the information needed to answer those questions.
Heavyweight techniques: Formal risk-based testing techniques that use defined procedures and detailed documentation, involve broad groups of stakeholders, and use mathematical formulas; examples include hazard analysis, cost of exposure, FMEA, and fault tree analysis.
Hybrid software development model: A software development model that integrates elements from both traditional sequential approaches and Agile practices to suit specific project needs or organizational transitions.
Hygiene factor: A factor that is usually taken for granted so that fulfilling it does not automatically increase satisfaction, but whose absence has a demotivating effect; examples include appropriate remuneration, appreciative personnel policy, pleasant working conditions, safety, and good interpersonal relationships.
IDEAL: A test process improvement model, based on similar ideas as the plan-do-check-act (PDCA) cycle, whose five phases are Initiating, Diagnosing, Establishing, Acting and Learning.
Incremental development model: A software development lifecycle model that the syllabus, in its introduction to Section 1.5 (Improving the Test Process), notes alongside Agile software development as one to which the described test-process-improvement techniques can be applied.
Indicator: A measure that provides an estimate or evaluation of specified attributes derived from a model with respect to defined information needs.
Internal failure costs: The cost of all reactive activities for defects found before release, such as fixing defects found during testing and providing workarounds.
Iterative development model: A software development lifecycle model (e.g., SCRUM) contrasted with the sequential development model; test management activities under it use iterative estimation as part of story planning per iteration, minimal testware focused on acceptance criteria and definition of done, integrated roles such as a facilitator or coach, CI/CD-centred automation tooling, and continuous reporting with real-time dashboards.
Lightweight techniques: Less thorough risk-based testing techniques that require less effort from the test team and stakeholders, with risk factors usually reduced to risk impact and risk likelihood on an ordinal scale; examples include SST, PRAM, and PRISMA.
Measure: The number or category assigned to an attribute of an entity by making a measurement.
Methodological competence: One of the four areas of competence, comprising general skills that a person can use independently in a domain to perform complex or novel tasks; examples include analytical, conceptual, and judgmental skills.
Metric: A measurement scale and the method used for measurement; test metrics, which apply this concept to testing, help determine whether the test objectives have been met and are categorized as project, product, and process metrics.
Metric-based estimation technique: A category of test estimation technique that derives estimates from data, such as ratios and extrapolation using historical data from previous or similar projects; suited to topics of low complexity.
Motivation-hygiene theory: Herzberg's theory that distinguishes between motivators, which are perceived consciously and can lead to growth and satisfaction, and hygiene factors, which are taken for granted and have a demotivating effect when missing.
Motivator: A factor that is perceived consciously and can lead to growth and satisfaction for test team members; examples include recognition and appreciation, increased responsibility and autonomy, interesting and challenging tasks, and professional advancement.
Non-functional testing: A test type whose management revolves around verifying system attributes like performance and security, requiring the test manager to plan for the specialized tools, environments and skills needed to assess these attributes.
Personal competence: One of the four areas of competence, comprising the ability and willingness to develop oneself, one's talent, motivation, attitudes and personality; examples include self-management, personal responsibility, ability to receive criticism, reliability, resilience, discipline, openness to change, and the ability to delegate.
Phase containment: Removing each defect within the same phase in which it was introduced; the cost of quality is minimized when the software process achieves perfect phase containment.
Planning poker: An expert-based test estimation technique that relies on the judgments of team members and is easy to perform; the syllabus associates it with Agile software development models.
Priority: A mandatory defect-report item expressing how urgently the anomaly should be fixed relative to other work.
Process metrics: Test metrics that measure the capability of the testing process and the effectiveness of testing, and are therefore used to report process-related effectiveness and efficiency.
Product metrics: Test metrics that measure product attributes, such as the degree to which the product meets the quality expectations of its intended users.
Product risk: A potential situation where quality problems may exist in a product; when tests reveal defects, testing helps mitigate the product risk by providing awareness and opportunities to deal with them before release.
Professional competence: One of the four areas of competence, consisting of the skills to perform specialized tasks; examples include skills in test techniques, technological and business (domain) expertise, and project management skills. It is the area the ISTQB focuses on developing because it is specific to testing.
Project metrics: Test metrics that measure progress against existing project exit criteria, such as the percentage of tests executed, passed, and failed.
Quality risk: A risk related to the quality of the test object that is the focus of risk-based testing; quality risks are identified through techniques such as expert interviews, independent assessments, retrospectives, risk workshops, brainstorming, checklists and referring to past experience, and are then assessed and categorized by quality characteristics impacted.
Retrospective: A meeting in which a team reviews its methods and collaboration, captures lessons learned (good and bad), and decides on changes and actions to achieve improvements; a typical retrospective consists of introduction, collect data, derive improvements, decide on improvement actions, and close retrospective.
Return on investment (ROI): A measure of the value a test tool delivers relative to its costs; a positive ROI is required for tool introduction, it is the responsibility of the test manager to calculate and evaluate it, and the total ROI for an organization is usually a mix of the ROI of all tools used for testing.
Risk analysis: The part of the generic risk management process that consists of risk identification and risk assessment.
Risk assessment: Part of risk analysis in the generic risk management process; quality risk assessment includes categorizing identified risks by type (product or project risk) and by quality characteristics impacted, and typically involves assessing the risk likelihood and risk impact for each risk item to determine its risk level.
Risk control: The part of the generic risk management process that consists of risk monitoring and risk mitigation.
Risk identification: Part of risk analysis in the generic risk management process, in which the test manager gathers risks from as broad a sample of stakeholders as possible; missing key stakeholders at this stage can be problematic, and the process often produces by-products such as project risks or issues in referenced documents.
Risk impact: One of the two factors, together with risk likelihood, that the test manager combines to determine the risk level of a risk item; it can be expressed quantitatively (e.g., as a monetary amount) or, more commonly, qualitatively on an ordinal scale.
Risk level: A rating determined by combining the risk likelihood of occurrence and the risk impact upon occurrence for a risk item; usually a qualitative, relative rating on an ordinal scale.
Risk likelihood: One of the two factors, together with risk impact, that the test manager combines to determine the risk level of a risk item; it can be expressed quantitatively (e.g., as a percentage) or, more commonly, qualitatively on an ordinal scale (e.g., very high, high, medium, low, very low).
Risk management: The generic process, consisting of risk analysis (risk identification and risk assessment) and risk control (risk monitoring and risk mitigation), that risk-based testing follows; the test manager requires active involvement in project risk management with a focus on risks related to quality assurance.
Risk matrix: A tool that combines risk likelihood and risk impact values, typically on ordinal scales, to create an aggregate risk level interpreted as a qualitative, relative rating.
Risk mitigation: Part of risk control in the generic risk management process; in software development, testing is the most important quality risk mitigation activity, and mitigation is distributed over several test activities such as focusing test planning on the correct areas, guiding test condition selection in test analysis, and governing execution sequence via risk-based prioritization.
Risk monitoring: Part of risk control in the generic risk management process; test monitoring should include risk monitoring, which covers tracking the evolution of known quality risks and analyzing any new quality risks so the risk register can be adjusted.
Risk-based testing: Testing that involves the identification, assessment, monitoring and mitigation of risks to drive testing; the higher the risk level, the earlier testing should begin and the more intense and prolonged the test effort should be.
Root cause analysis: An analytical test process improvement technique that studies problems to identify their root causes, for example by selecting a set of defects, identifying clusters, and using cause-effect (Ishikawa or fishbone) diagrams, so that solutions remove the causes rather than merely the symptoms.
S.M.A.R.T. goal methodology: A methodology for defining test objectives and exit criteria so that each is Specific (clear and unambiguous), Measurable (quantifiable with criteria for progress), Achievable (feasible given resources, timeframe and capabilities), Relevant (aligned with overall project objectives), and Timely (with a defined deadline).
Sequential development model: A software development lifecycle model (e.g., V-Model) contrasted with the iterative development model; test management activities under it use early detailed estimation for each test level, testware including a strategy, plan, cases, schedule and reports, a test manager who oversees decisions and team management, and milestone-based reporting.
Severity: A mandatory defect-report item expressing the degree of impact of the anomaly on the system under test and/or the product stakeholders.
Skills matrix: A document in which the required skills for a project may be recorded and compared against the existing skills of the test team, so that test management can identify capabilities and competence-development needs.
Social competence: One of the four areas of competence, comprising skills related to communication, cooperation, and conflict management in intra- and intercultural contexts; examples include communication skills, conflict resolution, teamwork, adaptability, and assertiveness.
Software development lifecycle (SDLC): The model used for a project (e.g., sequential or iterative/Agile) that a test manager must understand and align testing with; the SDLC model used determines appropriate test levels, test effort, and entry/exit criteria, and effective defect reports and defect workflows must suit the SDLC in use.
Stakeholder matrix: A strategic tool, also called the power-interest matrix, that guides test managers in prioritizing stakeholder engagement and managing expectations; it has four quadrants: promoters, latents, defenders, and apathetics.
Test completion: The activity of finalizing and archiving the test artifacts, evaluating the test process and the test product, identifying test process improvement actions, and communicating the test closure to relevant stakeholders. It usually occurs at project milestones.
Test completion report: A report created at test completion that collects and summarizes relevant information from testware such as test plans, test results, test progress reports, and defect reports; it is approved and communicated to relevant stakeholders.
Test control: The ongoing activity that compares actual progress against the test plan and implements corrective actions as needed to guide the testing toward meeting the test strategies and objectives.
Test estimation: A test management activity that estimates how much time, effort, and cost a task will take to complete; effort is usually calculated in person hours or story points, and it is one of the major tasks in test management.
Test level: A grouping of test activities (e.g., component testing, component integration testing, system integration testing, system testing, and acceptance testing) that are organized and managed together; a test strategy may exist for a single test level, and test management activities differ across levels and across SDLC models.
Test Maturity Model integration (TMMi): A model-based test process improvement framework composed of five maturity levels; each level except level 1 contains test process areas and improvement goals, supported by practices, sub-practices and examples.
Test metrics: Indicators that help determine whether the test objectives have been met; they can be categorized as project metrics (progress against project exit criteria), product metrics (product attributes such as how far the product meets users' quality expectations), and process metrics (the capability and effectiveness of the testing process).
Test monitoring: The ongoing activity of collecting and recording test results, identifying deviations from planned testing, identifying and analyzing new risks that require testing, and monitoring changes for identified risks.
Test objective: A reason or purpose for testing that, together with the exit criteria, is defined for a project and should address all targeted aspects of quality and quantity in a measurable or evaluable way, ideally defined using the S.M.A.R.T. goal methodology; test metrics help determine whether the test objectives have been met.
Test plan: The work product established during test planning that must be accepted by all stakeholders; each level of testing (project, program, or portfolio) can have its own test plan that aligns with the higher level test plan.
Test planning: The test management activity of identifying the activities and resources required to achieve the test objectives, including defining the test objectives, test approach, test scope, test resources, test schedule, test deliverables, and test participants.
Test process improvement: The activity of optimizing the effectiveness and efficiency of testing by learning from existing best practices and one's own errors; it is typically most useful at an organizational level but can also be tailored to and applied at a project or team level.
Test progress: The state of the testing activity relative to the test plan, which the test manager must assess and report to project stakeholders; test metrics primarily used to measure and report progress (e.g., percentage of tests executed, passed, and failed) appear in test progress reports.
Test stakeholders: Individuals or groups with a direct or indirect interest in the product's quality, such as developers, testers, project managers, product owners, business users, the operations team, customers, and users.
Test strategy: A description of the approach to testing in a specific context so that the organization's objectives, particularly those relating to product quality and test activities, can be met; a project test strategy is the main outcome of test planning and is typically documented in a test plan.
Test type: A group of test activities aimed at testing specific characteristics of a component or system, such as functional, non-functional, black-box, or white-box testing; a test strategy may exist for a single test type, and effective test management requires an integrated approach considering the unique demands of each test type.
Three-point estimation: A test estimation technique that uses the optimistic, pessimistic, and most likely estimates to calculate the expected value and the standard deviation of the estimate, providing a measure of the uncertainty or variability of the estimate.
TPI NEXT: A model-based test process improvement model that defines 16 key areas, each covering a specific aspect of the test process, with four maturity levels per key area assessed via specific checkpoints and summarized in a maturity matrix.
Tuckman model: A model of small-group development whose phases are Forming, Storming, Norming, Performing, and Adjourning; each phase requires different situational leadership skills from a test team leader.
Wideband Delphi: An expert-based test estimation technique that relies on the consensus judgment of experts; the syllabus associates it with sequential development models.