Research & Evidence-Based Practice Flashcards

What is 'gray literature' in the context of software testing evidence gathering?

Answer: Non-peer-reviewed sources such as technical reports, blogs, and industry white papers

Gray literature includes practitioner knowledge published outside formal academic channels and is valuable for capturing industrial practice.

In a software testing experiment comparing two code coverage criteria, what is the 'dependent variable'?

Answer: The outcome being measured, such as defect detection rate or branch coverage achieved

The dependent variable is what is measured as the outcome; the independent variable is the coverage criterion being manipulated.

What does 'inter-rater reliability' measure in a qualitative software testing study?

Answer: The degree to which independent raters agree when classifying or coding the same data

High inter-rater reliability indicates that different researchers reach the same conclusions when analyzing qualitative data, strengthening credibility.

A meta-analysis of unit testing studies reports a 'weighted mean effect size.' Why is weighting applied?

Answer: To give larger or more precise studies more influence on the combined estimate

Weighting ensures that higher-quality or larger studies contribute more to the pooled effect size estimate than small, imprecise studies.

What is the primary risk of 'HARKing' (Hypothesizing After Results are Known) in software testing research?

Answer: It presents post-hoc observations as pre-planned hypotheses, inflating false-positive rates

HARKing misrepresents exploratory findings as confirmatory, bypassing the protection against chance results that pre-registration provides.

Which EBSE (Evidence-Based Software Engineering) step involves critically appraising study quality before applying findings?

Answer: Evidence evaluation

Evidence evaluation assesses the rigor, relevance, and risk of bias in studies before their conclusions are used to guide practice.

A practitioner reads that 'automated UI testing reduces regression time by 40%.' What follow-up question is most aligned with evidence-based practice?

Answer: What was the study context, sample size, and comparison baseline that produced that figure?

Evidence-based practice requires understanding the study's context and methodology to judge whether the finding applies to one's own situation.