HSRT Assessing Credibility of Claims 2 — Questions and Answers
Question 1: A health blogger claims that 'Vitamin C megadoses cure cancer, and thousands of patients have been healed.' Which credibility criterion is MOST undermined by this statement?
- Verifiability — the claim lacks citations, study references, or reproducible evidence (Correct answer)
- Expertise — the blogger may not have medical credentials
- Consistency — the claim contradicts itself
- Objectivity — the blogger has a financial interest in supplements
Correct answer: Verifiability — the claim lacks citations, study references, or reproducible evidence
A credible claim must be verifiable through reproducible, referenced evidence. 'Thousands healed' without citations is unverifiable anecdote.
Verifiability is a foundational criterion of credibility: claims should be traceable to primary sources that can be independently examined and reproduced. An anecdotal count ('thousands of patients') without study references, author names, or publication data cannot be verified. While expertise and potential bias are also relevant credibility criteria, the most immediately undermined criterion here is verifiability. This aligns with Facione's critical thinking standard of assessing the epistemic quality of claims before accepting them.
Question 2: A pharmaceutical company publishes a study showing its new antibiotic is 40% more effective than the standard of care. The study was funded exclusively by the company. Which credibility concern is MOST salient?
- Conflict of interest — industry-funded studies have a documented tendency toward favorable outcomes (Correct answer)
- Sample size — the study may not have enrolled enough patients
- Peer review — the journal may not have used double-blind review
- Statistical power — the 40% difference may not be clinically meaningful
Correct answer: Conflict of interest — industry-funded studies have a documented tendency toward favorable outcomes
Industry funding is a well-documented source of bias that inflates positive findings. This represents a conflict of interest that compromises credibility.
Numerous systematic reviews have shown that industry-funded clinical trials are significantly more likely to report favorable results than independently funded studies, a phenomenon known as funding bias or sponsorship bias. This is a conflict of interest that directly threatens research credibility. While sample size, peer review quality, and clinical significance are all legitimate concerns, the most salient and immediate credibility issue presented in this scenario is the funding source. Credibility assessment includes examining who benefits from a claim being accepted.
Question 3: A respiratory therapist hears a claim from a colleague: 'High-flow oxygen always prevents intubation in COPD exacerbations.' Which credibility question should be asked FIRST?
- Is this consistent with current evidence-based guidelines for COPD management? (Correct answer)
- How long has the colleague been practicing respiratory therapy?
- Has the colleague published this claim in a journal?
- Was this observed in more than one patient?
Correct answer: Is this consistent with current evidence-based guidelines for COPD management?
Evidence-based guidelines synthesize the best available research and are the primary reference for assessing the credibility of clinical claims.
Clinical claims should be evaluated against current evidence-based guidelines, which represent systematic reviews of the available literature. In fact, uncontrolled high-flow oxygen in COPD exacerbations is potentially harmful due to hypercapnic drive suppression — guidelines recommend controlled O2 titration to SpO2 88–92%. Colleague experience is relevant but subordinate to guideline-level evidence. The first credibility check for any clinical claim is: does the evidence support it? This reflects Facione's credibility assessment criterion of consistency with known facts.
Question 4: A hospital administrator cites a testimonial from one nurse as evidence that a new scheduling system 'greatly improved staff satisfaction.' Which limitation of this evidence is MOST critical to identify?
- A single testimonial is anecdotal and cannot represent the distribution of staff experiences (Correct answer)
- The nurse may be biased because she helped design the scheduling system
- Testimonials are never credible in healthcare settings
- The administrator should have surveyed only full-time nurses
Correct answer: A single testimonial is anecdotal and cannot represent the distribution of staff experiences
A single testimonial is an n=1 anecdote and provides no information about how representative that experience is across the broader staff population.
Anecdotal evidence — a single case or testimonial — lacks representativeness. It conveys one person's experience but provides no information about the central tendency or variance of staff satisfaction across the unit or hospital. While the nurse's potential involvement in design is a legitimate bias concern, the most fundamental limitation is the sample size of one. Credible evidence for a policy claim requires systematic data collection (surveys, structured interviews) from a representative sample. This addresses Facione's criterion of adequate sample or evidence base.
Question 5: A clinical dietitian encounters a study that claims: 'Our proprietary protein supplement increases muscle mass by 300% in athletes.' The use of '300%' should trigger which credibility scrutiny?
- Proportionality — extraordinary claims require extraordinary evidence; a 300% effect size is implausibly large (Correct answer)
- Unit analysis — the percentage unit is incorrect for measuring muscle mass
- Replication — the study should be replicated in non-athletes first
- Disclosure — the supplement ingredients should be fully listed
Correct answer: Proportionality — extraordinary claims require extraordinary evidence; a 300% effect size is implausibly large
Extraordinary claims demand extraordinary evidence. A 300% increase in muscle mass far exceeds known physiological limits, warranting extreme skepticism.
The principle of proportionality of evidence holds that the burden of proof increases with the magnitude of a claim. A 300% increase in muscle mass would be physiologically unprecedented — known ergogenic interventions (resistance training, optimal protein timing) yield far more modest results. This implausibility should trigger immediate skepticism. Credibility assessment requires checking whether claimed effect sizes are biologically plausible, consistent with established physiology, and supported by methodology rigorous enough to support such a strong claim.
Question 6: Two systematic reviews report conflicting conclusions about the effectiveness of acupuncture for chronic low back pain. Which approach BEST assesses the relative credibility of each review?
- Compare their methodological quality: study selection criteria, risk of bias assessment tools, and heterogeneity management (Correct answer)
- Accept the most recently published review as more credible because newer evidence supersedes older evidence
- Accept the review that included more total studies regardless of their quality
- Defer to the review authored by researchers affiliated with a university hospital
Correct answer: Compare their methodological quality: study selection criteria, risk of bias assessment tools, and heterogeneity management
Methodological quality — not recency, quantity, or institutional prestige — is the primary determinant of a systematic review's credibility.
Systematic reviews vary widely in methodological rigor. The AMSTAR-2 tool assesses review quality across domains including: pre-registration of protocol, comprehensive search strategy, duplicate screening, risk of bias assessment of included studies, and appropriate handling of heterogeneity. A well-conducted older review may be more credible than a poorly conducted newer one. Quantity of included studies does not ensure quality if those studies are themselves biased. Institutional affiliation is a heuristic proxy that should not override methodological evaluation.
HSRT Assessing Credibility of Claims 2
A health blogger claims that 'Vitamin C megadoses cure cancer, and thousands of patients have been healed.' Which credibility criterion is MOST undermined by this statement?