CFP Research & Evidence-Based Practice 5 — Questions and Answers
Question 1: A CFP practitioner reads two contradictory studies on sequence-of-returns risk. One is a practitioner journal article; the other is a peer-reviewed academic study. Which should generally be weighted more heavily?
- The practitioner journal article because it is more practically relevant
- The peer-reviewed academic study because it underwent independent scientific review (Correct answer)
- Neither, since contradictory findings cancel each other out
- Whichever study has the larger sample size, regardless of peer review
Correct answer: The peer-reviewed academic study because it underwent independent scientific review
Peer-reviewed academic studies undergo independent expert evaluation of methods and findings, making them a higher-quality evidence source than practitioner journals that may not have the same review rigor.
Question 2: A planner notices that whenever stock markets decline, client calls about switching to cash increase. The planner avoids recommending cash based on this pattern. This informal observation is best described as:
- A valid quasi-experiment
- Anecdotal evidence with limited generalizability (Correct answer)
- A high-quality observational study
- Evidence at the top of the research hierarchy
Correct answer: Anecdotal evidence with limited generalizability
Informal personal observations are anecdotal evidence, which sits at the bottom of the evidence hierarchy and cannot be generalized reliably.
Question 3: Which of the following best defines 'effect size' in financial planning research?
- The probability that a result is not due to chance
- The practical magnitude of a difference or relationship between variables (Correct answer)
- The number of participants needed to detect a true effect
- The confidence level used to construct an interval estimate
Correct answer: The practical magnitude of a difference or relationship between variables
Effect size quantifies the practical magnitude of a relationship or difference, independent of sample size, helping determine whether a statistically significant result is also practically meaningful.
Question 4: A financial planner implements a behavioral coaching intervention and measures clients' savings rates before and after. No control group is used. The biggest threat to concluding the intervention caused improved savings is:
- Attrition bias
- History and maturation threats to internal validity (Correct answer)
- Regression to the mean only
- Lack of statistical power
Correct answer: History and maturation threats to internal validity
Without a control group, external events (history) or natural changes over time (maturation) could explain the improvement, threatening internal validity.
Question 5: A researcher conducting a survey on estate planning intentions sends questionnaires to 1,000 clients; only 120 respond. The primary concern is:
- Type II error from insufficient statistical power
- Non-response bias, as non-responders may differ systematically from responders (Correct answer)
- Instrumentation error from the questionnaire format
- Demand characteristics from in-person contact
Correct answer: Non-response bias, as non-responders may differ systematically from responders
A low response rate creates non-response bias risk because those who did not respond may have systematically different attitudes or behaviors from those who did.
Question 6: Which of the following describes the appropriate use of 'practice-based evidence' in financial planning?
- Relying solely on peer-reviewed studies and ignoring client feedback
- Systematically gathering and analyzing client outcome data from one's own practice to inform decisions (Correct answer)
- Using intuition developed over years without documenting outcomes
- Only applying interventions tested in randomized controlled trials
Correct answer: Systematically gathering and analyzing client outcome data from one's own practice to inform decisions
Practice-based evidence involves systematically collecting and analyzing outcomes from one's own clients to complement and inform evidence-based practice.
Question 7: A CFP practitioner finds a study showing that fee-only advisors generate better client outcomes than commission-based advisors. Before applying this finding, the planner should consider which of the following?
- Whether the study was funded by a fee-only advisor association, which could indicate potential conflict of interest (Correct answer)
- That all industry-funded research is invalid and should be discarded
- That regulatory bodies validate all published financial planning research
- Whether the Journal Impact Factor is above 3.0
Correct answer: Whether the study was funded by a fee-only advisor association, which could indicate potential conflict of interest
Funding source is a key component of critical appraisal because studies funded by parties with a financial interest in the outcome may have conflicts of interest that bias findings.
CFP Research & Evidence-Based Practice 5
A CFP practitioner reads two contradictory studies on sequence-of-returns risk.
One is a practitioner journal article; the other is a peer-reviewed academic study.
Which should generally be weighted more heavily?