← Every exhibit

Table Table 1 From the research bench

Survey bias pathway map

Trace the threat to the decision before choosing a sample-size or wording fix.

PathThreatened objectPossible distortionEvidence checkDecision remains unresolved when
FramePopulation and eligibilityRelevant units are absent or the wrong units are includedCompare frame, reachable population, and decision populationThe inclusion boundary is not recorded
ResponseWho is visibleResponders differ from nonresponders on relevant experience or outcomeRecord reachability, timing, response, partial completion, and known differencesNonresponse direction is assumed
WordingInterpretation of the itemFraming, recall, or social desirability changes the answerPilot alternatives and inspect item meaning in the decision contextThe construct is not defined
MeasurementConstruct and scaleA reliable response measures the wrong object or periodName construct, scale, reference period, and validity checkThe measure cannot be tied to the decision
InterpretationFrom answer to conclusionA pattern is treated as population, causal, or predictive evidenceSeparate description, association, prediction, and causationThe inferential step is hidden
DecisionAction and thresholdEvidence is collected without a rule for what changesName action, threshold, comparison, and follow-up outcomeNo action or outcome is specified

Swipe or scroll horizontally if the table is wider than your screen.

Cite Embed

Reference & Evidence

Source: Author's framework grounded in Guest, Bunce, and Johnson (2006), Hennink, Kaiser, and Marconi (2017), Hagaman and Wutich (2017), Malterud, Siersma, and Guassora (2016), and Schoonenboom and Johnson (2017). This is an evidence review, not a coded survey dataset.

Each line is a claim from the register this journal publishes against, resolved from the register at build time.

  • B The number is conditional on that study's design: "they found that saturation occurred within the first twelve interviews, although basic elements for metathemes were present as early as six interviews" Guest, Bunce & Johnson. (2006) · GUEST06-C1
  • B The 12-interview result is conditional and not a universal rule, and the authors bound it themselves: saturation occurred within the first twelve interviews, "at least based on the codebook we devel"oped, in a study of relatively homogeneous respondents Guest, Bunce & Johnson. (2006) · GUEST06-C2
  • B Two different closure questions, named: "Our study compared two approaches to assessing saturation: code saturation and meaning saturation", and "we found that code saturation was reached at nine interviews" while meaning saturation took longer Hennink, Kaiser & Marconi. (2017) · HENN17-C1
  • B Code saturation and meaning saturation answer different questions, and the paper reports different sample sizes for each: more interviews "were needed to reach meaning saturation where we developed a richly textured understanding of issues", and the authors present the finding as conditions rather than a number, since "We used our results to develop parameters that influence saturation" Hennink, Kaiser & Marconi. (2017) · HENN17-C2
  • B In a four-site cross-cultural study, larger samples were needed for metathemes: the purpose was "to reach data saturation for metathemes in multisited and cross-cultural research", the analysis "Analysis of the data yielded 240 site-specific themes and nine cross-cultural metathemes", and the authors report larger sample sizes "ranging from 20 to 40 interviews" Hagaman & Wutich. (2017) · HAG17-C1
  • B Homogeneity is what moves the number: "We found that 16 or fewer interviews were enough to identify common themes from sites with relatively homogeneous groups" Hagaman & Wutich. (2017) · HAG17-C2
  • B Information power is defined by five items, verbatim: "information power depends on (a) the aim of the study, (b) sample specificity, (c) use of established theory, (d) quality" of dialogue, and (e) analysis strategy Malterud, Siersma & Guassora. (2016) · MAL16-C1
  • B Higher information power can support a smaller sample, but the concept is an appraisal and not a formula: "Information power indicates that the more information the sample holds, relevant" for the study, the lower the number of participants needed Malterud, Siersma & Guassora. (2016) · MAL16-C2
  • B The design dimensions are the authors' own list: "We explain the seven major design dimensions: purpose, theoretical drive, timing (simultaneity and dependency), point of integration, typological versus interactive design approaches, planned versus emergent design, and design complexity" Schoonenboom & Johnson. (2017) · SCHOON17-C1
  • B A supplemental strand still owes its own quality criteria, and the authors argue exactly that: "the idea of decreased rigor is in conflict with one key theme of the present article, namely that mixed methods designs should always meet the criterion of multiple validities legitimation" Schoonenboom & Johnson. (2017) · SCHOON17-C2
  • B Survey bias is actionable only when the threatened decision, population boundary, measurement object, and interpretation step are named together Author framework grounded in the methods sources above · B10-C1
  • B A sample-size rule cannot repair a frame, nonresponse, wording, construct, or interpretation problem that was never specified Author framework grounded in Guest, Hennink, Hagaman, Malterud, and Schoonenboom · B10-C2

Grades: A, verified against the printed page of the primary source · B, primary source, text layer only · C, authoritative secondary · D, reported.

Related exhibits