Strategies to Increase Response Rate and Reduce Nonresponse Bias in Population Health Research: Analysis of a Series of Randomized Controlled Experiments during a Large COVID-19 Study
Abstract BackgroundHigh response rates are needed in population-based studies, as nonresponse reduces effective sample size and bias affects accuracy and decreases the generalizability of the study findings. ObjectiveWe tested different strategies to improve respon...
Saved in:
| Main Authors: | , , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
JMIR Publications
2025-01-01
|
| Series: | JMIR Public Health and Surveillance |
| Online Access: | https://publichealth.jmir.org/2025/1/e60022 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841526661475991552 |
|---|---|
| author | Christina J Atchison Nicholas Gilby Galini Pantelidou Sam Clemens Kevin Pickering Marc Chadeau-Hyam Deborah Ashby Wendy S Barclay Graham S Cooke Ara Darzi Steven Riley Christl A Donnelly Helen Ward Paul Elliott |
| author_facet | Christina J Atchison Nicholas Gilby Galini Pantelidou Sam Clemens Kevin Pickering Marc Chadeau-Hyam Deborah Ashby Wendy S Barclay Graham S Cooke Ara Darzi Steven Riley Christl A Donnelly Helen Ward Paul Elliott |
| author_sort | Christina J Atchison |
| collection | DOAJ |
| description |
Abstract
BackgroundHigh response rates are needed in population-based studies, as nonresponse reduces effective sample size and bias affects accuracy and decreases the generalizability of the study findings.
ObjectiveWe tested different strategies to improve response rate and reduce nonresponse bias in a national population–based COVID-19 surveillance program in England, United Kingdom.
MethodsOver 19 rounds, a random sample of individuals aged 5 years and older from the general population in England were invited by mail to complete a web-based questionnaire and return a swab for SARS-CoV-2 testing. We carried out several nested randomized controlled experiments to measure the impact on response rates of different interventions, including (1) variations in invitation and reminder letters and SMS text messages and (2) the offer of a conditional monetary incentive to return a swab, reporting absolute changes in response and relative response rate (95% CIs).
ResultsMonetary incentives increased the response rate (completed swabs returned as a proportion of the number of individuals invited) across all age groups, sex at birth, and area deprivation with the biggest increase among the lowest responders, namely teenagers and young adults and those living in more deprived areas. With no monetary incentive, the response rate was 3.4% in participants aged 18‐22 years, increasing to 8.1% with a £10 (US $12.5) incentive, 11.9% with £20 (US $25.0), and 18.2% with £30 (US $37.5) (relative response rate 2.4 [95% CI 2.0-2.9], 3.5 [95% CI 3.0-4.2], and 5.4 [95% CI 4.4-6.7], respectively). Nonmonetary strategies had a modest, if any, impact on response rate. The largest effect was observed for sending an additional swab reminder (SMS text message or email). For example, those receiving an additional SMS text message were more likely to return a completed swab compared to those receiving the standard email-SMS approach, 73.3% versus 70.2%: percentage difference 3.1% (95% CI 2.2%-4.0%).
ConclusionsConditional monetary incentives improved response rates to a web-based survey, which required the return of a swab test, particularly for younger age groups. Used in a selective way, incentives may be an effective strategy for improving sample response and representativeness in population-based studies. |
| format | Article |
| id | doaj-art-9650b4da159b4f46b39146e3a1be7c2e |
| institution | Kabale University |
| issn | 2369-2960 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | JMIR Publications |
| record_format | Article |
| series | JMIR Public Health and Surveillance |
| spelling | doaj-art-9650b4da159b4f46b39146e3a1be7c2e2025-01-16T15:17:05ZengJMIR PublicationsJMIR Public Health and Surveillance2369-29602025-01-0111e60022e6002210.2196/60022Strategies to Increase Response Rate and Reduce Nonresponse Bias in Population Health Research: Analysis of a Series of Randomized Controlled Experiments during a Large COVID-19 StudyChristina J Atchisonhttp://orcid.org/0000-0001-8304-7389Nicholas Gilbyhttp://orcid.org/0009-0005-0915-8124Galini Pantelidouhttp://orcid.org/0009-0007-3105-5740Sam Clemenshttp://orcid.org/0000-0001-8204-9083Kevin Pickeringhttp://orcid.org/0000-0002-0509-0039Marc Chadeau-Hyamhttp://orcid.org/0000-0001-8341-5436Deborah Ashbyhttp://orcid.org/0000-0003-3146-7466Wendy S Barclayhttp://orcid.org/0000-0002-3948-0895Graham S Cookehttp://orcid.org/0000-0001-6475-5056Ara Darzihttp://orcid.org/0000-0001-7815-7989Steven Rileyhttp://orcid.org/0000-0001-7904-4804Christl A Donnellyhttp://orcid.org/0000-0002-0195-2463Helen Wardhttp://orcid.org/0000-0001-8238-5036Paul Elliotthttp://orcid.org/0000-0002-7511-5684 Abstract BackgroundHigh response rates are needed in population-based studies, as nonresponse reduces effective sample size and bias affects accuracy and decreases the generalizability of the study findings. ObjectiveWe tested different strategies to improve response rate and reduce nonresponse bias in a national population–based COVID-19 surveillance program in England, United Kingdom. MethodsOver 19 rounds, a random sample of individuals aged 5 years and older from the general population in England were invited by mail to complete a web-based questionnaire and return a swab for SARS-CoV-2 testing. We carried out several nested randomized controlled experiments to measure the impact on response rates of different interventions, including (1) variations in invitation and reminder letters and SMS text messages and (2) the offer of a conditional monetary incentive to return a swab, reporting absolute changes in response and relative response rate (95% CIs). ResultsMonetary incentives increased the response rate (completed swabs returned as a proportion of the number of individuals invited) across all age groups, sex at birth, and area deprivation with the biggest increase among the lowest responders, namely teenagers and young adults and those living in more deprived areas. With no monetary incentive, the response rate was 3.4% in participants aged 18‐22 years, increasing to 8.1% with a £10 (US $12.5) incentive, 11.9% with £20 (US $25.0), and 18.2% with £30 (US $37.5) (relative response rate 2.4 [95% CI 2.0-2.9], 3.5 [95% CI 3.0-4.2], and 5.4 [95% CI 4.4-6.7], respectively). Nonmonetary strategies had a modest, if any, impact on response rate. The largest effect was observed for sending an additional swab reminder (SMS text message or email). For example, those receiving an additional SMS text message were more likely to return a completed swab compared to those receiving the standard email-SMS approach, 73.3% versus 70.2%: percentage difference 3.1% (95% CI 2.2%-4.0%). ConclusionsConditional monetary incentives improved response rates to a web-based survey, which required the return of a swab test, particularly for younger age groups. Used in a selective way, incentives may be an effective strategy for improving sample response and representativeness in population-based studies.https://publichealth.jmir.org/2025/1/e60022 |
| spellingShingle | Christina J Atchison Nicholas Gilby Galini Pantelidou Sam Clemens Kevin Pickering Marc Chadeau-Hyam Deborah Ashby Wendy S Barclay Graham S Cooke Ara Darzi Steven Riley Christl A Donnelly Helen Ward Paul Elliott Strategies to Increase Response Rate and Reduce Nonresponse Bias in Population Health Research: Analysis of a Series of Randomized Controlled Experiments during a Large COVID-19 Study JMIR Public Health and Surveillance |
| title | Strategies to Increase Response Rate and Reduce Nonresponse Bias in Population Health Research: Analysis of a Series of Randomized Controlled Experiments during a Large COVID-19 Study |
| title_full | Strategies to Increase Response Rate and Reduce Nonresponse Bias in Population Health Research: Analysis of a Series of Randomized Controlled Experiments during a Large COVID-19 Study |
| title_fullStr | Strategies to Increase Response Rate and Reduce Nonresponse Bias in Population Health Research: Analysis of a Series of Randomized Controlled Experiments during a Large COVID-19 Study |
| title_full_unstemmed | Strategies to Increase Response Rate and Reduce Nonresponse Bias in Population Health Research: Analysis of a Series of Randomized Controlled Experiments during a Large COVID-19 Study |
| title_short | Strategies to Increase Response Rate and Reduce Nonresponse Bias in Population Health Research: Analysis of a Series of Randomized Controlled Experiments during a Large COVID-19 Study |
| title_sort | strategies to increase response rate and reduce nonresponse bias in population health research analysis of a series of randomized controlled experiments during a large covid 19 study |
| url | https://publichealth.jmir.org/2025/1/e60022 |
| work_keys_str_mv | AT christinajatchison strategiestoincreaseresponserateandreducenonresponsebiasinpopulationhealthresearchanalysisofaseriesofrandomizedcontrolledexperimentsduringalargecovid19study AT nicholasgilby strategiestoincreaseresponserateandreducenonresponsebiasinpopulationhealthresearchanalysisofaseriesofrandomizedcontrolledexperimentsduringalargecovid19study AT galinipantelidou strategiestoincreaseresponserateandreducenonresponsebiasinpopulationhealthresearchanalysisofaseriesofrandomizedcontrolledexperimentsduringalargecovid19study AT samclemens strategiestoincreaseresponserateandreducenonresponsebiasinpopulationhealthresearchanalysisofaseriesofrandomizedcontrolledexperimentsduringalargecovid19study AT kevinpickering strategiestoincreaseresponserateandreducenonresponsebiasinpopulationhealthresearchanalysisofaseriesofrandomizedcontrolledexperimentsduringalargecovid19study AT marcchadeauhyam strategiestoincreaseresponserateandreducenonresponsebiasinpopulationhealthresearchanalysisofaseriesofrandomizedcontrolledexperimentsduringalargecovid19study AT deborahashby strategiestoincreaseresponserateandreducenonresponsebiasinpopulationhealthresearchanalysisofaseriesofrandomizedcontrolledexperimentsduringalargecovid19study AT wendysbarclay strategiestoincreaseresponserateandreducenonresponsebiasinpopulationhealthresearchanalysisofaseriesofrandomizedcontrolledexperimentsduringalargecovid19study AT grahamscooke strategiestoincreaseresponserateandreducenonresponsebiasinpopulationhealthresearchanalysisofaseriesofrandomizedcontrolledexperimentsduringalargecovid19study AT aradarzi strategiestoincreaseresponserateandreducenonresponsebiasinpopulationhealthresearchanalysisofaseriesofrandomizedcontrolledexperimentsduringalargecovid19study AT stevenriley strategiestoincreaseresponserateandreducenonresponsebiasinpopulationhealthresearchanalysisofaseriesofrandomizedcontrolledexperimentsduringalargecovid19study AT christladonnelly strategiestoincreaseresponserateandreducenonresponsebiasinpopulationhealthresearchanalysisofaseriesofrandomizedcontrolledexperimentsduringalargecovid19study AT helenward strategiestoincreaseresponserateandreducenonresponsebiasinpopulationhealthresearchanalysisofaseriesofrandomizedcontrolledexperimentsduringalargecovid19study AT paulelliott strategiestoincreaseresponserateandreducenonresponsebiasinpopulationhealthresearchanalysisofaseriesofrandomizedcontrolledexperimentsduringalargecovid19study |