Integration of non-randomized studies with randomized controlled trials in meta-analyses of clinical studies: a meta-epidemiological study on effect estimation of interventions
Abstract Backgrounds Syntheses of non-randomized studies of interventions (NRSIs) and randomized controlled trials (RCTs) are increasingly used in decision-making. This study aimed to summarize when NRSIs are included in evidence syntheses of RCTs, with a particular focus on the methodological issue...
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2024-12-01
|
| Series: | BMC Medicine |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s12916-024-03778-1 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1846136987814199296 |
|---|---|
| author | Fan Mei Minghong Yao Yuning Wang Jiayidaer Huan Yu Ma Guowei Li Kang Zou Ling Li Xin Sun |
| author_facet | Fan Mei Minghong Yao Yuning Wang Jiayidaer Huan Yu Ma Guowei Li Kang Zou Ling Li Xin Sun |
| author_sort | Fan Mei |
| collection | DOAJ |
| description | Abstract Backgrounds Syntheses of non-randomized studies of interventions (NRSIs) and randomized controlled trials (RCTs) are increasingly used in decision-making. This study aimed to summarize when NRSIs are included in evidence syntheses of RCTs, with a particular focus on the methodological issues associated with combining NRSIs and RCTs. Methods We searched PubMed to identify clinical systematic reviews published between 9 December 2017 and 9 December 2022, randomly sampling reviews in a 1:1 ratio of Core and non-Core clinical journals. We included systematic reviews with RCTs and NRSIs for the same clinical question. Clinical scenarios for considering the inclusion of NRSIs in eligible studies were classified. We extracted the methodological characteristics of the included studies, assessed the concordance of estimates between RCTs and NRSIs, calculated the ratio of the relative effect estimate from NRSIs to that from RCTs, and evaluated the impact on the estimates of pooled estimates when NRSIs are included. Results Two hundred twenty systematic reviews were included in the analysis. The clinical scenarios for including NRSIs were grouped into four main justifications: adverse outcomes (n = 140, 63.6%), long-term outcomes (n = 36, 16.4%), the applicability of RCT results to broader populations (n = 11, 5.0%), and other (n = 33, 15.0%). When conducting a meta-analysis, none of these reviews assessed the compatibility of the different types of evidence prior, 203 (92.3%) combined estimates from RCTs and NRSIs in the same meta-analysis. Of the 203 studies, 169 (76.8%) used crude estimates of NRSIs, and 28 (13.8%) combined RCTs and multiple types of NRSIs. Seventy-seven studies (35.5%) showed “qualitative disagree” between estimates from RCTs and NRSIs, and 101 studies (46.5%) found “important difference”. The integration of NRSIs changed the qualitative direction of estimates from RCTs in 72 out of 200 studies (36.0%). Conclusions Systematic reviews typically include NRSIs in the context of assessing adverse or long-term outcomes. The inclusion of NRSIs in a meta-analysis of RCTs has a substantial impact on effect estimates, but discrepancies between RCTs and NRSIs are often ignored. Our proposed recommendations will help researchers to consider carefully when and how to synthesis evidence from RCTs and NRSIs. |
| format | Article |
| id | doaj-art-618b0080c2e14e5398c2cb236ae076f9 |
| institution | Kabale University |
| issn | 1741-7015 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | BMC |
| record_format | Article |
| series | BMC Medicine |
| spelling | doaj-art-618b0080c2e14e5398c2cb236ae076f92024-12-08T12:33:11ZengBMCBMC Medicine1741-70152024-12-0122111710.1186/s12916-024-03778-1Integration of non-randomized studies with randomized controlled trials in meta-analyses of clinical studies: a meta-epidemiological study on effect estimation of interventionsFan Mei0Minghong Yao1Yuning Wang2Jiayidaer Huan3Yu Ma4Guowei Li5Kang Zou6Ling Li7Xin Sun8Institute of Integrated Traditional Chinese and Western Medicine, Chinese Evidence-Based Medicine Center, Cochrane China and MAGIC China Center, West China Hospital, Sichuan UniversityInstitute of Integrated Traditional Chinese and Western Medicine, Chinese Evidence-Based Medicine Center, Cochrane China and MAGIC China Center, West China Hospital, Sichuan UniversityInstitute of Integrated Traditional Chinese and Western Medicine, Chinese Evidence-Based Medicine Center, Cochrane China and MAGIC China Center, West China Hospital, Sichuan UniversityInstitute of Integrated Traditional Chinese and Western Medicine, Chinese Evidence-Based Medicine Center, Cochrane China and MAGIC China Center, West China Hospital, Sichuan UniversityInstitute of Integrated Traditional Chinese and Western Medicine, Chinese Evidence-Based Medicine Center, Cochrane China and MAGIC China Center, West China Hospital, Sichuan UniversityDepartment of Health Research Methods, Evidence and Impact, McMaster UniversityInstitute of Integrated Traditional Chinese and Western Medicine, Chinese Evidence-Based Medicine Center, Cochrane China and MAGIC China Center, West China Hospital, Sichuan UniversityInstitute of Integrated Traditional Chinese and Western Medicine, Chinese Evidence-Based Medicine Center, Cochrane China and MAGIC China Center, West China Hospital, Sichuan UniversityInstitute of Integrated Traditional Chinese and Western Medicine, Chinese Evidence-Based Medicine Center, Cochrane China and MAGIC China Center, West China Hospital, Sichuan UniversityAbstract Backgrounds Syntheses of non-randomized studies of interventions (NRSIs) and randomized controlled trials (RCTs) are increasingly used in decision-making. This study aimed to summarize when NRSIs are included in evidence syntheses of RCTs, with a particular focus on the methodological issues associated with combining NRSIs and RCTs. Methods We searched PubMed to identify clinical systematic reviews published between 9 December 2017 and 9 December 2022, randomly sampling reviews in a 1:1 ratio of Core and non-Core clinical journals. We included systematic reviews with RCTs and NRSIs for the same clinical question. Clinical scenarios for considering the inclusion of NRSIs in eligible studies were classified. We extracted the methodological characteristics of the included studies, assessed the concordance of estimates between RCTs and NRSIs, calculated the ratio of the relative effect estimate from NRSIs to that from RCTs, and evaluated the impact on the estimates of pooled estimates when NRSIs are included. Results Two hundred twenty systematic reviews were included in the analysis. The clinical scenarios for including NRSIs were grouped into four main justifications: adverse outcomes (n = 140, 63.6%), long-term outcomes (n = 36, 16.4%), the applicability of RCT results to broader populations (n = 11, 5.0%), and other (n = 33, 15.0%). When conducting a meta-analysis, none of these reviews assessed the compatibility of the different types of evidence prior, 203 (92.3%) combined estimates from RCTs and NRSIs in the same meta-analysis. Of the 203 studies, 169 (76.8%) used crude estimates of NRSIs, and 28 (13.8%) combined RCTs and multiple types of NRSIs. Seventy-seven studies (35.5%) showed “qualitative disagree” between estimates from RCTs and NRSIs, and 101 studies (46.5%) found “important difference”. The integration of NRSIs changed the qualitative direction of estimates from RCTs in 72 out of 200 studies (36.0%). Conclusions Systematic reviews typically include NRSIs in the context of assessing adverse or long-term outcomes. The inclusion of NRSIs in a meta-analysis of RCTs has a substantial impact on effect estimates, but discrepancies between RCTs and NRSIs are often ignored. Our proposed recommendations will help researchers to consider carefully when and how to synthesis evidence from RCTs and NRSIs.https://doi.org/10.1186/s12916-024-03778-1Randomized controlled trialsNon-randomized studies of interventionsSystematic reviewMeta-analysisMeta-epidemiology |
| spellingShingle | Fan Mei Minghong Yao Yuning Wang Jiayidaer Huan Yu Ma Guowei Li Kang Zou Ling Li Xin Sun Integration of non-randomized studies with randomized controlled trials in meta-analyses of clinical studies: a meta-epidemiological study on effect estimation of interventions BMC Medicine Randomized controlled trials Non-randomized studies of interventions Systematic review Meta-analysis Meta-epidemiology |
| title | Integration of non-randomized studies with randomized controlled trials in meta-analyses of clinical studies: a meta-epidemiological study on effect estimation of interventions |
| title_full | Integration of non-randomized studies with randomized controlled trials in meta-analyses of clinical studies: a meta-epidemiological study on effect estimation of interventions |
| title_fullStr | Integration of non-randomized studies with randomized controlled trials in meta-analyses of clinical studies: a meta-epidemiological study on effect estimation of interventions |
| title_full_unstemmed | Integration of non-randomized studies with randomized controlled trials in meta-analyses of clinical studies: a meta-epidemiological study on effect estimation of interventions |
| title_short | Integration of non-randomized studies with randomized controlled trials in meta-analyses of clinical studies: a meta-epidemiological study on effect estimation of interventions |
| title_sort | integration of non randomized studies with randomized controlled trials in meta analyses of clinical studies a meta epidemiological study on effect estimation of interventions |
| topic | Randomized controlled trials Non-randomized studies of interventions Systematic review Meta-analysis Meta-epidemiology |
| url | https://doi.org/10.1186/s12916-024-03778-1 |
| work_keys_str_mv | AT fanmei integrationofnonrandomizedstudieswithrandomizedcontrolledtrialsinmetaanalysesofclinicalstudiesametaepidemiologicalstudyoneffectestimationofinterventions AT minghongyao integrationofnonrandomizedstudieswithrandomizedcontrolledtrialsinmetaanalysesofclinicalstudiesametaepidemiologicalstudyoneffectestimationofinterventions AT yuningwang integrationofnonrandomizedstudieswithrandomizedcontrolledtrialsinmetaanalysesofclinicalstudiesametaepidemiologicalstudyoneffectestimationofinterventions AT jiayidaerhuan integrationofnonrandomizedstudieswithrandomizedcontrolledtrialsinmetaanalysesofclinicalstudiesametaepidemiologicalstudyoneffectestimationofinterventions AT yuma integrationofnonrandomizedstudieswithrandomizedcontrolledtrialsinmetaanalysesofclinicalstudiesametaepidemiologicalstudyoneffectestimationofinterventions AT guoweili integrationofnonrandomizedstudieswithrandomizedcontrolledtrialsinmetaanalysesofclinicalstudiesametaepidemiologicalstudyoneffectestimationofinterventions AT kangzou integrationofnonrandomizedstudieswithrandomizedcontrolledtrialsinmetaanalysesofclinicalstudiesametaepidemiologicalstudyoneffectestimationofinterventions AT lingli integrationofnonrandomizedstudieswithrandomizedcontrolledtrialsinmetaanalysesofclinicalstudiesametaepidemiologicalstudyoneffectestimationofinterventions AT xinsun integrationofnonrandomizedstudieswithrandomizedcontrolledtrialsinmetaanalysesofclinicalstudiesametaepidemiologicalstudyoneffectestimationofinterventions |