Protocol for a systematic review on the methodological and reporting quality of prediction model studies using machine learning techniques
Introduction Studies addressing the development and/or validation of diagnostic and prognostic prediction models are abundant in most clinical domains. Systematic reviews have shown that the methodological and reporting quality of prediction model studies is suboptimal. Due to the increasing availab...
Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMJ Publishing Group
2020-11-01
|
| Series: | BMJ Open |
| Online Access: | https://bmjopen.bmj.com/content/10/11/e038832.full |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1846155188951318528 |
|---|---|
| author | Gary S Collins Richard D Riley Johanna A A G Damen Lotty Hooft Ram Bajpai Jie Ma Constanza L Andaur Navarro Toshihiko Takada Steven W J Nijman Paula Dhiman Karel GM Moons |
| author_facet | Gary S Collins Richard D Riley Johanna A A G Damen Lotty Hooft Ram Bajpai Jie Ma Constanza L Andaur Navarro Toshihiko Takada Steven W J Nijman Paula Dhiman Karel GM Moons |
| author_sort | Gary S Collins |
| collection | DOAJ |
| description | Introduction Studies addressing the development and/or validation of diagnostic and prognostic prediction models are abundant in most clinical domains. Systematic reviews have shown that the methodological and reporting quality of prediction model studies is suboptimal. Due to the increasing availability of larger, routinely collected and complex medical data, and the rising application of Artificial Intelligence (AI) or machine learning (ML) techniques, the number of prediction model studies is expected to increase even further. Prediction models developed using AI or ML techniques are often labelled as a ‘black box’ and little is known about their methodological and reporting quality. Therefore, this comprehensive systematic review aims to evaluate the reporting quality, the methodological conduct, and the risk of bias of prediction model studies that applied ML techniques for model development and/or validation.Methods and analysis A search will be performed in PubMed to identify studies developing and/or validating prediction models using any ML methodology and across all medical fields. Studies will be included if they were published between January 2018 and December 2019, predict patient-related outcomes, use any study design or data source, and available in English. Screening of search results and data extraction from included articles will be performed by two independent reviewers. The primary outcomes of this systematic review are: (1) the adherence of ML-based prediction model studies to the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD), and (2) the risk of bias in such studies as assessed using the Prediction model Risk Of Bias ASsessment Tool (PROBAST). A narrative synthesis will be conducted for all included studies. Findings will be stratified by study type, medical field and prevalent ML methods, and will inform necessary extensions or updates of TRIPOD and PROBAST to better address prediction model studies that used AI or ML techniques.Ethics and dissemination Ethical approval is not required for this study because only available published data will be analysed. Findings will be disseminated through peer-reviewed publications and scientific conferences.Systematic review registration PROSPERO, CRD42019161764. |
| format | Article |
| id | doaj-art-3a9beb531820437fbb035f610229206c |
| institution | Kabale University |
| issn | 2044-6055 |
| language | English |
| publishDate | 2020-11-01 |
| publisher | BMJ Publishing Group |
| record_format | Article |
| series | BMJ Open |
| spelling | doaj-art-3a9beb531820437fbb035f610229206c2024-11-26T13:50:13ZengBMJ Publishing GroupBMJ Open2044-60552020-11-01101110.1136/bmjopen-2020-038832Protocol for a systematic review on the methodological and reporting quality of prediction model studies using machine learning techniquesGary S Collins0Richard D Riley1Johanna A A G Damen2Lotty Hooft3Ram Bajpai4Jie Ma5Constanza L Andaur Navarro6Toshihiko Takada7Steven W J Nijman8Paula Dhiman9Karel GM Moons10professorInstitute of Applied Health Research, University of Birmingham, Birmingham, UKPhD fellowJulius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands2 School of Medicine, Keele University, Keele, Staffordshire, UKDepartment of Hematology, The First Affiliated Hospital of Zhengzhou University, Zhengzhou, Henan, Chinadoctoral studentDepartment of General Medicine, Shirakawa Satellite for Teaching And Research (STAR), Fukushima Medical University, Fukushima, Japandoctoral studentNuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford Centre for Statistics in Medicine, Oxford, UKJulius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The NetherlandsIntroduction Studies addressing the development and/or validation of diagnostic and prognostic prediction models are abundant in most clinical domains. Systematic reviews have shown that the methodological and reporting quality of prediction model studies is suboptimal. Due to the increasing availability of larger, routinely collected and complex medical data, and the rising application of Artificial Intelligence (AI) or machine learning (ML) techniques, the number of prediction model studies is expected to increase even further. Prediction models developed using AI or ML techniques are often labelled as a ‘black box’ and little is known about their methodological and reporting quality. Therefore, this comprehensive systematic review aims to evaluate the reporting quality, the methodological conduct, and the risk of bias of prediction model studies that applied ML techniques for model development and/or validation.Methods and analysis A search will be performed in PubMed to identify studies developing and/or validating prediction models using any ML methodology and across all medical fields. Studies will be included if they were published between January 2018 and December 2019, predict patient-related outcomes, use any study design or data source, and available in English. Screening of search results and data extraction from included articles will be performed by two independent reviewers. The primary outcomes of this systematic review are: (1) the adherence of ML-based prediction model studies to the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD), and (2) the risk of bias in such studies as assessed using the Prediction model Risk Of Bias ASsessment Tool (PROBAST). A narrative synthesis will be conducted for all included studies. Findings will be stratified by study type, medical field and prevalent ML methods, and will inform necessary extensions or updates of TRIPOD and PROBAST to better address prediction model studies that used AI or ML techniques.Ethics and dissemination Ethical approval is not required for this study because only available published data will be analysed. Findings will be disseminated through peer-reviewed publications and scientific conferences.Systematic review registration PROSPERO, CRD42019161764.https://bmjopen.bmj.com/content/10/11/e038832.full |
| spellingShingle | Gary S Collins Richard D Riley Johanna A A G Damen Lotty Hooft Ram Bajpai Jie Ma Constanza L Andaur Navarro Toshihiko Takada Steven W J Nijman Paula Dhiman Karel GM Moons Protocol for a systematic review on the methodological and reporting quality of prediction model studies using machine learning techniques BMJ Open |
| title | Protocol for a systematic review on the methodological and reporting quality of prediction model studies using machine learning techniques |
| title_full | Protocol for a systematic review on the methodological and reporting quality of prediction model studies using machine learning techniques |
| title_fullStr | Protocol for a systematic review on the methodological and reporting quality of prediction model studies using machine learning techniques |
| title_full_unstemmed | Protocol for a systematic review on the methodological and reporting quality of prediction model studies using machine learning techniques |
| title_short | Protocol for a systematic review on the methodological and reporting quality of prediction model studies using machine learning techniques |
| title_sort | protocol for a systematic review on the methodological and reporting quality of prediction model studies using machine learning techniques |
| url | https://bmjopen.bmj.com/content/10/11/e038832.full |
| work_keys_str_mv | AT garyscollins protocolforasystematicreviewonthemethodologicalandreportingqualityofpredictionmodelstudiesusingmachinelearningtechniques AT richarddriley protocolforasystematicreviewonthemethodologicalandreportingqualityofpredictionmodelstudiesusingmachinelearningtechniques AT johannaaagdamen protocolforasystematicreviewonthemethodologicalandreportingqualityofpredictionmodelstudiesusingmachinelearningtechniques AT lottyhooft protocolforasystematicreviewonthemethodologicalandreportingqualityofpredictionmodelstudiesusingmachinelearningtechniques AT rambajpai protocolforasystematicreviewonthemethodologicalandreportingqualityofpredictionmodelstudiesusingmachinelearningtechniques AT jiema protocolforasystematicreviewonthemethodologicalandreportingqualityofpredictionmodelstudiesusingmachinelearningtechniques AT constanzalandaurnavarro protocolforasystematicreviewonthemethodologicalandreportingqualityofpredictionmodelstudiesusingmachinelearningtechniques AT toshihikotakada protocolforasystematicreviewonthemethodologicalandreportingqualityofpredictionmodelstudiesusingmachinelearningtechniques AT stevenwjnijman protocolforasystematicreviewonthemethodologicalandreportingqualityofpredictionmodelstudiesusingmachinelearningtechniques AT pauladhiman protocolforasystematicreviewonthemethodologicalandreportingqualityofpredictionmodelstudiesusingmachinelearningtechniques AT karelgmmoons protocolforasystematicreviewonthemethodologicalandreportingqualityofpredictionmodelstudiesusingmachinelearningtechniques |