External validation of SpineNetV2 on a comprehensive set of radiological features for grading lumbosacral disc pathologies
ABSTRACT: Background: In recent years, the integration of Artificial Intelligence (AI) models has revolutionized the diagnosis of Low Back Pain (LBP) and associated disc pathologies. Among these, SpineNetV2 stands out as a state-of-the-art, open-access model for detecting and grading various interv...
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2024-12-01
|
| Series: | North American Spine Society Journal |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2666548424002579 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1846117672763260928 |
|---|---|
| author | Alemu Sisay Nigru, MSc Sergio Benini, PhD Matteo Bonetti, MD Graziella Bragaglio, MSc Michele Frigerio, MD Federico Maffezzoni, MSc Riccardo Leonardi, PhD |
| author_facet | Alemu Sisay Nigru, MSc Sergio Benini, PhD Matteo Bonetti, MD Graziella Bragaglio, MSc Michele Frigerio, MD Federico Maffezzoni, MSc Riccardo Leonardi, PhD |
| author_sort | Alemu Sisay Nigru, MSc |
| collection | DOAJ |
| description | ABSTRACT: Background: In recent years, the integration of Artificial Intelligence (AI) models has revolutionized the diagnosis of Low Back Pain (LBP) and associated disc pathologies. Among these, SpineNetV2 stands out as a state-of-the-art, open-access model for detecting and grading various intervertebral disc pathologies. However, ensuring the reliability and applicability of AI models like SpineNetV2 is paramount. Rigorous validation is essential to guarantee their robustness and generalizability across diverse patient cohorts and imaging protocols. Methods: We conducted a retrospective analysis of MRI images of 1747 lumbosacral intervertebral discs (IVDs) from 353 patients (mean age, 54 ± 15.4 years, 44.5% female) with various spinal disorders, collected between September 2021 and February 2023 at X-Ray Service s.r.l. The SpineNetV2 system was used to grade 11 distinct lumbosacral disc pathologies, including Pfirrmann grading, disc narrowing, central canal stenosis, spondylolisthesis, (upper and lower) endplate defects, (upper and lower) marrow changes, (right and left) foraminal stenosis, and disc herniation, using T2-weighted sagittal MR images. Performance metrics included accuracy, balanced accuracy, precision, F1 score, Matthew's Correlation Coefficient, Brier Score Loss, Lin's concordance correlation coefficients, and Cohen's kappa coefficients. Two expert radiologists provide annotations for these discs. The evaluation of SpineNetV2′s grading is compared against expert radiologists' assessments. Results: SpineNetV2 demonstrated strong performance across various metrics, with high agreement scores (Cohen's Kappa, Lin's Concordance, and Matthew's Correlation Coefficient exceeding 0.7) for most pathologies. However, lower agreement was found for foraminal stenosis and disc herniation, underscoring the limitations of sagittal MR images for evaluating these conditions. Conclusions: This study highlights the importance of external validation, emphasizing the need for comprehensive assessments of deep learning models. SpineNetV2 exhibits promising results in predicting disc pathologies, with findings guiding further improvements. The open-source release of SpineNetV2 enables researchers to independently validate and extend the model's capabilities. This collaborative approach promotes innovation and accelerates the development of more reliable and comprehensive deep learning tools for the assessment of spine pathology. |
| format | Article |
| id | doaj-art-e1ce5924f92948528879f67b239eaaa5 |
| institution | Kabale University |
| issn | 2666-5484 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | Elsevier |
| record_format | Article |
| series | North American Spine Society Journal |
| spelling | doaj-art-e1ce5924f92948528879f67b239eaaa52024-12-18T08:53:13ZengElsevierNorth American Spine Society Journal2666-54842024-12-0120100564External validation of SpineNetV2 on a comprehensive set of radiological features for grading lumbosacral disc pathologiesAlemu Sisay Nigru, MSc0Sergio Benini, PhD1Matteo Bonetti, MD2Graziella Bragaglio, MSc3Michele Frigerio, MD4Federico Maffezzoni, MSc5Riccardo Leonardi, PhD6Department of Information Engineering, University of Brescia, via Branze 38, Brescia 25123, Italy; Department of Clinical and Experimental Sciences, University of Brescia, Viale Europa, 11, Brescia 25123, Italy; Corresponding author at: U.O.C. Inclusione, Partecipazione e Residenze Universitarie, Via Valotti 3/B - 25133 Brescia, Italy.Department of Information Engineering, University of Brescia, via Branze 38, Brescia 25123, ItalyX-Ray Service s.r.l., Via Guglielmo Oberdan 126, Brescia, 25128, Italy; Poliambulatorio Oberdan, Via Guglielmo Oberdan 126, Brescia, 25128, ItalyX-Ray Service s.r.l., Via Guglielmo Oberdan 126, Brescia, 25128, Italy; Poliambulatorio Oberdan, Via Guglielmo Oberdan 126, Brescia, 25128, ItalyPoliambulatorio Oberdan, Via Guglielmo Oberdan 126, Brescia, 25128, ItalyX-Ray Service s.r.l., Via Guglielmo Oberdan 126, Brescia, 25128, Italy; Poliambulatorio Oberdan, Via Guglielmo Oberdan 126, Brescia, 25128, ItalyDepartment of Information Engineering, University of Brescia, via Branze 38, Brescia 25123, ItalyABSTRACT: Background: In recent years, the integration of Artificial Intelligence (AI) models has revolutionized the diagnosis of Low Back Pain (LBP) and associated disc pathologies. Among these, SpineNetV2 stands out as a state-of-the-art, open-access model for detecting and grading various intervertebral disc pathologies. However, ensuring the reliability and applicability of AI models like SpineNetV2 is paramount. Rigorous validation is essential to guarantee their robustness and generalizability across diverse patient cohorts and imaging protocols. Methods: We conducted a retrospective analysis of MRI images of 1747 lumbosacral intervertebral discs (IVDs) from 353 patients (mean age, 54 ± 15.4 years, 44.5% female) with various spinal disorders, collected between September 2021 and February 2023 at X-Ray Service s.r.l. The SpineNetV2 system was used to grade 11 distinct lumbosacral disc pathologies, including Pfirrmann grading, disc narrowing, central canal stenosis, spondylolisthesis, (upper and lower) endplate defects, (upper and lower) marrow changes, (right and left) foraminal stenosis, and disc herniation, using T2-weighted sagittal MR images. Performance metrics included accuracy, balanced accuracy, precision, F1 score, Matthew's Correlation Coefficient, Brier Score Loss, Lin's concordance correlation coefficients, and Cohen's kappa coefficients. Two expert radiologists provide annotations for these discs. The evaluation of SpineNetV2′s grading is compared against expert radiologists' assessments. Results: SpineNetV2 demonstrated strong performance across various metrics, with high agreement scores (Cohen's Kappa, Lin's Concordance, and Matthew's Correlation Coefficient exceeding 0.7) for most pathologies. However, lower agreement was found for foraminal stenosis and disc herniation, underscoring the limitations of sagittal MR images for evaluating these conditions. Conclusions: This study highlights the importance of external validation, emphasizing the need for comprehensive assessments of deep learning models. SpineNetV2 exhibits promising results in predicting disc pathologies, with findings guiding further improvements. The open-source release of SpineNetV2 enables researchers to independently validate and extend the model's capabilities. This collaborative approach promotes innovation and accelerates the development of more reliable and comprehensive deep learning tools for the assessment of spine pathology.http://www.sciencedirect.com/science/article/pii/S2666548424002579AI for MedicineDisc degenerationHerniationLow Back PainSpine MRISpineNetV2 |
| spellingShingle | Alemu Sisay Nigru, MSc Sergio Benini, PhD Matteo Bonetti, MD Graziella Bragaglio, MSc Michele Frigerio, MD Federico Maffezzoni, MSc Riccardo Leonardi, PhD External validation of SpineNetV2 on a comprehensive set of radiological features for grading lumbosacral disc pathologies North American Spine Society Journal AI for Medicine Disc degeneration Herniation Low Back Pain Spine MRI SpineNetV2 |
| title | External validation of SpineNetV2 on a comprehensive set of radiological features for grading lumbosacral disc pathologies |
| title_full | External validation of SpineNetV2 on a comprehensive set of radiological features for grading lumbosacral disc pathologies |
| title_fullStr | External validation of SpineNetV2 on a comprehensive set of radiological features for grading lumbosacral disc pathologies |
| title_full_unstemmed | External validation of SpineNetV2 on a comprehensive set of radiological features for grading lumbosacral disc pathologies |
| title_short | External validation of SpineNetV2 on a comprehensive set of radiological features for grading lumbosacral disc pathologies |
| title_sort | external validation of spinenetv2 on a comprehensive set of radiological features for grading lumbosacral disc pathologies |
| topic | AI for Medicine Disc degeneration Herniation Low Back Pain Spine MRI SpineNetV2 |
| url | http://www.sciencedirect.com/science/article/pii/S2666548424002579 |
| work_keys_str_mv | AT alemusisaynigrumsc externalvalidationofspinenetv2onacomprehensivesetofradiologicalfeaturesforgradinglumbosacraldiscpathologies AT sergiobeniniphd externalvalidationofspinenetv2onacomprehensivesetofradiologicalfeaturesforgradinglumbosacraldiscpathologies AT matteobonettimd externalvalidationofspinenetv2onacomprehensivesetofradiologicalfeaturesforgradinglumbosacraldiscpathologies AT graziellabragagliomsc externalvalidationofspinenetv2onacomprehensivesetofradiologicalfeaturesforgradinglumbosacraldiscpathologies AT michelefrigeriomd externalvalidationofspinenetv2onacomprehensivesetofradiologicalfeaturesforgradinglumbosacraldiscpathologies AT federicomaffezzonimsc externalvalidationofspinenetv2onacomprehensivesetofradiologicalfeaturesforgradinglumbosacraldiscpathologies AT riccardoleonardiphd externalvalidationofspinenetv2onacomprehensivesetofradiologicalfeaturesforgradinglumbosacraldiscpathologies |