Ecologically sustainable benchmarking of AI models for histopathology
Abstract Deep learning (DL) holds great promise to improve medical diagnostics, including pathology. Current DL research mainly focuses on performance. DL implementation potentially leads to environmental consequences but approaches for assessment of both performance and carbon footprint are missing...
Saved in:
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2024-12-01
|
| Series: | npj Digital Medicine |
| Online Access: | https://doi.org/10.1038/s41746-024-01397-x |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1846100940242812928 |
|---|---|
| author | Yu-Chia Lan Martin Strauch Pourya Pilva Nikolas E. J. Schmitz Alireza Vafaei Sadr Leon Niggemeier Huong Quynh Nguyen David L. Hölscher Tri Q. Nguyen Jesper Kers Roman D. Bülow Peter Boor |
| author_facet | Yu-Chia Lan Martin Strauch Pourya Pilva Nikolas E. J. Schmitz Alireza Vafaei Sadr Leon Niggemeier Huong Quynh Nguyen David L. Hölscher Tri Q. Nguyen Jesper Kers Roman D. Bülow Peter Boor |
| author_sort | Yu-Chia Lan |
| collection | DOAJ |
| description | Abstract Deep learning (DL) holds great promise to improve medical diagnostics, including pathology. Current DL research mainly focuses on performance. DL implementation potentially leads to environmental consequences but approaches for assessment of both performance and carbon footprint are missing. Here, we explored an approach for developing DL for pathology, which considers both diagnostic performance and carbon footprint, calculated as CO2 or equivalent emissions (CO2eq). We evaluated various DL architectures used in computational pathology, including a large foundation model, across two diagnostic tasks of low and high complexity. We proposed a metric termed ‘environmentally sustainable performance’ (ESPer), which quantitatively integrates performance and operational CO2eq during training and inference. While some DL models showed comparable diagnostic performance, ESPer enabled prioritizing those with less carbon footprint. We also investigated how data reduction approaches can improve the ESPer of individual models. This study provides an approach facilitating the development of environmentally friendly, sustainable medical AI. |
| format | Article |
| id | doaj-art-f08a737b92404f7d9aca3eff1b3b96f7 |
| institution | Kabale University |
| issn | 2398-6352 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | npj Digital Medicine |
| spelling | doaj-art-f08a737b92404f7d9aca3eff1b3b96f72024-12-29T12:48:27ZengNature Portfolionpj Digital Medicine2398-63522024-12-017111110.1038/s41746-024-01397-xEcologically sustainable benchmarking of AI models for histopathologyYu-Chia Lan0Martin Strauch1Pourya Pilva2Nikolas E. J. Schmitz3Alireza Vafaei Sadr4Leon Niggemeier5Huong Quynh Nguyen6David L. Hölscher7Tri Q. Nguyen8Jesper Kers9Roman D. Bülow10Peter Boor11Institute of Pathology, University Clinic Aachen, RWTH Aachen UniversityInstitute of Pathology, University Clinic Aachen, RWTH Aachen UniversityInstitute of Pathology, University Clinic Aachen, RWTH Aachen UniversityInstitute of Pathology, University Clinic Aachen, RWTH Aachen UniversityInstitute of Pathology, University Clinic Aachen, RWTH Aachen UniversityInstitute of Pathology, University Clinic Aachen, RWTH Aachen UniversityInstitute of Pathology, University Clinic Aachen, RWTH Aachen UniversityInstitute of Pathology, University Clinic Aachen, RWTH Aachen UniversityDepartment of Pathology, University Medical Centre UtrechtDepartment of Pathology, Amsterdam UMC, University of AmsterdamInstitute of Pathology, University Clinic Aachen, RWTH Aachen UniversityInstitute of Pathology, University Clinic Aachen, RWTH Aachen UniversityAbstract Deep learning (DL) holds great promise to improve medical diagnostics, including pathology. Current DL research mainly focuses on performance. DL implementation potentially leads to environmental consequences but approaches for assessment of both performance and carbon footprint are missing. Here, we explored an approach for developing DL for pathology, which considers both diagnostic performance and carbon footprint, calculated as CO2 or equivalent emissions (CO2eq). We evaluated various DL architectures used in computational pathology, including a large foundation model, across two diagnostic tasks of low and high complexity. We proposed a metric termed ‘environmentally sustainable performance’ (ESPer), which quantitatively integrates performance and operational CO2eq during training and inference. While some DL models showed comparable diagnostic performance, ESPer enabled prioritizing those with less carbon footprint. We also investigated how data reduction approaches can improve the ESPer of individual models. This study provides an approach facilitating the development of environmentally friendly, sustainable medical AI.https://doi.org/10.1038/s41746-024-01397-x |
| spellingShingle | Yu-Chia Lan Martin Strauch Pourya Pilva Nikolas E. J. Schmitz Alireza Vafaei Sadr Leon Niggemeier Huong Quynh Nguyen David L. Hölscher Tri Q. Nguyen Jesper Kers Roman D. Bülow Peter Boor Ecologically sustainable benchmarking of AI models for histopathology npj Digital Medicine |
| title | Ecologically sustainable benchmarking of AI models for histopathology |
| title_full | Ecologically sustainable benchmarking of AI models for histopathology |
| title_fullStr | Ecologically sustainable benchmarking of AI models for histopathology |
| title_full_unstemmed | Ecologically sustainable benchmarking of AI models for histopathology |
| title_short | Ecologically sustainable benchmarking of AI models for histopathology |
| title_sort | ecologically sustainable benchmarking of ai models for histopathology |
| url | https://doi.org/10.1038/s41746-024-01397-x |
| work_keys_str_mv | AT yuchialan ecologicallysustainablebenchmarkingofaimodelsforhistopathology AT martinstrauch ecologicallysustainablebenchmarkingofaimodelsforhistopathology AT pouryapilva ecologicallysustainablebenchmarkingofaimodelsforhistopathology AT nikolasejschmitz ecologicallysustainablebenchmarkingofaimodelsforhistopathology AT alirezavafaeisadr ecologicallysustainablebenchmarkingofaimodelsforhistopathology AT leonniggemeier ecologicallysustainablebenchmarkingofaimodelsforhistopathology AT huongquynhnguyen ecologicallysustainablebenchmarkingofaimodelsforhistopathology AT davidlholscher ecologicallysustainablebenchmarkingofaimodelsforhistopathology AT triqnguyen ecologicallysustainablebenchmarkingofaimodelsforhistopathology AT jesperkers ecologicallysustainablebenchmarkingofaimodelsforhistopathology AT romandbulow ecologicallysustainablebenchmarkingofaimodelsforhistopathology AT peterboor ecologicallysustainablebenchmarkingofaimodelsforhistopathology |