Two-Step Transfer Learning Improves Deep Learning–Based Drug Response Prediction in Small Datasets: A Case Study of Glioblastoma

While deep learning (DL) is used in patients’ outcome predictions, the insufficiency of patient samples limits the accuracy. In this study, we investigated how transfer learning (TL) alleviates the small sample size problem. A 2-step TL framework was constructed for a difficult task: predicting the...

Full description

Saved in:
Bibliographic Details
Main Authors: Jie Ju, Ioannis Ntafoulis, Michelle Klein, Marcel JT Reinders, Martine Lamfers, Andrew P Stubbs, Yunlei Li
Format: Article
Language:English
Published: SAGE Publishing 2025-01-01
Series:Bioinformatics and Biology Insights
Online Access:https://doi.org/10.1177/11779322241301507
Tags: Add Tag
No Tags, Be the first to tag this record!
_version_ 1841560384890208256
author Jie Ju
Ioannis Ntafoulis
Michelle Klein
Marcel JT Reinders
Martine Lamfers
Andrew P Stubbs
Yunlei Li
author_facet Jie Ju
Ioannis Ntafoulis
Michelle Klein
Marcel JT Reinders
Martine Lamfers
Andrew P Stubbs
Yunlei Li
author_sort Jie Ju
collection DOAJ
description While deep learning (DL) is used in patients’ outcome predictions, the insufficiency of patient samples limits the accuracy. In this study, we investigated how transfer learning (TL) alleviates the small sample size problem. A 2-step TL framework was constructed for a difficult task: predicting the response of the drug temozolomide (TMZ) in glioblastoma (GBM) cell cultures. The GBM is aggressive, and most patients do not benefit from the only approved chemotherapeutic agent TMZ. O6-methylguanine-DNA methyltransferase (MGMT) promoter methylation status is the only biomarker for TMZ responsiveness but has shown limited predictive power. The 2-step TL framework was built on 3 datasets: (1) the subset of the Genomics of Drug Sensitivity in Cancer (GDSC) dataset, including miscellaneous cell cultures treated by TMZ, cyclophosphamide, bortezomib, and oxaliplatin, as the source dataset; (2) the Human Glioblastoma Cell Culture (HGCC) dataset, for fine-tuning; and (3) a small target dataset GSE232173, for validation. The latter two included specifically TMZ-treated GBM cell cultures. The DL models were pretrained on the cell cultures treated by each of the 4 drugs from GDSC, respectively. Then, the DL models were refined on HGCC, where the best source drug was identified. Finally, the DL model was validated on GSE232173. Using 2-step TL with pretraining on oxaliplatin was not only superior to those without TL and with 1-step TL but also better than 3 benchmark methods, including MGMT. The oxaliplatin-based TL improved the performance probably by increasing the weights of cell cycle-related genes, which relates to the TMZ response processes. Our findings support the potential of oxaliplatin being an alternative therapy for patients with GBM and TL facilitating drug repurposing research. We recommend that following our methodology, using mixed cancers and a related drug as the source and then fine-tuning the model with the target cancer and the target drug will enhance drug response prediction.
format Article
id doaj-art-09fe2beb7d2241e49ca32af6dcdd3ccc
institution Kabale University
issn 1177-9322
language English
publishDate 2025-01-01
publisher SAGE Publishing
record_format Article
series Bioinformatics and Biology Insights
spelling doaj-art-09fe2beb7d2241e49ca32af6dcdd3ccc2025-01-04T07:03:23ZengSAGE PublishingBioinformatics and Biology Insights1177-93222025-01-011910.1177/11779322241301507Two-Step Transfer Learning Improves Deep Learning–Based Drug Response Prediction in Small Datasets: A Case Study of GlioblastomaJie Ju0Ioannis Ntafoulis1Michelle Klein2Marcel JT Reinders3Martine Lamfers4Andrew P Stubbs5Yunlei Li6Department of Pathology & Clinical Bioinformatics, Erasmus MC Cancer Institute, University Medical Center Rotterdam, Rotterdam, The NetherlandsDepartment of Neurosurgery, Brain Tumor Center, Erasmus MC Cancer Institute, University Medical Center Rotterdam, Rotterdam, The NetherlandsDepartment of Neurosurgery, Brain Tumor Center, Erasmus MC Cancer Institute, University Medical Center Rotterdam, Rotterdam, The NetherlandsThe Delft Bioinformatics Lab, Delft University of Technology, Delft, The NetherlandsDepartment of Neurosurgery, Brain Tumor Center, Erasmus MC Cancer Institute, University Medical Center Rotterdam, Rotterdam, The NetherlandsDepartment of Pathology & Clinical Bioinformatics, Erasmus MC Cancer Institute, University Medical Center Rotterdam, Rotterdam, The NetherlandsDepartment of Pathology & Clinical Bioinformatics, Erasmus MC Cancer Institute, University Medical Center Rotterdam, Rotterdam, The NetherlandsWhile deep learning (DL) is used in patients’ outcome predictions, the insufficiency of patient samples limits the accuracy. In this study, we investigated how transfer learning (TL) alleviates the small sample size problem. A 2-step TL framework was constructed for a difficult task: predicting the response of the drug temozolomide (TMZ) in glioblastoma (GBM) cell cultures. The GBM is aggressive, and most patients do not benefit from the only approved chemotherapeutic agent TMZ. O6-methylguanine-DNA methyltransferase (MGMT) promoter methylation status is the only biomarker for TMZ responsiveness but has shown limited predictive power. The 2-step TL framework was built on 3 datasets: (1) the subset of the Genomics of Drug Sensitivity in Cancer (GDSC) dataset, including miscellaneous cell cultures treated by TMZ, cyclophosphamide, bortezomib, and oxaliplatin, as the source dataset; (2) the Human Glioblastoma Cell Culture (HGCC) dataset, for fine-tuning; and (3) a small target dataset GSE232173, for validation. The latter two included specifically TMZ-treated GBM cell cultures. The DL models were pretrained on the cell cultures treated by each of the 4 drugs from GDSC, respectively. Then, the DL models were refined on HGCC, where the best source drug was identified. Finally, the DL model was validated on GSE232173. Using 2-step TL with pretraining on oxaliplatin was not only superior to those without TL and with 1-step TL but also better than 3 benchmark methods, including MGMT. The oxaliplatin-based TL improved the performance probably by increasing the weights of cell cycle-related genes, which relates to the TMZ response processes. Our findings support the potential of oxaliplatin being an alternative therapy for patients with GBM and TL facilitating drug repurposing research. We recommend that following our methodology, using mixed cancers and a related drug as the source and then fine-tuning the model with the target cancer and the target drug will enhance drug response prediction.https://doi.org/10.1177/11779322241301507
spellingShingle Jie Ju
Ioannis Ntafoulis
Michelle Klein
Marcel JT Reinders
Martine Lamfers
Andrew P Stubbs
Yunlei Li
Two-Step Transfer Learning Improves Deep Learning–Based Drug Response Prediction in Small Datasets: A Case Study of Glioblastoma
Bioinformatics and Biology Insights
title Two-Step Transfer Learning Improves Deep Learning–Based Drug Response Prediction in Small Datasets: A Case Study of Glioblastoma
title_full Two-Step Transfer Learning Improves Deep Learning–Based Drug Response Prediction in Small Datasets: A Case Study of Glioblastoma
title_fullStr Two-Step Transfer Learning Improves Deep Learning–Based Drug Response Prediction in Small Datasets: A Case Study of Glioblastoma
title_full_unstemmed Two-Step Transfer Learning Improves Deep Learning–Based Drug Response Prediction in Small Datasets: A Case Study of Glioblastoma
title_short Two-Step Transfer Learning Improves Deep Learning–Based Drug Response Prediction in Small Datasets: A Case Study of Glioblastoma
title_sort two step transfer learning improves deep learning based drug response prediction in small datasets a case study of glioblastoma
url https://doi.org/10.1177/11779322241301507
work_keys_str_mv AT jieju twosteptransferlearningimprovesdeeplearningbaseddrugresponsepredictioninsmalldatasetsacasestudyofglioblastoma
AT ioannisntafoulis twosteptransferlearningimprovesdeeplearningbaseddrugresponsepredictioninsmalldatasetsacasestudyofglioblastoma
AT michelleklein twosteptransferlearningimprovesdeeplearningbaseddrugresponsepredictioninsmalldatasetsacasestudyofglioblastoma
AT marceljtreinders twosteptransferlearningimprovesdeeplearningbaseddrugresponsepredictioninsmalldatasetsacasestudyofglioblastoma
AT martinelamfers twosteptransferlearningimprovesdeeplearningbaseddrugresponsepredictioninsmalldatasetsacasestudyofglioblastoma
AT andrewpstubbs twosteptransferlearningimprovesdeeplearningbaseddrugresponsepredictioninsmalldatasetsacasestudyofglioblastoma
AT yunleili twosteptransferlearningimprovesdeeplearningbaseddrugresponsepredictioninsmalldatasetsacasestudyofglioblastoma