Making the most out of timeseries symptom data: A machine learning study on symptom predictions of internet-based CBT
Objective: Predicting who will not benefit enough from Internet-Based Cognitive Behavioral (ICBT) Therapy early on can assist in better allocation of limited mental health care resources. Repeated measures of symptoms during treatment is the strongest predictor of outcome, and we want to investigate...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2024-12-01
|
| Series: | Internet Interventions |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2214782924000666 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1846127207277133824 |
|---|---|
| author | Nils Hentati Isacsson Kirsten Zantvoort Erik Forsell Magnus Boman Viktor Kaldo |
| author_facet | Nils Hentati Isacsson Kirsten Zantvoort Erik Forsell Magnus Boman Viktor Kaldo |
| author_sort | Nils Hentati Isacsson |
| collection | DOAJ |
| description | Objective: Predicting who will not benefit enough from Internet-Based Cognitive Behavioral (ICBT) Therapy early on can assist in better allocation of limited mental health care resources. Repeated measures of symptoms during treatment is the strongest predictor of outcome, and we want to investigate if methods that explicitly account for time-dependency are superior to methods that do not, with data from (a) only two pre-treatment timepoints and (b) the pre-treatment timepoints and three timepoints during initial treatment. Methods: We use 1) commonly used time-independent methods (i.e., Linear Regression and Random Forest models) and 2) time-dependent methods (i.e., multilevel model regression, mixed-effects random forest, and a Long Short-Term Memory model) to predict symptoms during treatment, including the final outcome. This is done with symptom scores from 6436 ICBT patients from regular care, using robust multiple imputation and nested cross-validation methods. Results: The models had a 14 %–12 % root mean squared error (RMSE) in predicting the post-treatment outcome, corresponding to a balanced accuracy of 67–74 %. Time-dependent models did not have higher accuracies. Using data for the initial treatment period (b) instead of only from before treatment (a) increased prediction results by 1.3 % percentage points (12 % to 10.7 %) RMSE and 6 % percentage points BACC (69 % to 75 %). Conclusion: Training prediction models on only symptom scores of the first few weeks is a promising avenue for symptom predictions in treatment, regardless of which model is used. Further research is necessary to better understand the interaction between model complexity, dataset length and width, and the prediction tasks at hand. |
| format | Article |
| id | doaj-art-05c52c4188774955afb6f3c0ec40962e |
| institution | Kabale University |
| issn | 2214-7829 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | Elsevier |
| record_format | Article |
| series | Internet Interventions |
| spelling | doaj-art-05c52c4188774955afb6f3c0ec40962e2024-12-12T05:21:53ZengElsevierInternet Interventions2214-78292024-12-0138100773Making the most out of timeseries symptom data: A machine learning study on symptom predictions of internet-based CBTNils Hentati Isacsson0Kirsten Zantvoort1Erik Forsell2Magnus Boman3Viktor Kaldo4Centre for Psychiatry Research, Department of Clinical Neuroscience, Karolinska Institutet, & Stockholm Health Care Services, Region Stockholm, Sweden; Corresponding author.Institute of Information Systems, Leuphana University, Lueneburg, GermanyCentre for Psychiatry Research, Department of Clinical Neuroscience, Karolinska Institutet, & Stockholm Health Care Services, Region Stockholm, SwedenDivision of Psychiatry, University College London, UK; Department of Medicine Solna, Clinical Epidemiology Division, Karolinska Institutet, Stockholm, SwedenCentre for Psychiatry Research, Department of Clinical Neuroscience, Karolinska Institutet, & Stockholm Health Care Services, Region Stockholm, Sweden; Department of Psychology, Faculty of Health and Life Sciences, Linnaeus University, Växjö, SwedenObjective: Predicting who will not benefit enough from Internet-Based Cognitive Behavioral (ICBT) Therapy early on can assist in better allocation of limited mental health care resources. Repeated measures of symptoms during treatment is the strongest predictor of outcome, and we want to investigate if methods that explicitly account for time-dependency are superior to methods that do not, with data from (a) only two pre-treatment timepoints and (b) the pre-treatment timepoints and three timepoints during initial treatment. Methods: We use 1) commonly used time-independent methods (i.e., Linear Regression and Random Forest models) and 2) time-dependent methods (i.e., multilevel model regression, mixed-effects random forest, and a Long Short-Term Memory model) to predict symptoms during treatment, including the final outcome. This is done with symptom scores from 6436 ICBT patients from regular care, using robust multiple imputation and nested cross-validation methods. Results: The models had a 14 %–12 % root mean squared error (RMSE) in predicting the post-treatment outcome, corresponding to a balanced accuracy of 67–74 %. Time-dependent models did not have higher accuracies. Using data for the initial treatment period (b) instead of only from before treatment (a) increased prediction results by 1.3 % percentage points (12 % to 10.7 %) RMSE and 6 % percentage points BACC (69 % to 75 %). Conclusion: Training prediction models on only symptom scores of the first few weeks is a promising avenue for symptom predictions in treatment, regardless of which model is used. Further research is necessary to better understand the interaction between model complexity, dataset length and width, and the prediction tasks at hand.http://www.sciencedirect.com/science/article/pii/S2214782924000666Machine learningPredictionTreatment outcomeAdaptive treatment strategyPrecision psychiatryTimeseries symptom |
| spellingShingle | Nils Hentati Isacsson Kirsten Zantvoort Erik Forsell Magnus Boman Viktor Kaldo Making the most out of timeseries symptom data: A machine learning study on symptom predictions of internet-based CBT Internet Interventions Machine learning Prediction Treatment outcome Adaptive treatment strategy Precision psychiatry Timeseries symptom |
| title | Making the most out of timeseries symptom data: A machine learning study on symptom predictions of internet-based CBT |
| title_full | Making the most out of timeseries symptom data: A machine learning study on symptom predictions of internet-based CBT |
| title_fullStr | Making the most out of timeseries symptom data: A machine learning study on symptom predictions of internet-based CBT |
| title_full_unstemmed | Making the most out of timeseries symptom data: A machine learning study on symptom predictions of internet-based CBT |
| title_short | Making the most out of timeseries symptom data: A machine learning study on symptom predictions of internet-based CBT |
| title_sort | making the most out of timeseries symptom data a machine learning study on symptom predictions of internet based cbt |
| topic | Machine learning Prediction Treatment outcome Adaptive treatment strategy Precision psychiatry Timeseries symptom |
| url | http://www.sciencedirect.com/science/article/pii/S2214782924000666 |
| work_keys_str_mv | AT nilshentatiisacsson makingthemostoutoftimeseriessymptomdataamachinelearningstudyonsymptompredictionsofinternetbasedcbt AT kirstenzantvoort makingthemostoutoftimeseriessymptomdataamachinelearningstudyonsymptompredictionsofinternetbasedcbt AT erikforsell makingthemostoutoftimeseriessymptomdataamachinelearningstudyonsymptompredictionsofinternetbasedcbt AT magnusboman makingthemostoutoftimeseriessymptomdataamachinelearningstudyonsymptompredictionsofinternetbasedcbt AT viktorkaldo makingthemostoutoftimeseriessymptomdataamachinelearningstudyonsymptompredictionsofinternetbasedcbt |