The Virtual Client Experience Survey for Mental Health and Addictions: Revalidation of a Survey to Measure Client and Family Experiences of Virtual Care
BackgroundThe onset of the COVID-19 pandemic precipitated a rapid shift to virtual care in health care settings, inclusive of mental health care. Understanding clients’ perspectives on virtual mental health care quality will be critical to informing future policies and practi...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
JMIR Publications
2025-01-01
|
| Series: | Journal of Medical Internet Research |
| Online Access: | https://www.jmir.org/2025/1/e49844 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841560719178334208 |
|---|---|
| author | Allison Crawford Anne Kirvan Marcos Sanches Amanda Gambin Denise Canso Eva Serhal |
| author_facet | Allison Crawford Anne Kirvan Marcos Sanches Amanda Gambin Denise Canso Eva Serhal |
| author_sort | Allison Crawford |
| collection | DOAJ |
| description |
BackgroundThe onset of the COVID-19 pandemic precipitated a rapid shift to virtual care in health care settings, inclusive of mental health care. Understanding clients’ perspectives on virtual mental health care quality will be critical to informing future policies and practices.
ObjectiveThis study aimed to outline the process of redesigning and validating the Virtual Client Experience Survey (VCES), which can be used to evaluate client and family experiences of virtual care, specifically virtual mental health and addiction care.
MethodsThe VCES was adapted from a previously validated telepsychiatry survey. All items were reviewed and updated, with particular attention to the need to ensure relevance across mental health care sectors and settings. The survey was then revalidated using the 6 domains of health care quality of the Institute of Medicine (IOM) as a guiding framework. These 6 domains include being safe, effective, patient-centered, efficient, timely, and equitable. The VCES was piloted with a convenience sample of clients and family members accessing outpatient care at the Centre for Addiction and Mental Health (CAMH) in Toronto, Ontario, through video or telephone. A confirmatory factor analysis (CFA) was conducted in MPlus and used to test the factorial structures of the VCES, with minor respecification of the model based on modification indices, factor loadings, reliability, and item-total correlation. The respecifications were checked for alignment with the construct definitions and item interpretation. The reliability of the constructs was estimated by the Cronbach α coefficient.
ResultsThe survey was completed 181 times. The construct reliability was generally high. Timely was the only subscale with an α lower than 0.7; all others were above 0.8. In all cases, the corrected item-total correlation was higher than 0.3. For the CFA, the model was adjusted after multiple imputations with 20 datasets. The mean chi-square value was 437.5, with df=199 (P<.001). The mean root mean square error of approximation (RMSEA) was 0.08 (SD 0.002), the mean confirmatory fit index (CFI) was 0.987 (SD 0.001), the mean Tucker-Lewis Index (TLI) was 0.985 (SD 0.001), and the mean standardized root mean square residual (SRMR) was 0.04 (SD 0.001).
ConclusionsThis study describes the validation of the VCES to evaluate client and family experiences of virtual mental health and addictions care. Given the widespread uptake of virtual care, this survey has broad applicability across settings that provide mental health and addiction care. The VCES can be used to guide targeted quality improvement initiatives across health care quality domains. By effectively addressing challenges as they emerge, it is anticipated that we will continue to move toward hybrid modalities of practice that leverage the strengths and benefits of telephone, video, and in-person care to effectively respond to unique client and family needs and circumstances. |
| format | Article |
| id | doaj-art-a139559abb7f483494924d5878bb63ec |
| institution | Kabale University |
| issn | 1438-8871 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | JMIR Publications |
| record_format | Article |
| series | Journal of Medical Internet Research |
| spelling | doaj-art-a139559abb7f483494924d5878bb63ec2025-01-03T15:45:27ZengJMIR PublicationsJournal of Medical Internet Research1438-88712025-01-0127e4984410.2196/49844The Virtual Client Experience Survey for Mental Health and Addictions: Revalidation of a Survey to Measure Client and Family Experiences of Virtual CareAllison Crawfordhttps://orcid.org/0000-0002-1320-0664Anne Kirvanhttps://orcid.org/0000-0002-2168-0848Marcos Sancheshttps://orcid.org/0000-0001-9075-1761Amanda Gambinhttps://orcid.org/0009-0006-7810-8750Denise Cansohttps://orcid.org/0009-0008-5997-9862Eva Serhalhttps://orcid.org/0000-0003-0451-8694 BackgroundThe onset of the COVID-19 pandemic precipitated a rapid shift to virtual care in health care settings, inclusive of mental health care. Understanding clients’ perspectives on virtual mental health care quality will be critical to informing future policies and practices. ObjectiveThis study aimed to outline the process of redesigning and validating the Virtual Client Experience Survey (VCES), which can be used to evaluate client and family experiences of virtual care, specifically virtual mental health and addiction care. MethodsThe VCES was adapted from a previously validated telepsychiatry survey. All items were reviewed and updated, with particular attention to the need to ensure relevance across mental health care sectors and settings. The survey was then revalidated using the 6 domains of health care quality of the Institute of Medicine (IOM) as a guiding framework. These 6 domains include being safe, effective, patient-centered, efficient, timely, and equitable. The VCES was piloted with a convenience sample of clients and family members accessing outpatient care at the Centre for Addiction and Mental Health (CAMH) in Toronto, Ontario, through video or telephone. A confirmatory factor analysis (CFA) was conducted in MPlus and used to test the factorial structures of the VCES, with minor respecification of the model based on modification indices, factor loadings, reliability, and item-total correlation. The respecifications were checked for alignment with the construct definitions and item interpretation. The reliability of the constructs was estimated by the Cronbach α coefficient. ResultsThe survey was completed 181 times. The construct reliability was generally high. Timely was the only subscale with an α lower than 0.7; all others were above 0.8. In all cases, the corrected item-total correlation was higher than 0.3. For the CFA, the model was adjusted after multiple imputations with 20 datasets. The mean chi-square value was 437.5, with df=199 (P<.001). The mean root mean square error of approximation (RMSEA) was 0.08 (SD 0.002), the mean confirmatory fit index (CFI) was 0.987 (SD 0.001), the mean Tucker-Lewis Index (TLI) was 0.985 (SD 0.001), and the mean standardized root mean square residual (SRMR) was 0.04 (SD 0.001). ConclusionsThis study describes the validation of the VCES to evaluate client and family experiences of virtual mental health and addictions care. Given the widespread uptake of virtual care, this survey has broad applicability across settings that provide mental health and addiction care. The VCES can be used to guide targeted quality improvement initiatives across health care quality domains. By effectively addressing challenges as they emerge, it is anticipated that we will continue to move toward hybrid modalities of practice that leverage the strengths and benefits of telephone, video, and in-person care to effectively respond to unique client and family needs and circumstances.https://www.jmir.org/2025/1/e49844 |
| spellingShingle | Allison Crawford Anne Kirvan Marcos Sanches Amanda Gambin Denise Canso Eva Serhal The Virtual Client Experience Survey for Mental Health and Addictions: Revalidation of a Survey to Measure Client and Family Experiences of Virtual Care Journal of Medical Internet Research |
| title | The Virtual Client Experience Survey for Mental Health and Addictions: Revalidation of a Survey to Measure Client and Family Experiences of Virtual Care |
| title_full | The Virtual Client Experience Survey for Mental Health and Addictions: Revalidation of a Survey to Measure Client and Family Experiences of Virtual Care |
| title_fullStr | The Virtual Client Experience Survey for Mental Health and Addictions: Revalidation of a Survey to Measure Client and Family Experiences of Virtual Care |
| title_full_unstemmed | The Virtual Client Experience Survey for Mental Health and Addictions: Revalidation of a Survey to Measure Client and Family Experiences of Virtual Care |
| title_short | The Virtual Client Experience Survey for Mental Health and Addictions: Revalidation of a Survey to Measure Client and Family Experiences of Virtual Care |
| title_sort | virtual client experience survey for mental health and addictions revalidation of a survey to measure client and family experiences of virtual care |
| url | https://www.jmir.org/2025/1/e49844 |
| work_keys_str_mv | AT allisoncrawford thevirtualclientexperiencesurveyformentalhealthandaddictionsrevalidationofasurveytomeasureclientandfamilyexperiencesofvirtualcare AT annekirvan thevirtualclientexperiencesurveyformentalhealthandaddictionsrevalidationofasurveytomeasureclientandfamilyexperiencesofvirtualcare AT marcossanches thevirtualclientexperiencesurveyformentalhealthandaddictionsrevalidationofasurveytomeasureclientandfamilyexperiencesofvirtualcare AT amandagambin thevirtualclientexperiencesurveyformentalhealthandaddictionsrevalidationofasurveytomeasureclientandfamilyexperiencesofvirtualcare AT denisecanso thevirtualclientexperiencesurveyformentalhealthandaddictionsrevalidationofasurveytomeasureclientandfamilyexperiencesofvirtualcare AT evaserhal thevirtualclientexperiencesurveyformentalhealthandaddictionsrevalidationofasurveytomeasureclientandfamilyexperiencesofvirtualcare AT allisoncrawford virtualclientexperiencesurveyformentalhealthandaddictionsrevalidationofasurveytomeasureclientandfamilyexperiencesofvirtualcare AT annekirvan virtualclientexperiencesurveyformentalhealthandaddictionsrevalidationofasurveytomeasureclientandfamilyexperiencesofvirtualcare AT marcossanches virtualclientexperiencesurveyformentalhealthandaddictionsrevalidationofasurveytomeasureclientandfamilyexperiencesofvirtualcare AT amandagambin virtualclientexperiencesurveyformentalhealthandaddictionsrevalidationofasurveytomeasureclientandfamilyexperiencesofvirtualcare AT denisecanso virtualclientexperiencesurveyformentalhealthandaddictionsrevalidationofasurveytomeasureclientandfamilyexperiencesofvirtualcare AT evaserhal virtualclientexperiencesurveyformentalhealthandaddictionsrevalidationofasurveytomeasureclientandfamilyexperiencesofvirtualcare |