Next generation of electronic medical record search engines to support chart reviews: A systematic user study and future research direction
Objective: Little research has been done on the user-centered document ranking approach, especially in a crowdsourcing chart review environment. As the starting point of designing and implementing the next generation of Electronic Medical Record (EMR) search engines, a systematic user study is neede...
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
KeAi Communications Co., Ltd.
2024-11-01
|
| Series: | Journal of Economy and Technology |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2949948824000179 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1846170008743313408 |
|---|---|
| author | Cheng Ye Daniel Fabbri |
| author_facet | Cheng Ye Daniel Fabbri |
| author_sort | Cheng Ye |
| collection | DOAJ |
| description | Objective: Little research has been done on the user-centered document ranking approach, especially in a crowdsourcing chart review environment. As the starting point of designing and implementing the next generation of Electronic Medical Record (EMR) search engines, a systematic user study is needed to better understand the users' needs, challenges, and future research directions of EMR search engines. Materials and methods: One primary observation during the user study is the need for a ranking method to better support the so-called ''early stopping'' reviewing strategy (i.e., reviewing only a subset of EMRs of one patient to make the final decision) during the clinical chart reviews. The authors proposed two novel user-centered ranking metrics: ''critical documents'' and ''negative guarantee ratio,'' to better measure the power of a ranking method in supporting the “early stopping” requirements during clinical chart reviews. Results: The evaluation results show that i) traditional information retrieval metrics, such as the precision-at-K, have limitations in guiding the design and development of EMR search engines to better support clinical chart reviews; ii) there is not a global optimal ranking method that fits the needs of different chart reviews and different users; iii) a learning-to-rank approach cannot guarantee a stable and optimal ranking for different chart reviews and different users; and iv) A user-centered ranking metric, such as the negative guarantee ratio (NGR) metric is able to measure the “early-stopping” performance of ranking methods. Conclusions: User-centered ranking metrics can better measure the power of ranking methods in supporting clinical chart reviews. Future research should explore more user-centered ranking metrics and evaluate their impact on real-world EMR search engines. |
| format | Article |
| id | doaj-art-4f6fe14151d5410d81b8535ed7da6023 |
| institution | Kabale University |
| issn | 2949-9488 |
| language | English |
| publishDate | 2024-11-01 |
| publisher | KeAi Communications Co., Ltd. |
| record_format | Article |
| series | Journal of Economy and Technology |
| spelling | doaj-art-4f6fe14151d5410d81b8535ed7da60232024-11-12T05:22:25ZengKeAi Communications Co., Ltd.Journal of Economy and Technology2949-94882024-11-0122230Next generation of electronic medical record search engines to support chart reviews: A systematic user study and future research directionCheng Ye0Daniel Fabbri1Correspondence to: Department of Biomedical Informatics, Vanderbilt University Medical Center, 2525 West End Ave # 1475, Nashville, TN 37203, USA.; Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN, USADepartment of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN, USAObjective: Little research has been done on the user-centered document ranking approach, especially in a crowdsourcing chart review environment. As the starting point of designing and implementing the next generation of Electronic Medical Record (EMR) search engines, a systematic user study is needed to better understand the users' needs, challenges, and future research directions of EMR search engines. Materials and methods: One primary observation during the user study is the need for a ranking method to better support the so-called ''early stopping'' reviewing strategy (i.e., reviewing only a subset of EMRs of one patient to make the final decision) during the clinical chart reviews. The authors proposed two novel user-centered ranking metrics: ''critical documents'' and ''negative guarantee ratio,'' to better measure the power of a ranking method in supporting the “early stopping” requirements during clinical chart reviews. Results: The evaluation results show that i) traditional information retrieval metrics, such as the precision-at-K, have limitations in guiding the design and development of EMR search engines to better support clinical chart reviews; ii) there is not a global optimal ranking method that fits the needs of different chart reviews and different users; iii) a learning-to-rank approach cannot guarantee a stable and optimal ranking for different chart reviews and different users; and iv) A user-centered ranking metric, such as the negative guarantee ratio (NGR) metric is able to measure the “early-stopping” performance of ranking methods. Conclusions: User-centered ranking metrics can better measure the power of ranking methods in supporting clinical chart reviews. Future research should explore more user-centered ranking metrics and evaluate their impact on real-world EMR search engines.http://www.sciencedirect.com/science/article/pii/S2949948824000179Electronic medical recordsClinical chart reviewsSearch engineRanking metricsUser studySemantic embeddings |
| spellingShingle | Cheng Ye Daniel Fabbri Next generation of electronic medical record search engines to support chart reviews: A systematic user study and future research direction Journal of Economy and Technology Electronic medical records Clinical chart reviews Search engine Ranking metrics User study Semantic embeddings |
| title | Next generation of electronic medical record search engines to support chart reviews: A systematic user study and future research direction |
| title_full | Next generation of electronic medical record search engines to support chart reviews: A systematic user study and future research direction |
| title_fullStr | Next generation of electronic medical record search engines to support chart reviews: A systematic user study and future research direction |
| title_full_unstemmed | Next generation of electronic medical record search engines to support chart reviews: A systematic user study and future research direction |
| title_short | Next generation of electronic medical record search engines to support chart reviews: A systematic user study and future research direction |
| title_sort | next generation of electronic medical record search engines to support chart reviews a systematic user study and future research direction |
| topic | Electronic medical records Clinical chart reviews Search engine Ranking metrics User study Semantic embeddings |
| url | http://www.sciencedirect.com/science/article/pii/S2949948824000179 |
| work_keys_str_mv | AT chengye nextgenerationofelectronicmedicalrecordsearchenginestosupportchartreviewsasystematicuserstudyandfutureresearchdirection AT danielfabbri nextgenerationofelectronicmedicalrecordsearchenginestosupportchartreviewsasystematicuserstudyandfutureresearchdirection |