Next generation of electronic medical record search engines to support chart reviews: A systematic user study and future research direction
Objective: Little research has been done on the user-centered document ranking approach, especially in a crowdsourcing chart review environment. As the starting point of designing and implementing the next generation of Electronic Medical Record (EMR) search engines, a systematic user study is neede...
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
KeAi Communications Co., Ltd.
2024-11-01
|
| Series: | Journal of Economy and Technology |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2949948824000179 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Summary: | Objective: Little research has been done on the user-centered document ranking approach, especially in a crowdsourcing chart review environment. As the starting point of designing and implementing the next generation of Electronic Medical Record (EMR) search engines, a systematic user study is needed to better understand the users' needs, challenges, and future research directions of EMR search engines. Materials and methods: One primary observation during the user study is the need for a ranking method to better support the so-called ''early stopping'' reviewing strategy (i.e., reviewing only a subset of EMRs of one patient to make the final decision) during the clinical chart reviews. The authors proposed two novel user-centered ranking metrics: ''critical documents'' and ''negative guarantee ratio,'' to better measure the power of a ranking method in supporting the “early stopping” requirements during clinical chart reviews. Results: The evaluation results show that i) traditional information retrieval metrics, such as the precision-at-K, have limitations in guiding the design and development of EMR search engines to better support clinical chart reviews; ii) there is not a global optimal ranking method that fits the needs of different chart reviews and different users; iii) a learning-to-rank approach cannot guarantee a stable and optimal ranking for different chart reviews and different users; and iv) A user-centered ranking metric, such as the negative guarantee ratio (NGR) metric is able to measure the “early-stopping” performance of ranking methods. Conclusions: User-centered ranking metrics can better measure the power of ranking methods in supporting clinical chart reviews. Future research should explore more user-centered ranking metrics and evaluate their impact on real-world EMR search engines. |
|---|---|
| ISSN: | 2949-9488 |