Multi-scale attention patching encoder network: a deployable model for continuous estimation of hand kinematics from surface electromyographic signals
Abstract Background Simultaneous and proportional control (SPC) based on surface electromyographic (sEMG) signals has emerged as a research hotspot in the field of human–machine interaction (HMI). However, the existing continuous motion estimation methods mostly have an average Pearson coefficient (...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2024-12-01
|
| Series: | Journal of NeuroEngineering and Rehabilitation |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s12984-024-01525-4 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841559810878734336 |
|---|---|
| author | Chuang Lin Qiong Xiao Penghui Zhao |
| author_facet | Chuang Lin Qiong Xiao Penghui Zhao |
| author_sort | Chuang Lin |
| collection | DOAJ |
| description | Abstract Background Simultaneous and proportional control (SPC) based on surface electromyographic (sEMG) signals has emerged as a research hotspot in the field of human–machine interaction (HMI). However, the existing continuous motion estimation methods mostly have an average Pearson coefficient (CC) of less than 0.85, while high-precision methods suffer from the problem of long inference time (> 200 ms) and can only estimate SPC of less than 15 hand movements, which limits their applications in HMI. To overcome these problems, we propose a smooth Multi-scale Attention Patching Encoder Network (sMAPEN). Methods The sMAPEN consists of three modules, the Multi-scale Attention Fusion (MAF) module, the Patching Encoder (PE) module, and a smoothing layer. The MAF module adaptively captures the local spatiotemporal features at multiple scales, the PE module acquires the global spatiotemporal features of sEMG, and the smoothing layer further improves prediction stability. Results To evaluate the performance of the model, we conducted continuous estimation of 40 subjects performing over 40 different hand movements on the Ninapro DB2. The results show that the average Pearson correlation coefficient (CC), normalized root mean square error (NRMSE), coefficient of determination (R2), and smoothness (SMOOTH) of the sMAPEN model are 0.9082, 0.0646°, 0.8163, and − 0.0017, respectively, which significantly outperforms that of the state-of-the-art methods in all metrics (p < 0.01). Furthermore, we tested the deployment performance of sMAPEN on the portable device, with a delay of only 97.93 ms. Conclusions Our model can predict up to 40 hand movements while achieving the highest predicting accuracy compared with other methods. Besides, the lightweight design strategy brings an improvement in inference speed, which enables the model to be deployed on wearable devices. All these promotions imply that sMAPEN holds great potential in HMI. |
| format | Article |
| id | doaj-art-6369d43691754501b8c8318cb4512579 |
| institution | Kabale University |
| issn | 1743-0003 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | BMC |
| record_format | Article |
| series | Journal of NeuroEngineering and Rehabilitation |
| spelling | doaj-art-6369d43691754501b8c8318cb45125792025-01-05T12:10:31ZengBMCJournal of NeuroEngineering and Rehabilitation1743-00032024-12-0121111410.1186/s12984-024-01525-4Multi-scale attention patching encoder network: a deployable model for continuous estimation of hand kinematics from surface electromyographic signalsChuang Lin0Qiong Xiao1Penghui Zhao2The School of Information Science and Technology, Dalian Maritime UniversityThe School of Information Science and Technology, Dalian Maritime UniversityThe School of Information Science and Technology, Dalian Maritime UniversityAbstract Background Simultaneous and proportional control (SPC) based on surface electromyographic (sEMG) signals has emerged as a research hotspot in the field of human–machine interaction (HMI). However, the existing continuous motion estimation methods mostly have an average Pearson coefficient (CC) of less than 0.85, while high-precision methods suffer from the problem of long inference time (> 200 ms) and can only estimate SPC of less than 15 hand movements, which limits their applications in HMI. To overcome these problems, we propose a smooth Multi-scale Attention Patching Encoder Network (sMAPEN). Methods The sMAPEN consists of three modules, the Multi-scale Attention Fusion (MAF) module, the Patching Encoder (PE) module, and a smoothing layer. The MAF module adaptively captures the local spatiotemporal features at multiple scales, the PE module acquires the global spatiotemporal features of sEMG, and the smoothing layer further improves prediction stability. Results To evaluate the performance of the model, we conducted continuous estimation of 40 subjects performing over 40 different hand movements on the Ninapro DB2. The results show that the average Pearson correlation coefficient (CC), normalized root mean square error (NRMSE), coefficient of determination (R2), and smoothness (SMOOTH) of the sMAPEN model are 0.9082, 0.0646°, 0.8163, and − 0.0017, respectively, which significantly outperforms that of the state-of-the-art methods in all metrics (p < 0.01). Furthermore, we tested the deployment performance of sMAPEN on the portable device, with a delay of only 97.93 ms. Conclusions Our model can predict up to 40 hand movements while achieving the highest predicting accuracy compared with other methods. Besides, the lightweight design strategy brings an improvement in inference speed, which enables the model to be deployed on wearable devices. All these promotions imply that sMAPEN holds great potential in HMI.https://doi.org/10.1186/s12984-024-01525-4sEMGHand kinematicsContinuous estimationMulti-scale attentionPatching encoderTransformer |
| spellingShingle | Chuang Lin Qiong Xiao Penghui Zhao Multi-scale attention patching encoder network: a deployable model for continuous estimation of hand kinematics from surface electromyographic signals Journal of NeuroEngineering and Rehabilitation sEMG Hand kinematics Continuous estimation Multi-scale attention Patching encoder Transformer |
| title | Multi-scale attention patching encoder network: a deployable model for continuous estimation of hand kinematics from surface electromyographic signals |
| title_full | Multi-scale attention patching encoder network: a deployable model for continuous estimation of hand kinematics from surface electromyographic signals |
| title_fullStr | Multi-scale attention patching encoder network: a deployable model for continuous estimation of hand kinematics from surface electromyographic signals |
| title_full_unstemmed | Multi-scale attention patching encoder network: a deployable model for continuous estimation of hand kinematics from surface electromyographic signals |
| title_short | Multi-scale attention patching encoder network: a deployable model for continuous estimation of hand kinematics from surface electromyographic signals |
| title_sort | multi scale attention patching encoder network a deployable model for continuous estimation of hand kinematics from surface electromyographic signals |
| topic | sEMG Hand kinematics Continuous estimation Multi-scale attention Patching encoder Transformer |
| url | https://doi.org/10.1186/s12984-024-01525-4 |
| work_keys_str_mv | AT chuanglin multiscaleattentionpatchingencodernetworkadeployablemodelforcontinuousestimationofhandkinematicsfromsurfaceelectromyographicsignals AT qiongxiao multiscaleattentionpatchingencodernetworkadeployablemodelforcontinuousestimationofhandkinematicsfromsurfaceelectromyographicsignals AT penghuizhao multiscaleattentionpatchingencodernetworkadeployablemodelforcontinuousestimationofhandkinematicsfromsurfaceelectromyographicsignals |