Reconstructing Multi-Stroke Characters From Brain Signals Toward Generalizable Handwriting Brain–Computer Interfaces
Handwriting Brain-Computer Interfaces (BCIs) provides a promising communication avenue for individuals with paralysis. While English-based handwriting BCIs have achieved rapid typewriting with 26 lowercase letters (mostly containing one stroke each), it is difficult to extend to complex characters,...
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2024-01-01
|
| Series: | IEEE Transactions on Neural Systems and Rehabilitation Engineering |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10745614/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1846126358788308992 |
|---|---|
| author | Xiaomeng Yang Xinzhu Xiong Xufei Li Qi Lian Junming Zhu Jianmin Zhang Yu Qi Yueming Wang |
| author_facet | Xiaomeng Yang Xinzhu Xiong Xufei Li Qi Lian Junming Zhu Jianmin Zhang Yu Qi Yueming Wang |
| author_sort | Xiaomeng Yang |
| collection | DOAJ |
| description | Handwriting Brain-Computer Interfaces (BCIs) provides a promising communication avenue for individuals with paralysis. While English-based handwriting BCIs have achieved rapid typewriting with 26 lowercase letters (mostly containing one stroke each), it is difficult to extend to complex characters, especially those with multiple strokes and large character sets. The Chinese characters, including over 3500 commonly used characters with 10.3 strokes per character on average, represent a highly complex writing system. This paper proposes a Chinese handwriting BCI system, which reconstructs multi-stroke handwriting trajectories from brain signals. Through the recording of cortical neural signals from the motor cortex, we reveal distinct neural representations for stroke-writing and pen-lift phases. Leveraging this finding, we propose a stroke-aware approach to decode stroke-writing trajectories and pen-lift movements individually, which can reconstruct recognizable characters (accuracy of 86% with 400 characters). Our approach demonstrates high stability over 5 months, shedding light on generalized and adaptable handwriting BCIs. |
| format | Article |
| id | doaj-art-7cb8811252b444608bcd1d6e97a8f52b |
| institution | Kabale University |
| issn | 1534-4320 1558-0210 |
| language | English |
| publishDate | 2024-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Transactions on Neural Systems and Rehabilitation Engineering |
| spelling | doaj-art-7cb8811252b444608bcd1d6e97a8f52b2024-12-13T00:00:11ZengIEEEIEEE Transactions on Neural Systems and Rehabilitation Engineering1534-43201558-02102024-01-01324230423910.1109/TNSRE.2024.349219110745614Reconstructing Multi-Stroke Characters From Brain Signals Toward Generalizable Handwriting Brain–Computer InterfacesXiaomeng Yang0https://orcid.org/0009-0002-7527-1211Xinzhu Xiong1Xufei Li2Qi Lian3Junming Zhu4https://orcid.org/0000-0002-2781-3306Jianmin Zhang5https://orcid.org/0000-0002-3184-1502Yu Qi6https://orcid.org/0000-0002-9789-8907Yueming Wang7https://orcid.org/0000-0001-7742-0722Mental Health Center and Hangzhou Seventh People’s Hospital, Zhejiang University School of Medicine, Hangzhou, ChinaMental Health Center and Hangzhou Seventh People’s Hospital, Zhejiang University School of Medicine, Hangzhou, ChinaNanhu Brain-computer Interface Institute, Hangzhou, ChinaCollege of Computer Science, Zhejiang University, Hangzhou, ChinaHospital of Zhejiang University School of Medicine, Hangzhou, ChinaHospital of Zhejiang University School of Medicine, Hangzhou, ChinaMental Health Center and Hangzhou Seventh People’s Hospital, Zhejiang University School of Medicine, Hangzhou, ChinaCollege of Computer Science, Zhejiang University, Hangzhou, ChinaHandwriting Brain-Computer Interfaces (BCIs) provides a promising communication avenue for individuals with paralysis. While English-based handwriting BCIs have achieved rapid typewriting with 26 lowercase letters (mostly containing one stroke each), it is difficult to extend to complex characters, especially those with multiple strokes and large character sets. The Chinese characters, including over 3500 commonly used characters with 10.3 strokes per character on average, represent a highly complex writing system. This paper proposes a Chinese handwriting BCI system, which reconstructs multi-stroke handwriting trajectories from brain signals. Through the recording of cortical neural signals from the motor cortex, we reveal distinct neural representations for stroke-writing and pen-lift phases. Leveraging this finding, we propose a stroke-aware approach to decode stroke-writing trajectories and pen-lift movements individually, which can reconstruct recognizable characters (accuracy of 86% with 400 characters). Our approach demonstrates high stability over 5 months, shedding light on generalized and adaptable handwriting BCIs.https://ieeexplore.ieee.org/document/10745614/Brain-computer interfaces (BCIs)neural signal decodinghandwriting reconstructionmulti-stroke characters |
| spellingShingle | Xiaomeng Yang Xinzhu Xiong Xufei Li Qi Lian Junming Zhu Jianmin Zhang Yu Qi Yueming Wang Reconstructing Multi-Stroke Characters From Brain Signals Toward Generalizable Handwriting Brain–Computer Interfaces IEEE Transactions on Neural Systems and Rehabilitation Engineering Brain-computer interfaces (BCIs) neural signal decoding handwriting reconstruction multi-stroke characters |
| title | Reconstructing Multi-Stroke Characters From Brain Signals Toward Generalizable Handwriting Brain–Computer Interfaces |
| title_full | Reconstructing Multi-Stroke Characters From Brain Signals Toward Generalizable Handwriting Brain–Computer Interfaces |
| title_fullStr | Reconstructing Multi-Stroke Characters From Brain Signals Toward Generalizable Handwriting Brain–Computer Interfaces |
| title_full_unstemmed | Reconstructing Multi-Stroke Characters From Brain Signals Toward Generalizable Handwriting Brain–Computer Interfaces |
| title_short | Reconstructing Multi-Stroke Characters From Brain Signals Toward Generalizable Handwriting Brain–Computer Interfaces |
| title_sort | reconstructing multi stroke characters from brain signals toward generalizable handwriting brain x2013 computer interfaces |
| topic | Brain-computer interfaces (BCIs) neural signal decoding handwriting reconstruction multi-stroke characters |
| url | https://ieeexplore.ieee.org/document/10745614/ |
| work_keys_str_mv | AT xiaomengyang reconstructingmultistrokecharactersfrombrainsignalstowardgeneralizablehandwritingbrainx2013computerinterfaces AT xinzhuxiong reconstructingmultistrokecharactersfrombrainsignalstowardgeneralizablehandwritingbrainx2013computerinterfaces AT xufeili reconstructingmultistrokecharactersfrombrainsignalstowardgeneralizablehandwritingbrainx2013computerinterfaces AT qilian reconstructingmultistrokecharactersfrombrainsignalstowardgeneralizablehandwritingbrainx2013computerinterfaces AT junmingzhu reconstructingmultistrokecharactersfrombrainsignalstowardgeneralizablehandwritingbrainx2013computerinterfaces AT jianminzhang reconstructingmultistrokecharactersfrombrainsignalstowardgeneralizablehandwritingbrainx2013computerinterfaces AT yuqi reconstructingmultistrokecharactersfrombrainsignalstowardgeneralizablehandwritingbrainx2013computerinterfaces AT yuemingwang reconstructingmultistrokecharactersfrombrainsignalstowardgeneralizablehandwritingbrainx2013computerinterfaces |