mmWave radar based robust sign language recognition for the smart museum
A smart museum is a new form of a museum, which uses devices or technologies including the Internet of things (IoT) and artificial intelligence (AI) to build the information interaction channels between people, things, and space.Sign language recognition not only assists the visitors who have hearin...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | zho |
| Published: |
Beijing Xintong Media Co., Ltd
2023-08-01
|
| Series: | Dianxin kexue |
| Subjects: | |
| Online Access: | http://www.telecomsci.com/zh/article/doi/10.11959/j.issn.1000-0801.2023144/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841530808939053056 |
|---|---|
| author | Xuerong ZHAO Xuan WANG Tong LIU Xia ZHENG Yicheng JIANG |
| author_facet | Xuerong ZHAO Xuan WANG Tong LIU Xia ZHENG Yicheng JIANG |
| author_sort | Xuerong ZHAO |
| collection | DOAJ |
| description | A smart museum is a new form of a museum, which uses devices or technologies including the Internet of things (IoT) and artificial intelligence (AI) to build the information interaction channels between people, things, and space.Sign language recognition not only assists the visitors who have hearing or speech impairment to visit the museum without barriers but also helps study the visitors’ natural gesture interaction.However, the methods based on cameras and wearable devices mayhave issues like privacy or usability when applied to museum spaces.Therefore, a robust sign language recognition method based on millimeter-wave radar was proposed.Different features of distance and velocity changes relative to the radar device were firstly extracted in this method, then a physical data enhancement method was adopted to expand the training data.Finally, a ResNet based on the pre-processed distance time features and Doppler time features was designed to further remove the environment-related information and perform feature fusion for classification.Experimental results show that this method can effectively recognize sign language and achieve an averaged recognition accuracy of over 90% when the testing environment and the user＇s location change, providing a new method for smart museum wireless sign language and gesture recognition. |
| format | Article |
| id | doaj-art-19b02fc11f6d44b681bc6d9edada6d00 |
| institution | Kabale University |
| issn | 1000-0801 |
| language | zho |
| publishDate | 2023-08-01 |
| publisher | Beijing Xintong Media Co., Ltd |
| record_format | Article |
| series | Dianxin kexue |
| spelling | doaj-art-19b02fc11f6d44b681bc6d9edada6d002025-01-15T02:58:19ZzhoBeijing Xintong Media Co., LtdDianxin kexue1000-08012023-08-013910911759562800mmWave radar based robust sign language recognition for the smart museumXuerong ZHAOXuan WANGTong LIUXia ZHENGYicheng JIANGA smart museum is a new form of a museum, which uses devices or technologies including the Internet of things (IoT) and artificial intelligence (AI) to build the information interaction channels between people, things, and space.Sign language recognition not only assists the visitors who have hearing or speech impairment to visit the museum without barriers but also helps study the visitors’ natural gesture interaction.However, the methods based on cameras and wearable devices mayhave issues like privacy or usability when applied to museum spaces.Therefore, a robust sign language recognition method based on millimeter-wave radar was proposed.Different features of distance and velocity changes relative to the radar device were firstly extracted in this method, then a physical data enhancement method was adopted to expand the training data.Finally, a ResNet based on the pre-processed distance time features and Doppler time features was designed to further remove the environment-related information and perform feature fusion for classification.Experimental results show that this method can effectively recognize sign language and achieve an averaged recognition accuracy of over 90% when the testing environment and the user＇s location change, providing a new method for smart museum wireless sign language and gesture recognition.http://www.telecomsci.com/zh/article/doi/10.11959/j.issn.1000-0801.2023144/sign language recognitionmillimeter-wave radarResNetsmart museum |
| spellingShingle | Xuerong ZHAO Xuan WANG Tong LIU Xia ZHENG Yicheng JIANG mmWave radar based robust sign language recognition for the smart museum Dianxin kexue sign language recognition millimeter-wave radar ResNet smart museum |
| title | mmWave radar based robust sign language recognition for the smart museum |
| title_full | mmWave radar based robust sign language recognition for the smart museum |
| title_fullStr | mmWave radar based robust sign language recognition for the smart museum |
| title_full_unstemmed | mmWave radar based robust sign language recognition for the smart museum |
| title_short | mmWave radar based robust sign language recognition for the smart museum |
| title_sort | mmwave radar based robust sign language recognition for the smart museum |
| topic | sign language recognition millimeter-wave radar ResNet smart museum |
| url | http://www.telecomsci.com/zh/article/doi/10.11959/j.issn.1000-0801.2023144/ |
| work_keys_str_mv | AT xuerongzhao mmwaveradarbasedrobustsignlanguagerecognitionforthesmartmuseum AT xuanwang mmwaveradarbasedrobustsignlanguagerecognitionforthesmartmuseum AT tongliu mmwaveradarbasedrobustsignlanguagerecognitionforthesmartmuseum AT xiazheng mmwaveradarbasedrobustsignlanguagerecognitionforthesmartmuseum AT yichengjiang mmwaveradarbasedrobustsignlanguagerecognitionforthesmartmuseum |