mmWave radar based robust sign language recognition for the smart museum
A smart museum is a new form of a museum, which uses devices or technologies including the Internet of things (IoT) and artificial intelligence (AI) to build the information interaction channels between people, things, and space.Sign language recognition not only assists the visitors who have hearin...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | zho |
| Published: |
Beijing Xintong Media Co., Ltd
2023-08-01
|
| Series: | Dianxin kexue |
| Subjects: | |
| Online Access: | http://www.telecomsci.com/zh/article/doi/10.11959/j.issn.1000-0801.2023144/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Summary: | A smart museum is a new form of a museum, which uses devices or technologies including the Internet of things (IoT) and artificial intelligence (AI) to build the information interaction channels between people, things, and space.Sign language recognition not only assists the visitors who have hearing or speech impairment to visit the museum without barriers but also helps study the visitors’ natural gesture interaction.However, the methods based on cameras and wearable devices mayhave issues like privacy or usability when applied to museum spaces.Therefore, a robust sign language recognition method based on millimeter-wave radar was proposed.Different features of distance and velocity changes relative to the radar device were firstly extracted in this method, then a physical data enhancement method was adopted to expand the training data.Finally, a ResNet based on the pre-processed distance time features and Doppler time features was designed to further remove the environment-related information and perform feature fusion for classification.Experimental results show that this method can effectively recognize sign language and achieve an averaged recognition accuracy of over 90% when the testing environment and the user＇s location change, providing a new method for smart museum wireless sign language and gesture recognition. |
|---|---|
| ISSN: | 1000-0801 |