Teleoperation system for multiple robots with intuitive hand recognition interface
Abstract Robotic teleoperation is essential for hazardous environments where human safety is at risk. However, efficient and intuitive human–machine interaction for multi-robot systems remains challenging. This article aims to demonstrate a robotic teleoperation system, denominated AutoNav, centered...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2024-12-01
|
| Series: | Scientific Reports |
| Subjects: | |
| Online Access: | https://doi.org/10.1038/s41598-024-80898-x |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1846137083562819584 |
|---|---|
| author | Lucas Alexandre Zick Dieisson Martinelli André Schneider de Oliveira Vivian Cremer Kalempa |
| author_facet | Lucas Alexandre Zick Dieisson Martinelli André Schneider de Oliveira Vivian Cremer Kalempa |
| author_sort | Lucas Alexandre Zick |
| collection | DOAJ |
| description | Abstract Robotic teleoperation is essential for hazardous environments where human safety is at risk. However, efficient and intuitive human–machine interaction for multi-robot systems remains challenging. This article aims to demonstrate a robotic teleoperation system, denominated AutoNav, centered around autonomous navigation and gesture commands interpreted through computer vision. The central focus is on recognizing the palm of the hand as a control interface to facilitate human–machine interaction in the context of multi-robots. The MediaPipe framework was integrated to implement gesture recognition from a USB camera. The system was developed using the Robot Operating System, employing a simulated environment that includes the Gazebo and RViz applications with multiple TurtleBot 3 robots. The main results show a reduction of approximately 50% in the execution time, coupled with an increase in free time during teleoperation, reaching up to 94% of the total execution time. Furthermore, there is a decrease in collisions. These results demonstrate the effectiveness and practicality of the robotic control algorithm, showcasing its promise in managing teleoperations across multi-robots. This study fills a knowledge gap by developing a hand gesture-based control interface for more efficient and safer multi-robot teleoperation. These findings enhance human–machine interaction in complex robotic operations. A video showing the system working is available at https://youtu.be/94S4nJ3IwUw . |
| format | Article |
| id | doaj-art-3c722e7ea7d448d7aceef2e7b3da3c0b |
| institution | Kabale University |
| issn | 2045-2322 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj-art-3c722e7ea7d448d7aceef2e7b3da3c0b2024-12-08T12:29:28ZengNature PortfolioScientific Reports2045-23222024-12-0114111110.1038/s41598-024-80898-xTeleoperation system for multiple robots with intuitive hand recognition interfaceLucas Alexandre Zick0Dieisson Martinelli1André Schneider de Oliveira2Vivian Cremer Kalempa3Graduate Program in Electrical and Computer Engineering, Universidade Tecnolágica Federal do Paraná (UTFPR)Department of Information Systems, Universidade do Estado de Santa Catarina (UDESC)Graduate Program in Electrical and Computer Engineering, Universidade Tecnolágica Federal do Paraná (UTFPR)Department of Information Systems, Universidade do Estado de Santa Catarina (UDESC)Abstract Robotic teleoperation is essential for hazardous environments where human safety is at risk. However, efficient and intuitive human–machine interaction for multi-robot systems remains challenging. This article aims to demonstrate a robotic teleoperation system, denominated AutoNav, centered around autonomous navigation and gesture commands interpreted through computer vision. The central focus is on recognizing the palm of the hand as a control interface to facilitate human–machine interaction in the context of multi-robots. The MediaPipe framework was integrated to implement gesture recognition from a USB camera. The system was developed using the Robot Operating System, employing a simulated environment that includes the Gazebo and RViz applications with multiple TurtleBot 3 robots. The main results show a reduction of approximately 50% in the execution time, coupled with an increase in free time during teleoperation, reaching up to 94% of the total execution time. Furthermore, there is a decrease in collisions. These results demonstrate the effectiveness and practicality of the robotic control algorithm, showcasing its promise in managing teleoperations across multi-robots. This study fills a knowledge gap by developing a hand gesture-based control interface for more efficient and safer multi-robot teleoperation. These findings enhance human–machine interaction in complex robotic operations. A video showing the system working is available at https://youtu.be/94S4nJ3IwUw .https://doi.org/10.1038/s41598-024-80898-xAutonomous navigationHuman–robotMulti-robotsROSTeleoperation |
| spellingShingle | Lucas Alexandre Zick Dieisson Martinelli André Schneider de Oliveira Vivian Cremer Kalempa Teleoperation system for multiple robots with intuitive hand recognition interface Scientific Reports Autonomous navigation Human–robot Multi-robots ROS Teleoperation |
| title | Teleoperation system for multiple robots with intuitive hand recognition interface |
| title_full | Teleoperation system for multiple robots with intuitive hand recognition interface |
| title_fullStr | Teleoperation system for multiple robots with intuitive hand recognition interface |
| title_full_unstemmed | Teleoperation system for multiple robots with intuitive hand recognition interface |
| title_short | Teleoperation system for multiple robots with intuitive hand recognition interface |
| title_sort | teleoperation system for multiple robots with intuitive hand recognition interface |
| topic | Autonomous navigation Human–robot Multi-robots ROS Teleoperation |
| url | https://doi.org/10.1038/s41598-024-80898-x |
| work_keys_str_mv | AT lucasalexandrezick teleoperationsystemformultiplerobotswithintuitivehandrecognitioninterface AT dieissonmartinelli teleoperationsystemformultiplerobotswithintuitivehandrecognitioninterface AT andreschneiderdeoliveira teleoperationsystemformultiplerobotswithintuitivehandrecognitioninterface AT viviancremerkalempa teleoperationsystemformultiplerobotswithintuitivehandrecognitioninterface |