Toward Semi-Autonomous Robotic Arm Manipulation Operator Intention Detection From Force Data
In hazardous environments like nuclear facilities, robotic systems are essential for executing tasks that would otherwise expose humans to dangerous radiation levels, which pose severe health risks and can be fatal. However, many operations in the nuclear environment require teleoperating robots, re...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10816636/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841563350522134528 |
|---|---|
| author | Abdullah S. Alharthi Ozan Tokatli Erwin Lopez Guido Herrmann |
| author_facet | Abdullah S. Alharthi Ozan Tokatli Erwin Lopez Guido Herrmann |
| author_sort | Abdullah S. Alharthi |
| collection | DOAJ |
| description | In hazardous environments like nuclear facilities, robotic systems are essential for executing tasks that would otherwise expose humans to dangerous radiation levels, which pose severe health risks and can be fatal. However, many operations in the nuclear environment require teleoperating robots, resulting in a significant cognitive load on operators as well as physical strain over extended periods of time. To address this challenge, we propose enhancing the teleoperation system with an assistive model capable of predicting operator intentions and dynamically adapting to their needs. The machine learning model processes robotic arm force data, analyzing spatiotemporal patterns to accurately detect the ongoing task before its completion. To support this approach, we collected a diverse dataset from teleoperation experiments involving glovebox tasks in nuclear applications. This dataset encompasses heterogeneous spatiotemporal data captured from the teleoperation system. We employ a hybrid Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM) model to learn and forecast operator intentions based on the spatiotemporal data. By accurately predicting these intentions, the robot can execute tasks more efficiently and effectively, requiring minimal input from the operator. Our experiments validated the model using the dataset, focusing on tasks such as radiation surveys and object grasping. The proposed approach demonstrated an F1-score of 89% for task classification and an F1-score of 86% classification forecasted operator intentions over a 5-second window. These results highlight the potential of our method to improve the safety, precision, and efficiency of robotic operations in hazardous environments, thereby significantly reducing human radiation exposure. |
| format | Article |
| id | doaj-art-2bd868c01bf64d36b4e868b1b57ec86f |
| institution | Kabale University |
| issn | 2169-3536 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-2bd868c01bf64d36b4e868b1b57ec86f2025-01-03T00:01:38ZengIEEEIEEE Access2169-35362025-01-011366468010.1109/ACCESS.2024.352332510816636Toward Semi-Autonomous Robotic Arm Manipulation Operator Intention Detection From Force DataAbdullah S. Alharthi0https://orcid.org/0000-0001-6776-7503Ozan Tokatli1https://orcid.org/0000-0002-4920-1634Erwin Lopez2https://orcid.org/0000-0001-9927-6688Guido Herrmann3https://orcid.org/0000-0001-5390-4538Electrical Engineering Department, College of Engineering, King Khalid University, Abha, Saudi ArabiaRemote Applications in Challenging Environments, United Kingdom Atomic Energy Authority, Abingdon, Oxford, U.K.Electrical Engineering Department, The University of Manchester, Manchester, U.K.Electrical Engineering Department, The University of Manchester, Manchester, U.K.In hazardous environments like nuclear facilities, robotic systems are essential for executing tasks that would otherwise expose humans to dangerous radiation levels, which pose severe health risks and can be fatal. However, many operations in the nuclear environment require teleoperating robots, resulting in a significant cognitive load on operators as well as physical strain over extended periods of time. To address this challenge, we propose enhancing the teleoperation system with an assistive model capable of predicting operator intentions and dynamically adapting to their needs. The machine learning model processes robotic arm force data, analyzing spatiotemporal patterns to accurately detect the ongoing task before its completion. To support this approach, we collected a diverse dataset from teleoperation experiments involving glovebox tasks in nuclear applications. This dataset encompasses heterogeneous spatiotemporal data captured from the teleoperation system. We employ a hybrid Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM) model to learn and forecast operator intentions based on the spatiotemporal data. By accurately predicting these intentions, the robot can execute tasks more efficiently and effectively, requiring minimal input from the operator. Our experiments validated the model using the dataset, focusing on tasks such as radiation surveys and object grasping. The proposed approach demonstrated an F1-score of 89% for task classification and an F1-score of 86% classification forecasted operator intentions over a 5-second window. These results highlight the potential of our method to improve the safety, precision, and efficiency of robotic operations in hazardous environments, thereby significantly reducing human radiation exposure.https://ieeexplore.ieee.org/document/10816636/Convolutional neural networkstele-manipulation systemsspatiotemporal datarobotic armglovebox |
| spellingShingle | Abdullah S. Alharthi Ozan Tokatli Erwin Lopez Guido Herrmann Toward Semi-Autonomous Robotic Arm Manipulation Operator Intention Detection From Force Data IEEE Access Convolutional neural networks tele-manipulation systems spatiotemporal data robotic arm glovebox |
| title | Toward Semi-Autonomous Robotic Arm Manipulation Operator Intention Detection From Force Data |
| title_full | Toward Semi-Autonomous Robotic Arm Manipulation Operator Intention Detection From Force Data |
| title_fullStr | Toward Semi-Autonomous Robotic Arm Manipulation Operator Intention Detection From Force Data |
| title_full_unstemmed | Toward Semi-Autonomous Robotic Arm Manipulation Operator Intention Detection From Force Data |
| title_short | Toward Semi-Autonomous Robotic Arm Manipulation Operator Intention Detection From Force Data |
| title_sort | toward semi autonomous robotic arm manipulation operator intention detection from force data |
| topic | Convolutional neural networks tele-manipulation systems spatiotemporal data robotic arm glovebox |
| url | https://ieeexplore.ieee.org/document/10816636/ |
| work_keys_str_mv | AT abdullahsalharthi towardsemiautonomousroboticarmmanipulationoperatorintentiondetectionfromforcedata AT ozantokatli towardsemiautonomousroboticarmmanipulationoperatorintentiondetectionfromforcedata AT erwinlopez towardsemiautonomousroboticarmmanipulationoperatorintentiondetectionfromforcedata AT guidoherrmann towardsemiautonomousroboticarmmanipulationoperatorintentiondetectionfromforcedata |