D-SPDH: Improving 3D Robot Pose Estimation in Sim2Real Scenario via Depth Data
In recent years, there has been a notable surge in the significance attributed to technologies facilitating secure and efficient cohabitation and collaboration between humans and machines, with a particular interest in robotic systems. A pivotal element in actualizing this novel and challenging coll...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2024-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10745501/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1846150414452391936 |
|---|---|
| author | Alessandro Simoni Guido Borghi Lorenzo Garattoni Gianpiero Francesca Roberto Vezzani |
| author_facet | Alessandro Simoni Guido Borghi Lorenzo Garattoni Gianpiero Francesca Roberto Vezzani |
| author_sort | Alessandro Simoni |
| collection | DOAJ |
| description | In recent years, there has been a notable surge in the significance attributed to technologies facilitating secure and efficient cohabitation and collaboration between humans and machines, with a particular interest in robotic systems. A pivotal element in actualizing this novel and challenging collaborative paradigm involves different technical tasks, including the comprehension of 3D poses exhibited by both humans and robots through the utilization of non-intrusive systems, such as cameras. In this scenario, the availability of vision-based systems capable of detecting in real-time the robot’s pose is needed as a first step towards a safe and effective interaction to, for instance, avoid collisions. Therefore, in this work, we propose a vision-based system, referred to as D-SPDH, able to estimate the 3D robot pose. The system is based on double-branch architecture and depth data as a single input; any additional information regarding the state of the internal encoders of the robot is not required. The working scenario is the Sim2Real, i.e., the system is trained only with synthetic data and then tested on real sequences, thus eliminating the time-consuming acquisition and annotation procedures of real data, common phases in deep learning algorithms. Moreover, we introduce SimBa++, a dataset featuring both synthetic and real sequences with new real-world double-arm movements, and that represents a challenging setting in which the proposed approach is tested. Experimental results show that our D-SPDH method achieves state-of-the-art and real-time performance, paving the way a possible future non-invasive systems to monitor human-robot interactions. |
| format | Article |
| id | doaj-art-bcda3121d1304ac6a9c69c97fde65f85 |
| institution | Kabale University |
| issn | 2169-3536 |
| language | English |
| publishDate | 2024-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-bcda3121d1304ac6a9c69c97fde65f852024-11-29T00:01:17ZengIEEEIEEE Access2169-35362024-01-011216666016667310.1109/ACCESS.2024.349281210745501D-SPDH: Improving 3D Robot Pose Estimation in Sim2Real Scenario via Depth DataAlessandro Simoni0https://orcid.org/0000-0003-3095-3294Guido Borghi1https://orcid.org/0000-0003-2441-7524Lorenzo Garattoni2https://orcid.org/0000-0002-6382-0335Gianpiero Francesca3Roberto Vezzani4https://orcid.org/0000-0002-1046-6870Department of Engineering “Enzo Ferrari,”, University of Modena and Reggio Emilia, Reggio Emilia, ItalyDipartimento di Educazione e Scienze Umane, University of Modena and Reggio Emilia, Reggio Emilia, ItalyToyota Motor Europe, Brussels, BelgiumToyota Motor Europe, Brussels, BelgiumDepartment of Engineering “Enzo Ferrari,”, University of Modena and Reggio Emilia, Reggio Emilia, ItalyIn recent years, there has been a notable surge in the significance attributed to technologies facilitating secure and efficient cohabitation and collaboration between humans and machines, with a particular interest in robotic systems. A pivotal element in actualizing this novel and challenging collaborative paradigm involves different technical tasks, including the comprehension of 3D poses exhibited by both humans and robots through the utilization of non-intrusive systems, such as cameras. In this scenario, the availability of vision-based systems capable of detecting in real-time the robot’s pose is needed as a first step towards a safe and effective interaction to, for instance, avoid collisions. Therefore, in this work, we propose a vision-based system, referred to as D-SPDH, able to estimate the 3D robot pose. The system is based on double-branch architecture and depth data as a single input; any additional information regarding the state of the internal encoders of the robot is not required. The working scenario is the Sim2Real, i.e., the system is trained only with synthetic data and then tested on real sequences, thus eliminating the time-consuming acquisition and annotation procedures of real data, common phases in deep learning algorithms. Moreover, we introduce SimBa++, a dataset featuring both synthetic and real sequences with new real-world double-arm movements, and that represents a challenging setting in which the proposed approach is tested. Experimental results show that our D-SPDH method achieves state-of-the-art and real-time performance, paving the way a possible future non-invasive systems to monitor human-robot interactions.https://ieeexplore.ieee.org/document/10745501/Human-machine interactionhuman-robot interactioncollaborative robots (Cobots)robot pose estimationdeep learningcomputer vision |
| spellingShingle | Alessandro Simoni Guido Borghi Lorenzo Garattoni Gianpiero Francesca Roberto Vezzani D-SPDH: Improving 3D Robot Pose Estimation in Sim2Real Scenario via Depth Data IEEE Access Human-machine interaction human-robot interaction collaborative robots (Cobots) robot pose estimation deep learning computer vision |
| title | D-SPDH: Improving 3D Robot Pose Estimation in Sim2Real Scenario via Depth Data |
| title_full | D-SPDH: Improving 3D Robot Pose Estimation in Sim2Real Scenario via Depth Data |
| title_fullStr | D-SPDH: Improving 3D Robot Pose Estimation in Sim2Real Scenario via Depth Data |
| title_full_unstemmed | D-SPDH: Improving 3D Robot Pose Estimation in Sim2Real Scenario via Depth Data |
| title_short | D-SPDH: Improving 3D Robot Pose Estimation in Sim2Real Scenario via Depth Data |
| title_sort | d spdh improving 3d robot pose estimation in sim2real scenario via depth data |
| topic | Human-machine interaction human-robot interaction collaborative robots (Cobots) robot pose estimation deep learning computer vision |
| url | https://ieeexplore.ieee.org/document/10745501/ |
| work_keys_str_mv | AT alessandrosimoni dspdhimproving3drobotposeestimationinsim2realscenarioviadepthdata AT guidoborghi dspdhimproving3drobotposeestimationinsim2realscenarioviadepthdata AT lorenzogarattoni dspdhimproving3drobotposeestimationinsim2realscenarioviadepthdata AT gianpierofrancesca dspdhimproving3drobotposeestimationinsim2realscenarioviadepthdata AT robertovezzani dspdhimproving3drobotposeestimationinsim2realscenarioviadepthdata |