See-Then-Grasp: Object Full 3D Reconstruction via Two-Stage Active Robotic Reconstruction Using Single Manipulator
In this paper, we propose an active robotic 3D reconstruction methodology for achieving full object 3D reconstruction. Existing robotic 3D reconstruction approaches often struggle to cover the entire view space of the object or reconstruct occluded regions, such as the bottom or back side. To addres...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-12-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/15/1/272 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841549397641396224 |
|---|---|
| author | Youngtaek Hong Jonghyeon Kim Geonho Cha Eunwoo Kim Kyungjae Lee |
| author_facet | Youngtaek Hong Jonghyeon Kim Geonho Cha Eunwoo Kim Kyungjae Lee |
| author_sort | Youngtaek Hong |
| collection | DOAJ |
| description | In this paper, we propose an active robotic 3D reconstruction methodology for achieving full object 3D reconstruction. Existing robotic 3D reconstruction approaches often struggle to cover the entire view space of the object or reconstruct occluded regions, such as the bottom or back side. To address these limitations, we introduce a two-stage robotic active 3D reconstruction pipeline, named See-Then-Grasp (STG), that employs a robot manipulator for direct interaction with the object. The manipulator moves toward the points with the highest uncertainty, ensuring efficient data acquisition and rapid reconstruction. Our method expands the view space of the object to include the entire perspective, including occluded areas, making the previous fixed view candidate approach time-consuming for identifying uncertain regions. To overcome this, we propose a gradient-based next best view pose optimization method that efficiently identifies uncertain regions, enabling faster and more effective reconstruction. Our method optimizes the camera pose based on an uncertainty function, allowing it to identify the most uncertain regions in a short time. Through experiments with synthetic objects, we demonstrate that our approach effectively addresses the next best view selection problem, achieving significant improvements in computational efficiency while maintaining high-quality 3D reconstruction. Furthermore, we validate our method on a real robot, showing that it enables full 3D reconstruction of real-world objects. |
| format | Article |
| id | doaj-art-12b85ca30d6c41b793f95b4734224f49 |
| institution | Kabale University |
| issn | 2076-3417 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj-art-12b85ca30d6c41b793f95b4734224f492025-01-10T13:15:00ZengMDPI AGApplied Sciences2076-34172024-12-0115127210.3390/app15010272See-Then-Grasp: Object Full 3D Reconstruction via Two-Stage Active Robotic Reconstruction Using Single ManipulatorYoungtaek Hong0Jonghyeon Kim1Geonho Cha2Eunwoo Kim3Kyungjae Lee4Department of Artificial Intelligence, Chung-Ang University, Seoul 06974, Republic of KoreaDepartment of Artificial Intelligence, Chung-Ang University, Seoul 06974, Republic of KoreaNAVER Cloud, Seongnam 13561, Republic of KoreaDepartment of Computer Science and Engineering, Chung-Ang University, Seoul 06974, Republic of KoreaDepartment of Statistics, Korea University, Seoul 02841, Republic of KoreaIn this paper, we propose an active robotic 3D reconstruction methodology for achieving full object 3D reconstruction. Existing robotic 3D reconstruction approaches often struggle to cover the entire view space of the object or reconstruct occluded regions, such as the bottom or back side. To address these limitations, we introduce a two-stage robotic active 3D reconstruction pipeline, named See-Then-Grasp (STG), that employs a robot manipulator for direct interaction with the object. The manipulator moves toward the points with the highest uncertainty, ensuring efficient data acquisition and rapid reconstruction. Our method expands the view space of the object to include the entire perspective, including occluded areas, making the previous fixed view candidate approach time-consuming for identifying uncertain regions. To overcome this, we propose a gradient-based next best view pose optimization method that efficiently identifies uncertain regions, enabling faster and more effective reconstruction. Our method optimizes the camera pose based on an uncertainty function, allowing it to identify the most uncertain regions in a short time. Through experiments with synthetic objects, we demonstrate that our approach effectively addresses the next best view selection problem, achieving significant improvements in computational efficiency while maintaining high-quality 3D reconstruction. Furthermore, we validate our method on a real robot, showing that it enables full 3D reconstruction of real-world objects.https://www.mdpi.com/2076-3417/15/1/272active 3D reconstructionobject full 3D object reconstructionrobot vision |
| spellingShingle | Youngtaek Hong Jonghyeon Kim Geonho Cha Eunwoo Kim Kyungjae Lee See-Then-Grasp: Object Full 3D Reconstruction via Two-Stage Active Robotic Reconstruction Using Single Manipulator Applied Sciences active 3D reconstruction object full 3D object reconstruction robot vision |
| title | See-Then-Grasp: Object Full 3D Reconstruction via Two-Stage Active Robotic Reconstruction Using Single Manipulator |
| title_full | See-Then-Grasp: Object Full 3D Reconstruction via Two-Stage Active Robotic Reconstruction Using Single Manipulator |
| title_fullStr | See-Then-Grasp: Object Full 3D Reconstruction via Two-Stage Active Robotic Reconstruction Using Single Manipulator |
| title_full_unstemmed | See-Then-Grasp: Object Full 3D Reconstruction via Two-Stage Active Robotic Reconstruction Using Single Manipulator |
| title_short | See-Then-Grasp: Object Full 3D Reconstruction via Two-Stage Active Robotic Reconstruction Using Single Manipulator |
| title_sort | see then grasp object full 3d reconstruction via two stage active robotic reconstruction using single manipulator |
| topic | active 3D reconstruction object full 3D object reconstruction robot vision |
| url | https://www.mdpi.com/2076-3417/15/1/272 |
| work_keys_str_mv | AT youngtaekhong seethengraspobjectfull3dreconstructionviatwostageactiveroboticreconstructionusingsinglemanipulator AT jonghyeonkim seethengraspobjectfull3dreconstructionviatwostageactiveroboticreconstructionusingsinglemanipulator AT geonhocha seethengraspobjectfull3dreconstructionviatwostageactiveroboticreconstructionusingsinglemanipulator AT eunwookim seethengraspobjectfull3dreconstructionviatwostageactiveroboticreconstructionusingsinglemanipulator AT kyungjaelee seethengraspobjectfull3dreconstructionviatwostageactiveroboticreconstructionusingsinglemanipulator |