An Effective 3D Instance Map Reconstruction Method Based on RGBD Images for Indoor Scene
To enhance the intelligence of robots, constructing accurate object-level instance maps is essential. However, the diversity and clutter of objects in indoor scenes present significant challenges for instance map construction. To tackle this issue, we propose a method for constructing object-level i...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-01-01
|
| Series: | Remote Sensing |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-4292/17/1/139 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841549006380990464 |
|---|---|
| author | Heng Wu Yanjie Liu Chao Wang Yanlong Wei |
| author_facet | Heng Wu Yanjie Liu Chao Wang Yanlong Wei |
| author_sort | Heng Wu |
| collection | DOAJ |
| description | To enhance the intelligence of robots, constructing accurate object-level instance maps is essential. However, the diversity and clutter of objects in indoor scenes present significant challenges for instance map construction. To tackle this issue, we propose a method for constructing object-level instance maps based on RGBD images. First, we utilize the advanced visual odometer ORB-SLAM3 to estimate the poses of image frames and extract keyframes. Next, we perform semantic and geometric segmentation on the color and depth images of these keyframes, respectively, using semantic segmentation to optimize the geometric segmentation results and address inaccuracies in the target segmentation caused by small depth variations. The segmented depth images are then projected into point cloud segments, which are assigned corresponding semantic information. We integrate these point cloud segments into a global voxel map, updating each voxel’s class using color, distance constraints, and Bayesian methods to create an object-level instance map. Finally, we construct an ellipsoids scene from this map to test the robot’s localization capabilities in indoor environments using semantic information. Our experiments demonstrate that this method accurately and robustly constructs the environment, facilitating precise object-level scene segmentation. Furthermore, compared to manually labeled ellipsoidal maps, generating ellipsoidal maps from extracted objects enables accurate global localization. |
| format | Article |
| id | doaj-art-91eb11fdebbd4fcf848d19673f650197 |
| institution | Kabale University |
| issn | 2072-4292 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Remote Sensing |
| spelling | doaj-art-91eb11fdebbd4fcf848d19673f6501972025-01-10T13:20:21ZengMDPI AGRemote Sensing2072-42922025-01-0117113910.3390/rs17010139An Effective 3D Instance Map Reconstruction Method Based on RGBD Images for Indoor SceneHeng Wu0Yanjie Liu1Chao Wang2Yanlong Wei3State Key Laboratory of Robotics and Systems (HIT), School of Mechatronics Engineering, Harbin Institute of Technology, Harbin 150001, ChinaState Key Laboratory of Robotics and Systems (HIT), School of Mechatronics Engineering, Harbin Institute of Technology, Harbin 150001, ChinaState Key Laboratory of Robotics and Systems (HIT), School of Mechatronics Engineering, Harbin Institute of Technology, Harbin 150001, ChinaState Key Laboratory of Robotics and Systems (HIT), School of Mechatronics Engineering, Harbin Institute of Technology, Harbin 150001, ChinaTo enhance the intelligence of robots, constructing accurate object-level instance maps is essential. However, the diversity and clutter of objects in indoor scenes present significant challenges for instance map construction. To tackle this issue, we propose a method for constructing object-level instance maps based on RGBD images. First, we utilize the advanced visual odometer ORB-SLAM3 to estimate the poses of image frames and extract keyframes. Next, we perform semantic and geometric segmentation on the color and depth images of these keyframes, respectively, using semantic segmentation to optimize the geometric segmentation results and address inaccuracies in the target segmentation caused by small depth variations. The segmented depth images are then projected into point cloud segments, which are assigned corresponding semantic information. We integrate these point cloud segments into a global voxel map, updating each voxel’s class using color, distance constraints, and Bayesian methods to create an object-level instance map. Finally, we construct an ellipsoids scene from this map to test the robot’s localization capabilities in indoor environments using semantic information. Our experiments demonstrate that this method accurately and robustly constructs the environment, facilitating precise object-level scene segmentation. Furthermore, compared to manually labeled ellipsoidal maps, generating ellipsoidal maps from extracted objects enables accurate global localization.https://www.mdpi.com/2072-4292/17/1/139semantic segmentationgeometric segmentationBayesian updateobject-level instance mapellipse–ellipsoid localization |
| spellingShingle | Heng Wu Yanjie Liu Chao Wang Yanlong Wei An Effective 3D Instance Map Reconstruction Method Based on RGBD Images for Indoor Scene Remote Sensing semantic segmentation geometric segmentation Bayesian update object-level instance map ellipse–ellipsoid localization |
| title | An Effective 3D Instance Map Reconstruction Method Based on RGBD Images for Indoor Scene |
| title_full | An Effective 3D Instance Map Reconstruction Method Based on RGBD Images for Indoor Scene |
| title_fullStr | An Effective 3D Instance Map Reconstruction Method Based on RGBD Images for Indoor Scene |
| title_full_unstemmed | An Effective 3D Instance Map Reconstruction Method Based on RGBD Images for Indoor Scene |
| title_short | An Effective 3D Instance Map Reconstruction Method Based on RGBD Images for Indoor Scene |
| title_sort | effective 3d instance map reconstruction method based on rgbd images for indoor scene |
| topic | semantic segmentation geometric segmentation Bayesian update object-level instance map ellipse–ellipsoid localization |
| url | https://www.mdpi.com/2072-4292/17/1/139 |
| work_keys_str_mv | AT hengwu aneffective3dinstancemapreconstructionmethodbasedonrgbdimagesforindoorscene AT yanjieliu aneffective3dinstancemapreconstructionmethodbasedonrgbdimagesforindoorscene AT chaowang aneffective3dinstancemapreconstructionmethodbasedonrgbdimagesforindoorscene AT yanlongwei aneffective3dinstancemapreconstructionmethodbasedonrgbdimagesforindoorscene AT hengwu effective3dinstancemapreconstructionmethodbasedonrgbdimagesforindoorscene AT yanjieliu effective3dinstancemapreconstructionmethodbasedonrgbdimagesforindoorscene AT chaowang effective3dinstancemapreconstructionmethodbasedonrgbdimagesforindoorscene AT yanlongwei effective3dinstancemapreconstructionmethodbasedonrgbdimagesforindoorscene |