Sequential selection and calibration of video frames for 3D outdoor scene reconstruction
Abstract 3D scene understanding and reconstruction aims to obtain a concise scene representation from images and reconstruct the complete scene, including the scene layout, objects bounding boxes and shapes. Existing holistic scene understanding methods primarily recover scenes from single images, w...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2024-12-01
|
| Series: | CAAI Transactions on Intelligence Technology |
| Subjects: | |
| Online Access: | https://doi.org/10.1049/cit2.12338 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841543339181080576 |
|---|---|
| author | Weilin Sun Manyi Li Peng Li Xiao Cao Xiangxu Meng Lei Meng |
| author_facet | Weilin Sun Manyi Li Peng Li Xiao Cao Xiangxu Meng Lei Meng |
| author_sort | Weilin Sun |
| collection | DOAJ |
| description | Abstract 3D scene understanding and reconstruction aims to obtain a concise scene representation from images and reconstruct the complete scene, including the scene layout, objects bounding boxes and shapes. Existing holistic scene understanding methods primarily recover scenes from single images, with a focus on indoor scenes. Due to the complexity of real‐world, the information provided by a single image is limited, resulting in issues such as object occlusion and omission. Furthermore, captured data from outdoor scenes exhibits characteristics of sparsity, strong temporal dependencies and a lack of annotations. Consequently, the task of understanding and reconstructing outdoor scenes is highly challenging. The authors propose a sparse multi‐view images‐based 3D scene reconstruction framework (SMSR). It divides the scene reconstruction task into three stages: initial prediction, refinement, and fusion stage. The first two stages extract 3D scene representations from each viewpoint, while the final stage involves selection, calibration and fusion of object positions and orientations across different viewpoints. SMSR effectively address the issue of object omission by utilizing small‐scale sequential scene information. Experimental results on the general outdoor scene dataset UrbanScene3D‐Art Sci and our proprietary dataset Software College Aerial Time‐series Images, demonstrate that SMSR achieves superior performance in the scene understanding and reconstruction. |
| format | Article |
| id | doaj-art-cef229da1027460bada2c193120e41c1 |
| institution | Kabale University |
| issn | 2468-2322 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | Wiley |
| record_format | Article |
| series | CAAI Transactions on Intelligence Technology |
| spelling | doaj-art-cef229da1027460bada2c193120e41c12025-01-13T14:05:51ZengWileyCAAI Transactions on Intelligence Technology2468-23222024-12-01961500151410.1049/cit2.12338Sequential selection and calibration of video frames for 3D outdoor scene reconstructionWeilin Sun0Manyi Li1Peng Li2Xiao Cao3Xiangxu Meng4Lei Meng5Shandong University Jinan Shandong ChinaShandong University Jinan Shandong ChinaBeijing HQYJ Technology Development Co., LTD Jinan Shandong ChinaNational University of Singapore Singapore SingaporeShandong University Jinan Shandong ChinaShandong University Jinan Shandong ChinaAbstract 3D scene understanding and reconstruction aims to obtain a concise scene representation from images and reconstruct the complete scene, including the scene layout, objects bounding boxes and shapes. Existing holistic scene understanding methods primarily recover scenes from single images, with a focus on indoor scenes. Due to the complexity of real‐world, the information provided by a single image is limited, resulting in issues such as object occlusion and omission. Furthermore, captured data from outdoor scenes exhibits characteristics of sparsity, strong temporal dependencies and a lack of annotations. Consequently, the task of understanding and reconstructing outdoor scenes is highly challenging. The authors propose a sparse multi‐view images‐based 3D scene reconstruction framework (SMSR). It divides the scene reconstruction task into three stages: initial prediction, refinement, and fusion stage. The first two stages extract 3D scene representations from each viewpoint, while the final stage involves selection, calibration and fusion of object positions and orientations across different viewpoints. SMSR effectively address the issue of object omission by utilizing small‐scale sequential scene information. Experimental results on the general outdoor scene dataset UrbanScene3D‐Art Sci and our proprietary dataset Software College Aerial Time‐series Images, demonstrate that SMSR achieves superior performance in the scene understanding and reconstruction.https://doi.org/10.1049/cit2.123383D outdoor scene reconstruction3D scene understandingmulti‐view fusion |
| spellingShingle | Weilin Sun Manyi Li Peng Li Xiao Cao Xiangxu Meng Lei Meng Sequential selection and calibration of video frames for 3D outdoor scene reconstruction CAAI Transactions on Intelligence Technology 3D outdoor scene reconstruction 3D scene understanding multi‐view fusion |
| title | Sequential selection and calibration of video frames for 3D outdoor scene reconstruction |
| title_full | Sequential selection and calibration of video frames for 3D outdoor scene reconstruction |
| title_fullStr | Sequential selection and calibration of video frames for 3D outdoor scene reconstruction |
| title_full_unstemmed | Sequential selection and calibration of video frames for 3D outdoor scene reconstruction |
| title_short | Sequential selection and calibration of video frames for 3D outdoor scene reconstruction |
| title_sort | sequential selection and calibration of video frames for 3d outdoor scene reconstruction |
| topic | 3D outdoor scene reconstruction 3D scene understanding multi‐view fusion |
| url | https://doi.org/10.1049/cit2.12338 |
| work_keys_str_mv | AT weilinsun sequentialselectionandcalibrationofvideoframesfor3doutdoorscenereconstruction AT manyili sequentialselectionandcalibrationofvideoframesfor3doutdoorscenereconstruction AT pengli sequentialselectionandcalibrationofvideoframesfor3doutdoorscenereconstruction AT xiaocao sequentialselectionandcalibrationofvideoframesfor3doutdoorscenereconstruction AT xiangxumeng sequentialselectionandcalibrationofvideoframesfor3doutdoorscenereconstruction AT leimeng sequentialselectionandcalibrationofvideoframesfor3doutdoorscenereconstruction |