A Novel Panorama Depth Estimation Framework for Autonomous Driving Scenarios Based on a Vision Transformer
An accurate panorama depth estimation result is crucial to risk perception in autonomous driving practice. In this paper, an innovative framework is presented to address the challenges of imperfect observation and projection fusion in panorama depth estimation, enabling the accurate capture of dista...
Saved in:
Main Authors: | , , , , , |
---|---|
Format: | Article |
Language: | English |
Published: |
MDPI AG
2024-10-01
|
Series: | Sensors |
Subjects: | |
Online Access: | https://www.mdpi.com/1424-8220/24/21/7013 |
Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
_version_ | 1846173137836703744 |
---|---|
author | Yuqi Zhang Liang Chu Zixu Wang He Tong Jincheng Hu Jihao Li |
author_facet | Yuqi Zhang Liang Chu Zixu Wang He Tong Jincheng Hu Jihao Li |
author_sort | Yuqi Zhang |
collection | DOAJ |
description | An accurate panorama depth estimation result is crucial to risk perception in autonomous driving practice. In this paper, an innovative framework is presented to address the challenges of imperfect observation and projection fusion in panorama depth estimation, enabling the accurate capture of distances from surrounding images in driving scenarios. First, the Patch Filling method is proposed to alleviate the imperfect observation of panoramic depth in autonomous driving scenarios, which constructs a panoramic depth map based on the sparse distance data provided by the 3D point cloud. Then, in order to tackle the distortion challenge faced by outdoor panoramic images, a method for image context learning, ViT-Fuse, is proposed and specifically designed for equirectangular panoramic views. The experimental results show that the proposed ViT-Fuse reduces the estimation error by 9.15% on average in driving scenarios compared with the basic method and exhibits more robust and smoother results on the edge details of the depth estimation maps. |
format | Article |
id | doaj-art-f202d7f57c8f426ca0b74403840ed5f3 |
institution | Kabale University |
issn | 1424-8220 |
language | English |
publishDate | 2024-10-01 |
publisher | MDPI AG |
record_format | Article |
series | Sensors |
spelling | doaj-art-f202d7f57c8f426ca0b74403840ed5f32024-11-08T14:41:51ZengMDPI AGSensors1424-82202024-10-012421701310.3390/s24217013A Novel Panorama Depth Estimation Framework for Autonomous Driving Scenarios Based on a Vision TransformerYuqi Zhang0Liang Chu1Zixu Wang2He Tong3Jincheng Hu4Jihao Li5College of Automotive Engineering, Jilin University, Changchun 130022, ChinaCollege of Automotive Engineering, Jilin University, Changchun 130022, ChinaCollege of Automotive Engineering, Jilin University, Changchun 130022, ChinaCollege of Automotive Engineering, Jilin University, Changchun 130022, ChinaDepartment of Aeronautical and Automotive Engineering, Loughborough University, Loughborough LE11 3TU, UKDepartment of Aeronautical and Automotive Engineering, Loughborough University, Loughborough LE11 3TU, UKAn accurate panorama depth estimation result is crucial to risk perception in autonomous driving practice. In this paper, an innovative framework is presented to address the challenges of imperfect observation and projection fusion in panorama depth estimation, enabling the accurate capture of distances from surrounding images in driving scenarios. First, the Patch Filling method is proposed to alleviate the imperfect observation of panoramic depth in autonomous driving scenarios, which constructs a panoramic depth map based on the sparse distance data provided by the 3D point cloud. Then, in order to tackle the distortion challenge faced by outdoor panoramic images, a method for image context learning, ViT-Fuse, is proposed and specifically designed for equirectangular panoramic views. The experimental results show that the proposed ViT-Fuse reduces the estimation error by 9.15% on average in driving scenarios compared with the basic method and exhibits more robust and smoother results on the edge details of the depth estimation maps.https://www.mdpi.com/1424-8220/24/21/7013panorama depth estimationautonomous drivingdeep learningfeature fusionintelligent vehicle |
spellingShingle | Yuqi Zhang Liang Chu Zixu Wang He Tong Jincheng Hu Jihao Li A Novel Panorama Depth Estimation Framework for Autonomous Driving Scenarios Based on a Vision Transformer Sensors panorama depth estimation autonomous driving deep learning feature fusion intelligent vehicle |
title | A Novel Panorama Depth Estimation Framework for Autonomous Driving Scenarios Based on a Vision Transformer |
title_full | A Novel Panorama Depth Estimation Framework for Autonomous Driving Scenarios Based on a Vision Transformer |
title_fullStr | A Novel Panorama Depth Estimation Framework for Autonomous Driving Scenarios Based on a Vision Transformer |
title_full_unstemmed | A Novel Panorama Depth Estimation Framework for Autonomous Driving Scenarios Based on a Vision Transformer |
title_short | A Novel Panorama Depth Estimation Framework for Autonomous Driving Scenarios Based on a Vision Transformer |
title_sort | novel panorama depth estimation framework for autonomous driving scenarios based on a vision transformer |
topic | panorama depth estimation autonomous driving deep learning feature fusion intelligent vehicle |
url | https://www.mdpi.com/1424-8220/24/21/7013 |
work_keys_str_mv | AT yuqizhang anovelpanoramadepthestimationframeworkforautonomousdrivingscenariosbasedonavisiontransformer AT liangchu anovelpanoramadepthestimationframeworkforautonomousdrivingscenariosbasedonavisiontransformer AT zixuwang anovelpanoramadepthestimationframeworkforautonomousdrivingscenariosbasedonavisiontransformer AT hetong anovelpanoramadepthestimationframeworkforautonomousdrivingscenariosbasedonavisiontransformer AT jinchenghu anovelpanoramadepthestimationframeworkforautonomousdrivingscenariosbasedonavisiontransformer AT jihaoli anovelpanoramadepthestimationframeworkforautonomousdrivingscenariosbasedonavisiontransformer AT yuqizhang novelpanoramadepthestimationframeworkforautonomousdrivingscenariosbasedonavisiontransformer AT liangchu novelpanoramadepthestimationframeworkforautonomousdrivingscenariosbasedonavisiontransformer AT zixuwang novelpanoramadepthestimationframeworkforautonomousdrivingscenariosbasedonavisiontransformer AT hetong novelpanoramadepthestimationframeworkforautonomousdrivingscenariosbasedonavisiontransformer AT jinchenghu novelpanoramadepthestimationframeworkforautonomousdrivingscenariosbasedonavisiontransformer AT jihaoli novelpanoramadepthestimationframeworkforautonomousdrivingscenariosbasedonavisiontransformer |