A Novel Panorama Depth Estimation Framework for Autonomous Driving Scenarios Based on a Vision Transformer

An accurate panorama depth estimation result is crucial to risk perception in autonomous driving practice. In this paper, an innovative framework is presented to address the challenges of imperfect observation and projection fusion in panorama depth estimation, enabling the accurate capture of dista...

Full description

Saved in:
Bibliographic Details
Main Authors: Yuqi Zhang, Liang Chu, Zixu Wang, He Tong, Jincheng Hu, Jihao Li
Format: Article
Language:English
Published: MDPI AG 2024-10-01
Series:Sensors
Subjects:
Online Access:https://www.mdpi.com/1424-8220/24/21/7013
Tags: Add Tag
No Tags, Be the first to tag this record!
_version_ 1846173137836703744
author Yuqi Zhang
Liang Chu
Zixu Wang
He Tong
Jincheng Hu
Jihao Li
author_facet Yuqi Zhang
Liang Chu
Zixu Wang
He Tong
Jincheng Hu
Jihao Li
author_sort Yuqi Zhang
collection DOAJ
description An accurate panorama depth estimation result is crucial to risk perception in autonomous driving practice. In this paper, an innovative framework is presented to address the challenges of imperfect observation and projection fusion in panorama depth estimation, enabling the accurate capture of distances from surrounding images in driving scenarios. First, the Patch Filling method is proposed to alleviate the imperfect observation of panoramic depth in autonomous driving scenarios, which constructs a panoramic depth map based on the sparse distance data provided by the 3D point cloud. Then, in order to tackle the distortion challenge faced by outdoor panoramic images, a method for image context learning, ViT-Fuse, is proposed and specifically designed for equirectangular panoramic views. The experimental results show that the proposed ViT-Fuse reduces the estimation error by 9.15% on average in driving scenarios compared with the basic method and exhibits more robust and smoother results on the edge details of the depth estimation maps.
format Article
id doaj-art-f202d7f57c8f426ca0b74403840ed5f3
institution Kabale University
issn 1424-8220
language English
publishDate 2024-10-01
publisher MDPI AG
record_format Article
series Sensors
spelling doaj-art-f202d7f57c8f426ca0b74403840ed5f32024-11-08T14:41:51ZengMDPI AGSensors1424-82202024-10-012421701310.3390/s24217013A Novel Panorama Depth Estimation Framework for Autonomous Driving Scenarios Based on a Vision TransformerYuqi Zhang0Liang Chu1Zixu Wang2He Tong3Jincheng Hu4Jihao Li5College of Automotive Engineering, Jilin University, Changchun 130022, ChinaCollege of Automotive Engineering, Jilin University, Changchun 130022, ChinaCollege of Automotive Engineering, Jilin University, Changchun 130022, ChinaCollege of Automotive Engineering, Jilin University, Changchun 130022, ChinaDepartment of Aeronautical and Automotive Engineering, Loughborough University, Loughborough LE11 3TU, UKDepartment of Aeronautical and Automotive Engineering, Loughborough University, Loughborough LE11 3TU, UKAn accurate panorama depth estimation result is crucial to risk perception in autonomous driving practice. In this paper, an innovative framework is presented to address the challenges of imperfect observation and projection fusion in panorama depth estimation, enabling the accurate capture of distances from surrounding images in driving scenarios. First, the Patch Filling method is proposed to alleviate the imperfect observation of panoramic depth in autonomous driving scenarios, which constructs a panoramic depth map based on the sparse distance data provided by the 3D point cloud. Then, in order to tackle the distortion challenge faced by outdoor panoramic images, a method for image context learning, ViT-Fuse, is proposed and specifically designed for equirectangular panoramic views. The experimental results show that the proposed ViT-Fuse reduces the estimation error by 9.15% on average in driving scenarios compared with the basic method and exhibits more robust and smoother results on the edge details of the depth estimation maps.https://www.mdpi.com/1424-8220/24/21/7013panorama depth estimationautonomous drivingdeep learningfeature fusionintelligent vehicle
spellingShingle Yuqi Zhang
Liang Chu
Zixu Wang
He Tong
Jincheng Hu
Jihao Li
A Novel Panorama Depth Estimation Framework for Autonomous Driving Scenarios Based on a Vision Transformer
Sensors
panorama depth estimation
autonomous driving
deep learning
feature fusion
intelligent vehicle
title A Novel Panorama Depth Estimation Framework for Autonomous Driving Scenarios Based on a Vision Transformer
title_full A Novel Panorama Depth Estimation Framework for Autonomous Driving Scenarios Based on a Vision Transformer
title_fullStr A Novel Panorama Depth Estimation Framework for Autonomous Driving Scenarios Based on a Vision Transformer
title_full_unstemmed A Novel Panorama Depth Estimation Framework for Autonomous Driving Scenarios Based on a Vision Transformer
title_short A Novel Panorama Depth Estimation Framework for Autonomous Driving Scenarios Based on a Vision Transformer
title_sort novel panorama depth estimation framework for autonomous driving scenarios based on a vision transformer
topic panorama depth estimation
autonomous driving
deep learning
feature fusion
intelligent vehicle
url https://www.mdpi.com/1424-8220/24/21/7013
work_keys_str_mv AT yuqizhang anovelpanoramadepthestimationframeworkforautonomousdrivingscenariosbasedonavisiontransformer
AT liangchu anovelpanoramadepthestimationframeworkforautonomousdrivingscenariosbasedonavisiontransformer
AT zixuwang anovelpanoramadepthestimationframeworkforautonomousdrivingscenariosbasedonavisiontransformer
AT hetong anovelpanoramadepthestimationframeworkforautonomousdrivingscenariosbasedonavisiontransformer
AT jinchenghu anovelpanoramadepthestimationframeworkforautonomousdrivingscenariosbasedonavisiontransformer
AT jihaoli anovelpanoramadepthestimationframeworkforautonomousdrivingscenariosbasedonavisiontransformer
AT yuqizhang novelpanoramadepthestimationframeworkforautonomousdrivingscenariosbasedonavisiontransformer
AT liangchu novelpanoramadepthestimationframeworkforautonomousdrivingscenariosbasedonavisiontransformer
AT zixuwang novelpanoramadepthestimationframeworkforautonomousdrivingscenariosbasedonavisiontransformer
AT hetong novelpanoramadepthestimationframeworkforautonomousdrivingscenariosbasedonavisiontransformer
AT jinchenghu novelpanoramadepthestimationframeworkforautonomousdrivingscenariosbasedonavisiontransformer
AT jihaoli novelpanoramadepthestimationframeworkforautonomousdrivingscenariosbasedonavisiontransformer