Fully Synthetic Pedestrian Anomaly Behavior Dataset Generation in Metaverse for Enhancing Autonomous Driving Object Detection
Pedestrian detection is fundamental in the realm of autonomous driving, relying on comprehensive datasets for accurate deep-learning methods. Detecting diverse pedestrian behaviors, including rare and near-accident cases, is necessary, however, there were limitations in the development of such speci...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2024-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10749804/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1846163808410664960 |
|---|---|
| author | Nang Htet Htet Aung Paramin Sangwongngam Rungroj Jintamethasawat Lunchakorn Wuttisittikulkij |
| author_facet | Nang Htet Htet Aung Paramin Sangwongngam Rungroj Jintamethasawat Lunchakorn Wuttisittikulkij |
| author_sort | Nang Htet Htet Aung |
| collection | DOAJ |
| description | Pedestrian detection is fundamental in the realm of autonomous driving, relying on comprehensive datasets for accurate deep-learning methods. Detecting diverse pedestrian behaviors, including rare and near-accident cases, is necessary, however, there were limitations in the development of such specific datasets. Furthermore, involving humans in data collection is risky and raises privacy concerns since recording and storing data necessitates explicit consent from all participants. To tackle this challenge, we propose MetaPed, a synthetic pedestrian dataset generated from the Metaverse, where the avatars can represent a wide range of pedestrian behaviors without exposing real individuals to potential harm or privacy violations. Our Metaverse includes both programmed pedestrian behaviors and controllable avatar movements and interactions, ensuring a comprehensive and realistic representation of pedestrian actions. To validate and ensure the generalization capabilities of our synthetic dataset’s performance in real-world scenarios, we perform both cross-dataset and intra-dataset evaluations on comprehensive real-world datasets, including KITTI, Citypersons, and INRIA Person. Our experiments demonstrate that the network trained on our synthetic dataset exhibits robust generalization capabilities for unseen real-world situations and our dataset’s inclusion significantly enhanced the Average Precision (AP) and Average Recall (AR) of the pedestrian detection model. For cross-dataset validation, we generally observe improvements in both AP and AR with models trained on all datasets. The inclusion of our dataset also improves the intra-dataset validation of the INRIA Person dataset, where the AP and AR values increase from 0.958 to 0.988 and AR from 0.697 to 0.725, respectively. For Citypersons and KITTI datasets, the AP values increase from 0.630 to 0.639 and 0.745 to 0.752, respectively. The model training on our dataset achieved the highest AP and AR values across all test datasets and the second-highest AP observed when tested on Citypersons. |
| format | Article |
| id | doaj-art-05f3c3ea6bda409888f13f141df0fdfe |
| institution | Kabale University |
| issn | 2169-3536 |
| language | English |
| publishDate | 2024-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-05f3c3ea6bda409888f13f141df0fdfe2024-11-19T00:01:18ZengIEEEIEEE Access2169-35362024-01-011216663016664210.1109/ACCESS.2024.349550510749804Fully Synthetic Pedestrian Anomaly Behavior Dataset Generation in Metaverse for Enhancing Autonomous Driving Object DetectionNang Htet Htet Aung0https://orcid.org/0000-0001-8601-4443Paramin Sangwongngam1https://orcid.org/0000-0002-5097-6891Rungroj Jintamethasawat2Lunchakorn Wuttisittikulkij3https://orcid.org/0000-0002-3033-3020Wireless Communication Ecosystem Research Unit, Faculty of Engineering, Chulalongkorn University, Bangkok, ThailandNational Electronics and Computer Technology Center (NECTEC), National Science and Technology Development Agency (NSTDA), Khlong Luang, Pathum Thani, ThailandNational Electronics and Computer Technology Center (NECTEC), National Science and Technology Development Agency (NSTDA), Khlong Luang, Pathum Thani, ThailandWireless Communication Ecosystem Research Unit, Faculty of Engineering, Chulalongkorn University, Bangkok, ThailandPedestrian detection is fundamental in the realm of autonomous driving, relying on comprehensive datasets for accurate deep-learning methods. Detecting diverse pedestrian behaviors, including rare and near-accident cases, is necessary, however, there were limitations in the development of such specific datasets. Furthermore, involving humans in data collection is risky and raises privacy concerns since recording and storing data necessitates explicit consent from all participants. To tackle this challenge, we propose MetaPed, a synthetic pedestrian dataset generated from the Metaverse, where the avatars can represent a wide range of pedestrian behaviors without exposing real individuals to potential harm or privacy violations. Our Metaverse includes both programmed pedestrian behaviors and controllable avatar movements and interactions, ensuring a comprehensive and realistic representation of pedestrian actions. To validate and ensure the generalization capabilities of our synthetic dataset’s performance in real-world scenarios, we perform both cross-dataset and intra-dataset evaluations on comprehensive real-world datasets, including KITTI, Citypersons, and INRIA Person. Our experiments demonstrate that the network trained on our synthetic dataset exhibits robust generalization capabilities for unseen real-world situations and our dataset’s inclusion significantly enhanced the Average Precision (AP) and Average Recall (AR) of the pedestrian detection model. For cross-dataset validation, we generally observe improvements in both AP and AR with models trained on all datasets. The inclusion of our dataset also improves the intra-dataset validation of the INRIA Person dataset, where the AP and AR values increase from 0.958 to 0.988 and AR from 0.697 to 0.725, respectively. For Citypersons and KITTI datasets, the AP values increase from 0.630 to 0.639 and 0.745 to 0.752, respectively. The model training on our dataset achieved the highest AP and AR values across all test datasets and the second-highest AP observed when tested on Citypersons.https://ieeexplore.ieee.org/document/10749804/Autonomous vehicles (AVs)deep learningpedestrian detectionpedestrian anomaly behaviorssynthetic datasetmetaverse |
| spellingShingle | Nang Htet Htet Aung Paramin Sangwongngam Rungroj Jintamethasawat Lunchakorn Wuttisittikulkij Fully Synthetic Pedestrian Anomaly Behavior Dataset Generation in Metaverse for Enhancing Autonomous Driving Object Detection IEEE Access Autonomous vehicles (AVs) deep learning pedestrian detection pedestrian anomaly behaviors synthetic dataset metaverse |
| title | Fully Synthetic Pedestrian Anomaly Behavior Dataset Generation in Metaverse for Enhancing Autonomous Driving Object Detection |
| title_full | Fully Synthetic Pedestrian Anomaly Behavior Dataset Generation in Metaverse for Enhancing Autonomous Driving Object Detection |
| title_fullStr | Fully Synthetic Pedestrian Anomaly Behavior Dataset Generation in Metaverse for Enhancing Autonomous Driving Object Detection |
| title_full_unstemmed | Fully Synthetic Pedestrian Anomaly Behavior Dataset Generation in Metaverse for Enhancing Autonomous Driving Object Detection |
| title_short | Fully Synthetic Pedestrian Anomaly Behavior Dataset Generation in Metaverse for Enhancing Autonomous Driving Object Detection |
| title_sort | fully synthetic pedestrian anomaly behavior dataset generation in metaverse for enhancing autonomous driving object detection |
| topic | Autonomous vehicles (AVs) deep learning pedestrian detection pedestrian anomaly behaviors synthetic dataset metaverse |
| url | https://ieeexplore.ieee.org/document/10749804/ |
| work_keys_str_mv | AT nanghtethtetaung fullysyntheticpedestriananomalybehaviordatasetgenerationinmetaverseforenhancingautonomousdrivingobjectdetection AT paraminsangwongngam fullysyntheticpedestriananomalybehaviordatasetgenerationinmetaverseforenhancingautonomousdrivingobjectdetection AT rungrojjintamethasawat fullysyntheticpedestriananomalybehaviordatasetgenerationinmetaverseforenhancingautonomousdrivingobjectdetection AT lunchakornwuttisittikulkij fullysyntheticpedestriananomalybehaviordatasetgenerationinmetaverseforenhancingautonomousdrivingobjectdetection |