Specification-Based Testing of the Image-Recognition Performance of Automated Driving Systems
Automated driving systems (ADSs) are complex entities comprising numerous components, and traditional testing methods often struggle to ensure their safety, primarily due to the diversity driving environments. Interestingly, deep neural networks (DNNs) have proven effective for object detection in t...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10795175/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841542557071310848 |
|---|---|
| author | Kento Tanaka Toshiaki Aoki Takashi Tomita Daisuke Kawakami Nobuo Chida |
| author_facet | Kento Tanaka Toshiaki Aoki Takashi Tomita Daisuke Kawakami Nobuo Chida |
| author_sort | Kento Tanaka |
| collection | DOAJ |
| description | Automated driving systems (ADSs) are complex entities comprising numerous components, and traditional testing methods often struggle to ensure their safety, primarily due to the diversity driving environments. Interestingly, deep neural networks (DNNs) have proven effective for object detection in these settings. The safety of object detection in ADSs depends on the position of the detected objects and the specifications that guide the system’s response to them. Consequently, testing the object-detection process in ADSs must be grounded in these specifications. However, current specifications are informal regarding object locations and inadequate for object-detection testing. To address this issue, this article first introduces the bounding box specification language (BBSL), a framework capable of mathematically articulating the specifications for object and event detection and responses. Subsequently, we propose a specification-based testing approach for the object-detection process in ADS using BBSL. Remarkably, BBSL can formally delineate the positions of objects within the driving environment. Furthermore, our proposed approach can identify safety-critical defects that conventional tests, which focus solely on performance evaluation, might overlook. Furthermore, we propose two sets of test criteria. The first set reflects the diversity of object positions and sizes within an image, while the second set includes coverage metrics that determine whether the test cases cover all conditions outlined by the BBSL specifications. Overall, our contributions facilitate the implementation of specification-based testing for object-detection systems using DNNs, a challenge previously considered formidable. |
| format | Article |
| id | doaj-art-72ee67c334534c1a805b86c7b235ab6e |
| institution | Kabale University |
| issn | 2169-3536 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-72ee67c334534c1a805b86c7b235ab6e2025-01-14T00:02:15ZengIEEEIEEE Access2169-35362025-01-01136321634910.1109/ACCESS.2024.351608210795175Specification-Based Testing of the Image-Recognition Performance of Automated Driving SystemsKento Tanaka0https://orcid.org/0000-0002-3532-6954Toshiaki Aoki1Takashi Tomita2https://orcid.org/0000-0003-1249-7862Daisuke Kawakami3Nobuo Chida4Japan Advanced Institute of Science and Technology, Nomi, Ishikawa, JapanJapan Advanced Institute of Science and Technology, Nomi, Ishikawa, JapanJapan Advanced Institute of Science and Technology, Nomi, Ishikawa, JapanCommunication Systems Engineering Center, Mitsubishi Electric Corporation, Chiyoda, Tokyo, JapanAdvanced Technology Research and Development Center, Mitsubishi Electric Corporation, Amagasaki, Hyogo, JapanAutomated driving systems (ADSs) are complex entities comprising numerous components, and traditional testing methods often struggle to ensure their safety, primarily due to the diversity driving environments. Interestingly, deep neural networks (DNNs) have proven effective for object detection in these settings. The safety of object detection in ADSs depends on the position of the detected objects and the specifications that guide the system’s response to them. Consequently, testing the object-detection process in ADSs must be grounded in these specifications. However, current specifications are informal regarding object locations and inadequate for object-detection testing. To address this issue, this article first introduces the bounding box specification language (BBSL), a framework capable of mathematically articulating the specifications for object and event detection and responses. Subsequently, we propose a specification-based testing approach for the object-detection process in ADS using BBSL. Remarkably, BBSL can formally delineate the positions of objects within the driving environment. Furthermore, our proposed approach can identify safety-critical defects that conventional tests, which focus solely on performance evaluation, might overlook. Furthermore, we propose two sets of test criteria. The first set reflects the diversity of object positions and sizes within an image, while the second set includes coverage metrics that determine whether the test cases cover all conditions outlined by the BBSL specifications. Overall, our contributions facilitate the implementation of specification-based testing for object-detection systems using DNNs, a challenge previously considered formidable.https://ieeexplore.ieee.org/document/10795175/Automated drivingcoverageformal specificationobject detectiontesting |
| spellingShingle | Kento Tanaka Toshiaki Aoki Takashi Tomita Daisuke Kawakami Nobuo Chida Specification-Based Testing of the Image-Recognition Performance of Automated Driving Systems IEEE Access Automated driving coverage formal specification object detection testing |
| title | Specification-Based Testing of the Image-Recognition Performance of Automated Driving Systems |
| title_full | Specification-Based Testing of the Image-Recognition Performance of Automated Driving Systems |
| title_fullStr | Specification-Based Testing of the Image-Recognition Performance of Automated Driving Systems |
| title_full_unstemmed | Specification-Based Testing of the Image-Recognition Performance of Automated Driving Systems |
| title_short | Specification-Based Testing of the Image-Recognition Performance of Automated Driving Systems |
| title_sort | specification based testing of the image recognition performance of automated driving systems |
| topic | Automated driving coverage formal specification object detection testing |
| url | https://ieeexplore.ieee.org/document/10795175/ |
| work_keys_str_mv | AT kentotanaka specificationbasedtestingoftheimagerecognitionperformanceofautomateddrivingsystems AT toshiakiaoki specificationbasedtestingoftheimagerecognitionperformanceofautomateddrivingsystems AT takashitomita specificationbasedtestingoftheimagerecognitionperformanceofautomateddrivingsystems AT daisukekawakami specificationbasedtestingoftheimagerecognitionperformanceofautomateddrivingsystems AT nobuochida specificationbasedtestingoftheimagerecognitionperformanceofautomateddrivingsystems |