Universal Metrics to Characterize the Performance of Imaging 3-D Measurement Systems With a Focus on Static Indoor Scenes
Imaging 3-D measurement systems differ not only in design, power consumption, and resolution but, most notably, in their measurement performance, which is rarely documented in a reproducible or comparable manner. In this work, we introduce a simple, universal methodology for quantitatively character...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Open Journal of Instrumentation and Measurement |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/11080294/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849228510582800384 |
|---|---|
| author | Bastian Stahl Alexandra Mielke Robert Lange |
| author_facet | Bastian Stahl Alexandra Mielke Robert Lange |
| author_sort | Bastian Stahl |
| collection | DOAJ |
| description | Imaging 3-D measurement systems differ not only in design, power consumption, and resolution but, most notably, in their measurement performance, which is rarely documented in a reproducible or comparable manner. In this work, we introduce a simple, universal methodology for quantitatively characterizing indirect time-of-flight cameras, focusing on point cloud data acquired under reproducible conditions. Our metrics build upon the ISO 5725-1 standard and related research to enable standardized comparisons across different systems, independent of their measurement principles. Beyond trueness, accuracy, and precision, we propose additional metrics to evaluate surface texture quality, addressing visually disruptive systematic measurement errors. We condense these measurements into 14 key metrics for a straightforward camera comparison. In addition, we introduce the energy-density-weighted depth precision index (EPI) as a metric to relate an active measurement system’s performance to the emitted optical power. Finally, we demonstrate the applicability of our methodology by evaluating seven state-of-the-art 3-D systems. |
| format | Article |
| id | doaj-art-66b78f11b015460d93bb09019a4d277e |
| institution | Kabale University |
| issn | 2768-7236 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Open Journal of Instrumentation and Measurement |
| spelling | doaj-art-66b78f11b015460d93bb09019a4d277e2025-08-22T23:17:36ZengIEEEIEEE Open Journal of Instrumentation and Measurement2768-72362025-01-01411110.1109/OJIM.2025.358896211080294Universal Metrics to Characterize the Performance of Imaging 3-D Measurement Systems With a Focus on Static Indoor ScenesBastian Stahl0https://orcid.org/0009-0002-5304-8385Alexandra Mielke1https://orcid.org/0009-0003-1749-8220Robert Lange2https://orcid.org/0000-0002-4567-6907Institute for Safety and Security Research, Hochschule Bonn-Rhein-Sieg University of Applied Sciences, Sankt Augustin, GermanyInstitute for Safety and Security Research, Hochschule Bonn-Rhein-Sieg University of Applied Sciences, Sankt Augustin, GermanyInstitute for Safety and Security Research, Hochschule Bonn-Rhein-Sieg University of Applied Sciences, Sankt Augustin, GermanyImaging 3-D measurement systems differ not only in design, power consumption, and resolution but, most notably, in their measurement performance, which is rarely documented in a reproducible or comparable manner. In this work, we introduce a simple, universal methodology for quantitatively characterizing indirect time-of-flight cameras, focusing on point cloud data acquired under reproducible conditions. Our metrics build upon the ISO 5725-1 standard and related research to enable standardized comparisons across different systems, independent of their measurement principles. Beyond trueness, accuracy, and precision, we propose additional metrics to evaluate surface texture quality, addressing visually disruptive systematic measurement errors. We condense these measurements into 14 key metrics for a straightforward camera comparison. In addition, we introduce the energy-density-weighted depth precision index (EPI) as a metric to relate an active measurement system’s performance to the emitted optical power. Finally, we demonstrate the applicability of our methodology by evaluating seven state-of-the-art 3-D systems.https://ieeexplore.ieee.org/document/11080294/3-D depth sensing camerascharacterizationmeasurement erroroptical powerquantitative benchmarksurface texture |
| spellingShingle | Bastian Stahl Alexandra Mielke Robert Lange Universal Metrics to Characterize the Performance of Imaging 3-D Measurement Systems With a Focus on Static Indoor Scenes IEEE Open Journal of Instrumentation and Measurement 3-D depth sensing cameras characterization measurement error optical power quantitative benchmark surface texture |
| title | Universal Metrics to Characterize the Performance of Imaging 3-D Measurement Systems With a Focus on Static Indoor Scenes |
| title_full | Universal Metrics to Characterize the Performance of Imaging 3-D Measurement Systems With a Focus on Static Indoor Scenes |
| title_fullStr | Universal Metrics to Characterize the Performance of Imaging 3-D Measurement Systems With a Focus on Static Indoor Scenes |
| title_full_unstemmed | Universal Metrics to Characterize the Performance of Imaging 3-D Measurement Systems With a Focus on Static Indoor Scenes |
| title_short | Universal Metrics to Characterize the Performance of Imaging 3-D Measurement Systems With a Focus on Static Indoor Scenes |
| title_sort | universal metrics to characterize the performance of imaging 3 d measurement systems with a focus on static indoor scenes |
| topic | 3-D depth sensing cameras characterization measurement error optical power quantitative benchmark surface texture |
| url | https://ieeexplore.ieee.org/document/11080294/ |
| work_keys_str_mv | AT bastianstahl universalmetricstocharacterizetheperformanceofimaging3dmeasurementsystemswithafocusonstaticindoorscenes AT alexandramielke universalmetricstocharacterizetheperformanceofimaging3dmeasurementsystemswithafocusonstaticindoorscenes AT robertlange universalmetricstocharacterizetheperformanceofimaging3dmeasurementsystemswithafocusonstaticindoorscenes |