Multimodal People Re-Identification Using 3D Skeleton, Depth, and Color Information
Person re-identification aims at recognizing the same person across multiple cameras or over different video frames in varying viewpoints, environments, and lighting conditions. Appearance matching in these challenging situations can be complex. In the last few years, thanks to the availability of R...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2024-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10763524/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1846141906717769728 |
|---|---|
| author | Cosimo Patruno Vito Reno Grazia Cicirelli Tiziana D'Orazio |
| author_facet | Cosimo Patruno Vito Reno Grazia Cicirelli Tiziana D'Orazio |
| author_sort | Cosimo Patruno |
| collection | DOAJ |
| description | Person re-identification aims at recognizing the same person across multiple cameras or over different video frames in varying viewpoints, environments, and lighting conditions. Appearance matching in these challenging situations can be complex. In the last few years, thanks to the availability of RGB-D data, appearance matching has been addressed by using 3D body models in addition to the traditional 2D features commonly used in earlier methods. This paper proposes a novel method for person re-identification that builds a color-based signature adapted to the 3D body parts observed by multiple cameras. To be sure of comparing color features coming from the same part of the body, independently from the people's posture and shape, a spatial appearance signature has been built to capture more information from regions likely to contain more informative color content. The proposed method consists of a number of steps. In a preliminary phase, the point cloud related to the person to be recognized is opportunely registered to a standard reference system to keep viewpoint invariance. Then, the skeleton joints of the person enable the clustering of the 3D body reconstruction in different portions. Each part of the person's body can be analyzed using 3D adaptive partition grids of different sizes, and a color-based descriptor can be extracted from each cell to compose the person's signature. The signature is then compared with the other ones in the reference database to perform the person's re-identification. The robustness and effectiveness of the proposed solution have been extensively proven on three publicly available datasets (BIWI-RGBD-ID, KinectREID, RGBD-ID). An improvement of the current state of the art is obtained, achieving recognition rates of 99.6% (BIWI-RGBD-ID), 68.9% (KinectREID), and 92.8% (RGBD-ID), respectively. |
| format | Article |
| id | doaj-art-86bbb067fb8341838095f6bb1e84729f |
| institution | Kabale University |
| issn | 2169-3536 |
| language | English |
| publishDate | 2024-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-86bbb067fb8341838095f6bb1e84729f2024-12-04T00:02:09ZengIEEEIEEE Access2169-35362024-01-011217468917470410.1109/ACCESS.2024.350473810763524Multimodal People Re-Identification Using 3D Skeleton, Depth, and Color InformationCosimo Patruno0https://orcid.org/0000-0001-8624-5444Vito Reno1https://orcid.org/0000-0003-1830-4961Grazia Cicirelli2https://orcid.org/0000-0003-1562-0467Tiziana D'Orazio3https://orcid.org/0000-0003-1473-7110Institute of Intelligent Industrial Technologies and Systems for Advanced Manufacturing, National Research Council of Italy, Bari, ItalyInstitute of Intelligent Industrial Technologies and Systems for Advanced Manufacturing, National Research Council of Italy, Bari, ItalyInstitute of Intelligent Industrial Technologies and Systems for Advanced Manufacturing, National Research Council of Italy, Bari, ItalyInstitute of Intelligent Industrial Technologies and Systems for Advanced Manufacturing, National Research Council of Italy, Bari, ItalyPerson re-identification aims at recognizing the same person across multiple cameras or over different video frames in varying viewpoints, environments, and lighting conditions. Appearance matching in these challenging situations can be complex. In the last few years, thanks to the availability of RGB-D data, appearance matching has been addressed by using 3D body models in addition to the traditional 2D features commonly used in earlier methods. This paper proposes a novel method for person re-identification that builds a color-based signature adapted to the 3D body parts observed by multiple cameras. To be sure of comparing color features coming from the same part of the body, independently from the people's posture and shape, a spatial appearance signature has been built to capture more information from regions likely to contain more informative color content. The proposed method consists of a number of steps. In a preliminary phase, the point cloud related to the person to be recognized is opportunely registered to a standard reference system to keep viewpoint invariance. Then, the skeleton joints of the person enable the clustering of the 3D body reconstruction in different portions. Each part of the person's body can be analyzed using 3D adaptive partition grids of different sizes, and a color-based descriptor can be extracted from each cell to compose the person's signature. The signature is then compared with the other ones in the reference database to perform the person's re-identification. The robustness and effectiveness of the proposed solution have been extensively proven on three publicly available datasets (BIWI-RGBD-ID, KinectREID, RGBD-ID). An improvement of the current state of the art is obtained, achieving recognition rates of 99.6% (BIWI-RGBD-ID), 68.9% (KinectREID), and 92.8% (RGBD-ID), respectively.https://ieeexplore.ieee.org/document/10763524/Person re-identificationcolor-based signatureunevenly-spaced partition gridsRGB-D sensorcolor point cloud |
| spellingShingle | Cosimo Patruno Vito Reno Grazia Cicirelli Tiziana D'Orazio Multimodal People Re-Identification Using 3D Skeleton, Depth, and Color Information IEEE Access Person re-identification color-based signature unevenly-spaced partition grids RGB-D sensor color point cloud |
| title | Multimodal People Re-Identification Using 3D Skeleton, Depth, and Color Information |
| title_full | Multimodal People Re-Identification Using 3D Skeleton, Depth, and Color Information |
| title_fullStr | Multimodal People Re-Identification Using 3D Skeleton, Depth, and Color Information |
| title_full_unstemmed | Multimodal People Re-Identification Using 3D Skeleton, Depth, and Color Information |
| title_short | Multimodal People Re-Identification Using 3D Skeleton, Depth, and Color Information |
| title_sort | multimodal people re identification using 3d skeleton depth and color information |
| topic | Person re-identification color-based signature unevenly-spaced partition grids RGB-D sensor color point cloud |
| url | https://ieeexplore.ieee.org/document/10763524/ |
| work_keys_str_mv | AT cosimopatruno multimodalpeoplereidentificationusing3dskeletondepthandcolorinformation AT vitoreno multimodalpeoplereidentificationusing3dskeletondepthandcolorinformation AT graziacicirelli multimodalpeoplereidentificationusing3dskeletondepthandcolorinformation AT tizianadorazio multimodalpeoplereidentificationusing3dskeletondepthandcolorinformation |