A Multispectral Automated Transfer Technique (MATT) for Machine-Driven Image Labeling Utilizing the Segment Anything Model (SAM)
Segment Anything Model (SAM) is drastically accelerating the speed and accuracy of automatically segmenting and labeling large Red-Green-Blue (RGB) imagery datasets. However, SAM is unable to segment and label images outside of the visible light spectrum, for example, for multispectral or hyperspect...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10815733/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841550837860532224 |
|---|---|
| author | James E. Gallagher Aryav Gogia Edward J. Oughton |
| author_facet | James E. Gallagher Aryav Gogia Edward J. Oughton |
| author_sort | James E. Gallagher |
| collection | DOAJ |
| description | Segment Anything Model (SAM) is drastically accelerating the speed and accuracy of automatically segmenting and labeling large Red-Green-Blue (RGB) imagery datasets. However, SAM is unable to segment and label images outside of the visible light spectrum, for example, for multispectral or hyperspectral imagery. Therefore, this paper outlines a method we call the Multispectral Automated Transfer Technique (MATT). By transposing SAM segmentation masks from RGB images we can automatically segment and label multispectral imagery with high precision and efficiency. For example, the results demonstrate that segmenting and labeling a 2,400-image dataset utilizing MATT achieves a time reduction of 87.8% in developing a trained model, reducing roughly 20 hours of manual labeling, to only 2.4 hours. This efficiency gain is associated with only a 6.7% decrease in overall mean average precision (mAP) when training multispectral models via MATT, compared to a manually labeled dataset. We consider this an acceptable level of precision loss when considering the time saved during training, especially for rapidly prototyping experimental modeling methods. This research greatly contributes to the study of multispectral object detection by providing a novel and open-source method to rapidly segment, label, and train multispectral object detection models with minimal human interaction. Future research needs to focus on applying these methods to 1) space-based multispectral; and 2) drone-based hyperspectral imagery. |
| format | Article |
| id | doaj-art-4e2fcf0fa7a3445ea6161f3414f0835a |
| institution | Kabale University |
| issn | 2169-3536 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-4e2fcf0fa7a3445ea6161f3414f0835a2025-01-10T00:01:04ZengIEEEIEEE Access2169-35362025-01-01134499451610.1109/ACCESS.2024.352223610815733A Multispectral Automated Transfer Technique (MATT) for Machine-Driven Image Labeling Utilizing the Segment Anything Model (SAM)James E. Gallagher0https://orcid.org/0009-0002-5380-8436Aryav Gogia1https://orcid.org/0009-0007-9888-3946Edward J. Oughton2https://orcid.org/0000-0002-2766-008XDepartment of Geography and Geoinformation Science, George Mason University, Fairfax, VA, USADepartment of Geography and Geoinformation Science, George Mason University, Fairfax, VA, USADepartment of Geography and Geoinformation Science, George Mason University, Fairfax, VA, USASegment Anything Model (SAM) is drastically accelerating the speed and accuracy of automatically segmenting and labeling large Red-Green-Blue (RGB) imagery datasets. However, SAM is unable to segment and label images outside of the visible light spectrum, for example, for multispectral or hyperspectral imagery. Therefore, this paper outlines a method we call the Multispectral Automated Transfer Technique (MATT). By transposing SAM segmentation masks from RGB images we can automatically segment and label multispectral imagery with high precision and efficiency. For example, the results demonstrate that segmenting and labeling a 2,400-image dataset utilizing MATT achieves a time reduction of 87.8% in developing a trained model, reducing roughly 20 hours of manual labeling, to only 2.4 hours. This efficiency gain is associated with only a 6.7% decrease in overall mean average precision (mAP) when training multispectral models via MATT, compared to a manually labeled dataset. We consider this an acceptable level of precision loss when considering the time saved during training, especially for rapidly prototyping experimental modeling methods. This research greatly contributes to the study of multispectral object detection by providing a novel and open-source method to rapidly segment, label, and train multispectral object detection models with minimal human interaction. Future research needs to focus on applying these methods to 1) space-based multispectral; and 2) drone-based hyperspectral imagery.https://ieeexplore.ieee.org/document/10815733/Thermal object detectionRGB-thermal fusionlong-wave infrared (LWIR)multispectral imagery (MSI)computer visionmachine learning |
| spellingShingle | James E. Gallagher Aryav Gogia Edward J. Oughton A Multispectral Automated Transfer Technique (MATT) for Machine-Driven Image Labeling Utilizing the Segment Anything Model (SAM) IEEE Access Thermal object detection RGB-thermal fusion long-wave infrared (LWIR) multispectral imagery (MSI) computer vision machine learning |
| title | A Multispectral Automated Transfer Technique (MATT) for Machine-Driven Image Labeling Utilizing the Segment Anything Model (SAM) |
| title_full | A Multispectral Automated Transfer Technique (MATT) for Machine-Driven Image Labeling Utilizing the Segment Anything Model (SAM) |
| title_fullStr | A Multispectral Automated Transfer Technique (MATT) for Machine-Driven Image Labeling Utilizing the Segment Anything Model (SAM) |
| title_full_unstemmed | A Multispectral Automated Transfer Technique (MATT) for Machine-Driven Image Labeling Utilizing the Segment Anything Model (SAM) |
| title_short | A Multispectral Automated Transfer Technique (MATT) for Machine-Driven Image Labeling Utilizing the Segment Anything Model (SAM) |
| title_sort | multispectral automated transfer technique matt for machine driven image labeling utilizing the segment anything model sam |
| topic | Thermal object detection RGB-thermal fusion long-wave infrared (LWIR) multispectral imagery (MSI) computer vision machine learning |
| url | https://ieeexplore.ieee.org/document/10815733/ |
| work_keys_str_mv | AT jamesegallagher amultispectralautomatedtransfertechniquemattformachinedrivenimagelabelingutilizingthesegmentanythingmodelsam AT aryavgogia amultispectralautomatedtransfertechniquemattformachinedrivenimagelabelingutilizingthesegmentanythingmodelsam AT edwardjoughton amultispectralautomatedtransfertechniquemattformachinedrivenimagelabelingutilizingthesegmentanythingmodelsam AT jamesegallagher multispectralautomatedtransfertechniquemattformachinedrivenimagelabelingutilizingthesegmentanythingmodelsam AT aryavgogia multispectralautomatedtransfertechniquemattformachinedrivenimagelabelingutilizingthesegmentanythingmodelsam AT edwardjoughton multispectralautomatedtransfertechniquemattformachinedrivenimagelabelingutilizingthesegmentanythingmodelsam |