Taking it further: Leveraging pseudo-labels for field delineation across label-scarce smallholder regions
Satellite-based field delineation has entered a quasi-operational stage due to recent advances in machine learning for computer vision. Transfer learning allows for the resource-efficient transfer of pre-trained field delineation models across heterogeneous geographies. However, the scarcity of labe...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2024-11-01
|
| Series: | International Journal of Applied Earth Observations and Geoinformation |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S156984322400503X |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1846166239646318592 |
|---|---|
| author | Philippe Rufin Sherrie Wang Sá Nogueira Lisboa Jan Hemmerling Mirela G. Tulbure Patrick Meyfroidt |
| author_facet | Philippe Rufin Sherrie Wang Sá Nogueira Lisboa Jan Hemmerling Mirela G. Tulbure Patrick Meyfroidt |
| author_sort | Philippe Rufin |
| collection | DOAJ |
| description | Satellite-based field delineation has entered a quasi-operational stage due to recent advances in machine learning for computer vision. Transfer learning allows for the resource-efficient transfer of pre-trained field delineation models across heterogeneous geographies. However, the scarcity of labeled data for complex and dynamic smallholder landscapes remains a major bottleneck. The key innovation of this study is to overcome this challenge by using pre-trained models to generate sparse (i.e., not fully annotated) field delineation pseudo-labels for fine-tuning models across geographies and sensor characteristics. We build on a FracTAL ResUNet trained for crop field delineation in India (median field size of 0.24 ha) based on multi-spectral imagery at 1.5 m spatial resolution. We use this model to generate pseudo-labels for the use in Northern Mozambique (median field size of 0.06 ha) based on sub-meter resolution true-color satellite imagery. We designed multiple pseudo-label selection strategies based on field-level probability scores and compared the quantities, area properties, seasonal distribution, and spatial agreement of the pseudo-labels against human-annotated training labels (n = 1,512). We then used the human-annotated labels and the pseudo-labels for model fine-tuning and compared predictions against human field annotations (n = 2,199). We evaluated performance with regards to object-level spatial agreement and site-level field size estimation. Our results indicate i) a good baseline performance of the pre-trained model in both field delineation (mean intersection over union (mIoU) of 0.634) and field size estimation (mean root mean squared error (mRMSE) of 0.071 ha), and ii) the added value of regional fine-tuning with performance improvements in nearly all experiments (mIoU increases of up to 0.060, mRMSE decreases of up to 0.034 ha). Moreover, we found iii) substantial performance increases when using only pseudo-labels (up to 77 % of the mIoU increases and 68 % of the mRMSE decreases obtained by human-annotated labels), and iv) additional performance increases (mIoU+0.008, mRMSE: −0.003 ha) when complementing human annotations with pseudo-labels. Pseudo-labels are architecture-agnostic, can be efficiently generated at scale, and thus facilitate domain adaptation in label-scarce settings. The workflow presented here is a stepping stone for overcoming the persisting challenges in mapping heterogeneous smallholder agriculture. |
| format | Article |
| id | doaj-art-f4501e0c6bb44445bff01730b6e49de0 |
| institution | Kabale University |
| issn | 1569-8432 |
| language | English |
| publishDate | 2024-11-01 |
| publisher | Elsevier |
| record_format | Article |
| series | International Journal of Applied Earth Observations and Geoinformation |
| spelling | doaj-art-f4501e0c6bb44445bff01730b6e49de02024-11-16T05:10:00ZengElsevierInternational Journal of Applied Earth Observations and Geoinformation1569-84322024-11-01134104149Taking it further: Leveraging pseudo-labels for field delineation across label-scarce smallholder regionsPhilippe Rufin0Sherrie Wang1Sá Nogueira Lisboa2Jan Hemmerling3Mirela G. Tulbure4Patrick Meyfroidt5Earth and Life Institute, UCLouvain, Place Pasteur 3, 1348 Louvain-la-Neuve, Belgium; Geography Department, Humboldt-Universität zu Berlin, Unter den Linden 6, 10117 Berlin, Germany; Corresponding author.Department of Mechanical Engineering, Massachusetts Institute of Technology, 77 Massachusetts Avenue, Cambridge MA 02139-4307, USANitidae, 500, rue Jean-François Breton, 34 000 Montpellier, France; Department of Forestry Engineering, Universidade Eduardo Mondlane, Av. Julius Nyerere, Maputo, MozambiqueThünen-Institute of Farm Economics, Bundesallee 63, 38116 Braunschweig, GermanyCenter for Geospatial Analytics, North Carolina State University, Jordan Hall, 5112, 2800 Faucette Dr, Raleigh, NC 27695, USAEarth and Life Institute, UCLouvain, Place Pasteur 3, 1348 Louvain-la-Neuve, Belgium; F.R.S.-FNRS, Rue d’Egmont 5, 1000 Brussels, BelgiumSatellite-based field delineation has entered a quasi-operational stage due to recent advances in machine learning for computer vision. Transfer learning allows for the resource-efficient transfer of pre-trained field delineation models across heterogeneous geographies. However, the scarcity of labeled data for complex and dynamic smallholder landscapes remains a major bottleneck. The key innovation of this study is to overcome this challenge by using pre-trained models to generate sparse (i.e., not fully annotated) field delineation pseudo-labels for fine-tuning models across geographies and sensor characteristics. We build on a FracTAL ResUNet trained for crop field delineation in India (median field size of 0.24 ha) based on multi-spectral imagery at 1.5 m spatial resolution. We use this model to generate pseudo-labels for the use in Northern Mozambique (median field size of 0.06 ha) based on sub-meter resolution true-color satellite imagery. We designed multiple pseudo-label selection strategies based on field-level probability scores and compared the quantities, area properties, seasonal distribution, and spatial agreement of the pseudo-labels against human-annotated training labels (n = 1,512). We then used the human-annotated labels and the pseudo-labels for model fine-tuning and compared predictions against human field annotations (n = 2,199). We evaluated performance with regards to object-level spatial agreement and site-level field size estimation. Our results indicate i) a good baseline performance of the pre-trained model in both field delineation (mean intersection over union (mIoU) of 0.634) and field size estimation (mean root mean squared error (mRMSE) of 0.071 ha), and ii) the added value of regional fine-tuning with performance improvements in nearly all experiments (mIoU increases of up to 0.060, mRMSE decreases of up to 0.034 ha). Moreover, we found iii) substantial performance increases when using only pseudo-labels (up to 77 % of the mIoU increases and 68 % of the mRMSE decreases obtained by human-annotated labels), and iv) additional performance increases (mIoU+0.008, mRMSE: −0.003 ha) when complementing human annotations with pseudo-labels. Pseudo-labels are architecture-agnostic, can be efficiently generated at scale, and thus facilitate domain adaptation in label-scarce settings. The workflow presented here is a stepping stone for overcoming the persisting challenges in mapping heterogeneous smallholder agriculture.http://www.sciencedirect.com/science/article/pii/S156984322400503XMozambiqueSub-Saharan AfricaDeep LearningTransfer LearningEarth ObservationCropland |
| spellingShingle | Philippe Rufin Sherrie Wang Sá Nogueira Lisboa Jan Hemmerling Mirela G. Tulbure Patrick Meyfroidt Taking it further: Leveraging pseudo-labels for field delineation across label-scarce smallholder regions International Journal of Applied Earth Observations and Geoinformation Mozambique Sub-Saharan Africa Deep Learning Transfer Learning Earth Observation Cropland |
| title | Taking it further: Leveraging pseudo-labels for field delineation across label-scarce smallholder regions |
| title_full | Taking it further: Leveraging pseudo-labels for field delineation across label-scarce smallholder regions |
| title_fullStr | Taking it further: Leveraging pseudo-labels for field delineation across label-scarce smallholder regions |
| title_full_unstemmed | Taking it further: Leveraging pseudo-labels for field delineation across label-scarce smallholder regions |
| title_short | Taking it further: Leveraging pseudo-labels for field delineation across label-scarce smallholder regions |
| title_sort | taking it further leveraging pseudo labels for field delineation across label scarce smallholder regions |
| topic | Mozambique Sub-Saharan Africa Deep Learning Transfer Learning Earth Observation Cropland |
| url | http://www.sciencedirect.com/science/article/pii/S156984322400503X |
| work_keys_str_mv | AT philipperufin takingitfurtherleveragingpseudolabelsforfielddelineationacrosslabelscarcesmallholderregions AT sherriewang takingitfurtherleveragingpseudolabelsforfielddelineationacrosslabelscarcesmallholderregions AT sanogueiralisboa takingitfurtherleveragingpseudolabelsforfielddelineationacrosslabelscarcesmallholderregions AT janhemmerling takingitfurtherleveragingpseudolabelsforfielddelineationacrosslabelscarcesmallholderregions AT mirelagtulbure takingitfurtherleveragingpseudolabelsforfielddelineationacrosslabelscarcesmallholderregions AT patrickmeyfroidt takingitfurtherleveragingpseudolabelsforfielddelineationacrosslabelscarcesmallholderregions |