LITP-GAN: Leaf image translation pipeline using DETR and GLA-Net considering the lesion area of plant disease - Ri Zheng; Jimin Lee; Helin Yin; Dong Jin; Yeong Hyeon Gu (Journal of King Saud University: Computer and Information Sciences)
Abstract Quick disease diagnosis is crucial in agriculture to reduce crop yield losses. Deep learning techniques have already been successfully applied to several tasks in agriculture, but it requires sufficient datasets. Obtaining sufficient plant disease images is challenging due to seasonality and a shortage of experts. Generative Adversarial Networks (GANs) can address this issue by generating synthetic images, but existing methods struggle with complex diseases like fire blight, where symptoms are localized. To overcome this problem, we propose LITP-GAN, a novel image-to-image translation pipeline considering the lesion area. LITP-GAN includes three stages: (Stage 1) Lesion area detection using DEtection TRansformer (DETR) to detect lesion areas, (Stage 2) Image translation via Global and Local Alignment Networks (GLA-Net) to generate diseased leaf images, and (Stage 3) Post-Processing using Seamless Cloning method for generating realistic images by minimizing visual inconsistencies. We tested 400 healthy leaf images, and 236 generated images were identified as realistic diseased images. Applying these to a real field plant disease detection model showed significant improvements: 7.57% in precision, 4% in recall, and 3.5% in mAP compared to models trained on the original datasets. These findings suggest that LITP-GAN-augmented datasets can improve real field plant disease diagnosis, providing a promising tool for agricultural applications.