ReluformerN: Lightweight High-Low Frequency Enhanced for Hyperspectral Agricultural Lancover Classification - LIU Yi; ZHANG Yanjun (智慧农业)
[Objective]In order to intelligently monitor the distribution of agricultural land cover types, high-spectral cameras are usually mounted on drones to collect high-spectral data, followed by classification of the high-spectral data to automatically draw crop distribution maps. Different crops have similar shapes, and the same crop has significant differences in different growth stages, so the network model for agricultural land cover classification requires a high degree of accuracy. However, network models with high classification accuracy are often complex and cannot be deployed on hardware systems. In view of this problem, a lightweight high-low frequency enhanced Reluformer network (ReluformerN) was proposed in this research.[Methods]Firstly, an adaptive octave convolution was proposed, which utilized the softmax function to automatically adjust the spectral dimensions of high-frequency features and low-frequency features, effectively alleviating the influence of manually setting the spectral dimensions and benefiting the subsequent extraction of spatial and spectral domain features of hyperspectral images. Secondly, a Reluformer was proposed to extract global features, taking advantage of the fact that low-frequency information could capture global features. Reluformer replaced the softmax function with a function of quadratic computational complexity, and through theoretical and graphical analysised, Relu function, LeakRelu function, and Gelu function were compared, it was found that the ReLU function and the softmax function both had non-negativity, which could be used for feature relevance analysis. Meanwhile, the ReLU function has a linearization feature, which is more suitable for self-relevance analysis. Therefore, the ReLU self-attention mechanism was proposed, which used the ReLU function to perform feature self-attention analysis. In order to extract deep global features, multi-scale feature fusion was used, and the ReLU self-attention mechanism was used as the core to construct the multi-head ReLU self-attention mechanism. Similar to the transformer architecture, the Reluformer structure was built by combining multi-head ReLU self-attention mechanism, feedforward layers, and normalization layers. With Reluformer as the core, the Reluformer network (ReluformerN) was proposed. This network considered frequency from the perspective of high-frequency information, taking into account the local features of image high-frequency information, and used deep separable convolution to design a lightweight network for fine-grained feature extraction of high-frequency information. It proposed Reluformer to extract global features for low-frequency information, which represented the global features of the image. ReluformerN was experimented on three public high-spectral data sets (Indian Pines, WHU-Hi-LongKou and Salinas) for crop variety fine classification, and was compared with five popular classification networks (2D-CNN, HybirdSN, ViT, CTN and LSGA-VIT).[Results and Discussion]ReluformerN performed best in overall accuracy (OA), average accuracy (AA), and other accuracy evaluation indicators. In the evaluation indicators of model parameters, model computation (FLOPs), and model complexity, ReluformerN had the smallest number of parameters and was less than 0.3 M, and the lowest computation. In the visualization comparison, the classification effect diagram of the model using ReluformerN had clearer image edges and more complete morphological structures, with fewer classification errors. The validity of the adaptive octave convolution was verified by comparing it with the traditional eightfold convolution. The classification accuracy of the adaptive octave convolution was 0.1% higher than that of the traditional octave convolution. When the artificial parameters were set to different values, the maximum and minimum classification accuracies of the traditional octave convolution were about 0.3% apart, while those of the adaptive octave convolution were only 0.05%. This showed that the adaptive octave convolution not only had the highest classification accuracy, but was also less sensitive to the artificial parameter setting, effectively overcoming the influence of the artificial parameter setting on the classification result. To validated the Reluformer module, it was compared with transformer, LeakRelufromer, and Linformer in terms of accuracy evaluation metrics such as OA and AA. The Reluformer achieved the highest classification accuracy and the lowest model parameter count among these models. This indicated that Reluformer not only effectively extracted global features but also reduced computational complexity. Finally, the effectiveness of the high-frequency and low-frequency branch networks was verified. The effectiveness of the high-frequency and low-frequency feature extraction branches was verified, and the characteristics of the feature distribution after high-frequency feature extraction, after high-low frequency feature extraction, and after the classifier were displayed using a 2D t-sne, compared with the original feature distribution. It was found that after high-frequency feature extraction, similar features were generally clustered together, but the spacing between different features was small, and there were also some features with overlapping situations. After low-frequency feature extraction, it was obvious that similar features were clustered more tightly. After high-low frequency feature fusion, and after the classifier, it was obvious that similar features were clustered, and different types of features were clearly separated, indicating that high-low frequency feature extraction enhanced the classification effect.[Conclusion]This network achieves a good balance between crop variety classification accuracy and model complexity, and is expected to be deployed on hardware systems with limited resources in the future to achieve real-time classification functions.