An Improved Backbone Fusion Neural Network for Orchard Extraction
The monitoring and extraction of orchard planting information is of significant importance. Semantic segmentation deep learning models utilizing convolutional neural networks (CNNs) or vision transformers have become the cornerstone for such tasks. However, different backbone networks exhibit varyin...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/11072220/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1839611133840850944 |
|---|---|
| author | Baiyu Dong Ziqi Wang Chongzhi Chen Ke Wang Jing Zhang |
| author_facet | Baiyu Dong Ziqi Wang Chongzhi Chen Ke Wang Jing Zhang |
| author_sort | Baiyu Dong |
| collection | DOAJ |
| description | The monitoring and extraction of orchard planting information is of significant importance. Semantic segmentation deep learning models utilizing convolutional neural networks (CNNs) or vision transformers have become the cornerstone for such tasks. However, different backbone networks exhibit varying capabilities and characteristics in feature extraction, limiting the performance of a single backbone model. Traditional fusion backbone networks lack strategies for dynamically adjusting weights, thus failing to fully leverage the strengths of various backbone networks. Therefore, this study proposed a novel backbone fusion strategy (FSC) by a dual information-interactive fusion architecture and an attention gate mechanism. Based on the FSC strategy, we integrated two powerful backbone, ResNet50 and swin transformer, to develop a backbone fusion network (FSRNet) for orchard extraction. We further incorporated skip-connection attention block, optimized the loss function, and improved the dataset by adding more spectral and texture information to enhance the model performance. The results indicated that the FSRNet performs excellently in orchard extraction, with overall classification accuracy and F1 score both exceeding classical or state-of-the-art models. In addition, the FSC strategy yields the best results among different backbone fusing strategies, with an F1 score of 0.864 and an overall accuracy (OA) of 0.897. Compared to single backbone networks, the F1 score and OA obtained based on the FSR strategy have increased by an average of 8.5% and 4.2%, respectively. When compared to traditional backbone fusion strategies, they have risen by an average of 4.5% and 2.3%, respectively. This fully demonstrates the effectiveness and robustness of the proposed fusion strategy. The FSRNet combines the local feature extraction capability of ResNet50 with the global feature extraction ability of the swin transformer, showcasing the immense potential of fused backbone models in semantic segmentation tasks. |
| format | Article |
| id | doaj-art-dea8b7848b524932bf8a95e2c44b6be9 |
| institution | Matheson Library |
| issn | 1939-1404 2151-1535 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| spelling | doaj-art-dea8b7848b524932bf8a95e2c44b6be92025-07-28T23:00:11ZengIEEEIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing1939-14042151-15352025-01-0118179611797410.1109/JSTARS.2025.358632211072220An Improved Backbone Fusion Neural Network for Orchard ExtractionBaiyu Dong0https://orcid.org/0000-0001-7164-7475Ziqi Wang1Chongzhi Chen2Ke Wang3Jing Zhang4https://orcid.org/0009-0008-6436-2509College of Environmental and Resource Sciences, Zhejiang University, Hangzhou, ChinaCollege of Environmental and Resource Sciences, Zhejiang University, Hangzhou, ChinaCollege of Environmental and Resource Sciences, Zhejiang University, Hangzhou, ChinaCollege of Environmental and Resource Sciences, Zhejiang University, Hangzhou, ChinaCollege of Environmental and Resource Sciences, Zhejiang University, Hangzhou, ChinaThe monitoring and extraction of orchard planting information is of significant importance. Semantic segmentation deep learning models utilizing convolutional neural networks (CNNs) or vision transformers have become the cornerstone for such tasks. However, different backbone networks exhibit varying capabilities and characteristics in feature extraction, limiting the performance of a single backbone model. Traditional fusion backbone networks lack strategies for dynamically adjusting weights, thus failing to fully leverage the strengths of various backbone networks. Therefore, this study proposed a novel backbone fusion strategy (FSC) by a dual information-interactive fusion architecture and an attention gate mechanism. Based on the FSC strategy, we integrated two powerful backbone, ResNet50 and swin transformer, to develop a backbone fusion network (FSRNet) for orchard extraction. We further incorporated skip-connection attention block, optimized the loss function, and improved the dataset by adding more spectral and texture information to enhance the model performance. The results indicated that the FSRNet performs excellently in orchard extraction, with overall classification accuracy and F1 score both exceeding classical or state-of-the-art models. In addition, the FSC strategy yields the best results among different backbone fusing strategies, with an F1 score of 0.864 and an overall accuracy (OA) of 0.897. Compared to single backbone networks, the F1 score and OA obtained based on the FSR strategy have increased by an average of 8.5% and 4.2%, respectively. When compared to traditional backbone fusion strategies, they have risen by an average of 4.5% and 2.3%, respectively. This fully demonstrates the effectiveness and robustness of the proposed fusion strategy. The FSRNet combines the local feature extraction capability of ResNet50 with the global feature extraction ability of the swin transformer, showcasing the immense potential of fused backbone models in semantic segmentation tasks.https://ieeexplore.ieee.org/document/11072220/Attention blockbackbone fusiondeep learningorchard extractionResNetswin transformer |
| spellingShingle | Baiyu Dong Ziqi Wang Chongzhi Chen Ke Wang Jing Zhang An Improved Backbone Fusion Neural Network for Orchard Extraction IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing Attention block backbone fusion deep learning orchard extraction ResNet swin transformer |
| title | An Improved Backbone Fusion Neural Network for Orchard Extraction |
| title_full | An Improved Backbone Fusion Neural Network for Orchard Extraction |
| title_fullStr | An Improved Backbone Fusion Neural Network for Orchard Extraction |
| title_full_unstemmed | An Improved Backbone Fusion Neural Network for Orchard Extraction |
| title_short | An Improved Backbone Fusion Neural Network for Orchard Extraction |
| title_sort | improved backbone fusion neural network for orchard extraction |
| topic | Attention block backbone fusion deep learning orchard extraction ResNet swin transformer |
| url | https://ieeexplore.ieee.org/document/11072220/ |
| work_keys_str_mv | AT baiyudong animprovedbackbonefusionneuralnetworkfororchardextraction AT ziqiwang animprovedbackbonefusionneuralnetworkfororchardextraction AT chongzhichen animprovedbackbonefusionneuralnetworkfororchardextraction AT kewang animprovedbackbonefusionneuralnetworkfororchardextraction AT jingzhang animprovedbackbonefusionneuralnetworkfororchardextraction AT baiyudong improvedbackbonefusionneuralnetworkfororchardextraction AT ziqiwang improvedbackbonefusionneuralnetworkfororchardextraction AT chongzhichen improvedbackbonefusionneuralnetworkfororchardextraction AT kewang improvedbackbonefusionneuralnetworkfororchardextraction AT jingzhang improvedbackbonefusionneuralnetworkfororchardextraction |