Hierarchical fusion with consistent reasoning for RGB-T saliency object detection

YANG Luxia; LIU Haojie; ZHANG Hongrui; MA Yongjie

您当前的位置：

首页 >

文章列表页 >

Hierarchical fusion with consistent reasoning for RGB-T saliency object detection

Optical Sensing and Measurement | 更新时间：2026-03-24

- Hierarchical fusion with consistent reasoning for RGB-T saliency object detection
- OPTICAL TECHNIQUE Vol. 52, Issue 2, Pages: 236-244(2026)
- 作者机构：
  
  1.太原师范学院计算机科学与技术学院，山西晋中 030619
  2.智能优化计算与区块链技术山西省重点实验室，山西晋中 030619
  3.西北师范大学物理与电子工程学院，甘肃兰州 730070
- 作者简介：
- 基金信息：
- DOI：
  CLC： TP394.1;TH691.9
- Received：25 December 2025，
  
  Revised：2026-02-10，
  
  Published：20 March 2026
- 稿件说明：
移动端阅览
杨潞霞,刘昊杰,张红瑞等.基于分层融合与推理一致性的RGB-T显著目标检测方法[J].光学技术,2026,52(02):236-244.

YANG Luxia,LIU Haojie,ZHANG Hongrui,et al.Hierarchical fusion with consistent reasoning for RGB-T saliency object detection[J].OPTICAL TECHNIQUE,2026,52(02):236-244.
杨潞霞,刘昊杰,张红瑞等.基于分层融合与推理一致性的RGB-T显著目标检测方法[J].光学技术,2026,52(02):236-244. DOI：

YANG Luxia,LIU Haojie,ZHANG Hongrui,et al.Hierarchical fusion with consistent reasoning for RGB-T saliency object detection[J].OPTICAL TECHNIQUE,2026,52(02):236-244. DOI：

摘要

针对跨模态显著目标检测中常忽视层级差异与边界恢复精度不足的问题，提出一种基于分层融合与推理一致性的网络（HFCR-Net）。该网络以Swin Transformer双分支编码器提取多尺度特征；然后，设计双门控融合模块（DGF），通过像素级空间门控与通道对齐机制实现深层特征的自适应加权融合，在中低层构建跨模态细化模块（CMR）通过交互式引导与可学习的注意力调控机制细化边缘与纹理；同时设计了推理一致性策略以提升模型的稳定性。实验结果表明，HFCR-Net在VT821数据集上

https://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=102517073&type=

https://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=102517072&type=

3.30200005

3.21733332

、

https://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=102517086&type=

https://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=102517085&type=

3.55599999

2.37066650

、

https://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=102517077&type=

https://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=102517087&type=

3.47133350

2.45533323

与

https://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=102517124&type=

https://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=102517089&type=

7.53533268

2.45533323

分别达到了0.942、0.977、0.985和0.010；在更具挑战性的VT1000与VT5000数据集上

https://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=102517138&type=

https://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=102517125&type=

3.30200005

3.21733332

分别达到了0.927与0.856，进一步验证了方法在不同复杂场景下的有效性与鲁棒性。

Abstract

To address the common issues of overlooked hierarchical differences and insufficient boundary restoration accuracy in cross modal salient object detection

a network based on hierarchical fusion and reasoning consistency （HFCR-Net） is proposed. The network employs a Swin Transformer dual branch encoder to extract multi-scale features. Subsequently

a Dual Gated Fusion module （DGF） is designed to achieve adaptive weighted fusion of deep features through pixel-level spatial gating and channel alignment mechanisms. At the middle and low levels

a Cross Modal Refinement module （CMR） refines edges and textures through interactive guidance and a learnable attention regulation mechanism. In addition

a reasoning-consistent inference strategy is introduced to improve model stability. Experimental results demonstrate that HFCR-Net achieves

https://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=102517116&type=

https://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=102517139&type=

3.30200005

3.21733332

https://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=102517118&type=

https://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=102517117&type=

3.55599999

2.37066650

https://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=102517144&type=

https://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=102517119&type=

3.47133350

2.45533323

and MAE of 0.942

0.977

0.985 and 0.010 on VT821

and attains

https://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=102517170&type=

https://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=102517145&type=

3.30200005

3.21733332

of 0.927 and 0.856 on the more challenging VT1000 and VT5000

respectively

further validating its effectiveness and robustness in diverse complex scenarios.

关键词

Keywords

references

Li Z X , Xie X M , Ling F , et al . Matching images and texts with multi-head attention network for cross-media hashing retrieval [J ] . Engineering Applications of Artificial Intelligence , 2021 , 106 : 104475 .

Zhou W J , Liu J F , Lei J S , et al . GMNet: graded-feature multilabel-learning network for RGB-thermal urban scene semantic segmentation [J ] . IEEE Transactions on Image Processing , 2021 , 30 : 7790 — 7802 .

Zhuge M , Gao D H , Fan D P , et al . Kaleido-BERT: Vision-language pre-training on fashion domain [C ] // Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition（CVPR） . Nashville, TN, USA : IEEE , 2021 : 12642 — 12652 .

Borji A , Frintrop S , Sihite D N , et al . Adaptive object tracking by learning background context [C ] // Proceedings of the 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops . Providence, RI, USA : IEEE , 2012 : 23 — 30 .

杨超 , 官铮 , 王学 , 等 . 显著性增强和全局感知的RGB-T显著性目标检测 [J ] . 计算机辅助设计与图形学学报 , 2025 , 37 （ 08 ）: 1359 — 1370 .

Yang Chao , Guan Zheng , Wang Xue , et al . Saliency enhancement and global perception for RGB-T salient object detection [J ] . Journal of Computer-Aided Design & Computer Graphics , 2025 , 37 （ 08 ）: 1359 — 1370 .

魏明军 , 魏帅 , 刘亚志 , 等 . 基于跨层级注意力学习的RGB-T显著目标检测 [J ] . 郑州大学学报（理学版） , 2025 , 57 （ 03 ）: 42 — 48 .

Wei Mingjun , Wei Shuai , Liu Yazhi , et al . RGB-T salient object detection based on cross-level attention learning [J ] . Journal of Zhengzhou University （Natural Science Edition） , 2025 , 57 （ 03 ）: 42 — 48 .

Lv C , Zhou X , Wan B , et al . Transformer-based cross-modal integration network for RGB-T salient object detection [J ] . IEEE Transactions on Consumer Electronics , 2024 , 70 （ 2 ）: 4741 — 4755 .

Liu Z , Tan Y , He Q , et al . SwinNet: Swin transformer drives edge-aware RGB-D and RGB-T salient object detection [J ] . IEEE Transactions on Circuits and Systems for Video Technology , 2022 , 32 （ 7 ）: 4486 — 4497 .

Tang B , Liu Z , Tan Y , et al . HRTransNet: HRFormer-driven two-modality salient object detection [J ] . IEEE Transactions on Circuits and Systems for Video Technology , 2023 , 33 （ 2 ）: 728 — 742 .

Hou Q , Cheng M M , Hu X , et al . Deeply supervised salient object detection with short connections [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2019 , 41 （ 4 ）: 815 — 828 .

Luo Z , Mishra A K , Achkar A , et al . Non-local deep features for salient object detection [C ] // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 2017 : 6593 — 6601 .

Zhang Q , Huang N C , Yao L , et al . RGB-T salient object detection via fusing multi-level CNN features [J ] . IEEE Transactions on Image Processing , 2020 , 29 : 3321 — 3335 .

Wei J , Wang S , Huang Q . F3Net: Fusion, feedback and focus for salient object detection [C ] // Proceedings of the AAAI Conference on Artificial Intelligence . 2020 , 34 （ 07 ）: 12321 — 12328 .

Tu Z , Li Z , Li C , et al . Multi-interactive dual-decoder for RGB-thermal salient object detection [J ] . IEEE Transactions on Image Processing , 2021 , 30 : 5678 — 5691 .

Bi H , Zhang J , Wu R , et al . RGB-T salient object detection via excavating and enhancing CNN features [J ] . Applied Intelligence , 2023 , 53 （ 21 ）: 25543 — 25561 .

Huo F , Zhu X , Zhang L , et al . Efficient context-guided stacked refinement network for RGB-T salient object detection [J ] . IEEE Transactions on Circuits and Systems for Video Technology , 2022 , 32 （ 5 ）: 3111 — 3124 .

Jiang X , Zhu L , Hou Y , et al . Mirror complementary transformer network for RGB-thermal salient object detection [J ] . IET Computer Vision , 2023 , 18 （ 1 ）: 15 — 32 .

Huo F , Zhu X , Zhang Q , et al . Real-time one-stream semantic-guided refinement network for RGB-thermal salient object detection [J ] . IEEE Transactions on Instrumentation and Measurement , 2022 , 71 : 1 — 12 .

Zhang P , Xu M N , Zhang Z Y , et al . Feature aggregation with transformer for RGB-T salient object detection [J ] . Neurocomputing , 2023 , 546 : 126329 .

Zhou W , Zhu Y , Lei J , et al . LSNet: Lightweight spatial boosting network for detecting salient objects in RGB-thermal images [J ] . IEEE Transactions on Image Processing , 2023 , 32 : 1329 — 1340 .

Liu C , Yang G , Wang Y . TANet: Transformer-based asymmetric network for RGB-D salient object detection [J ] . IET Computer Vision , 2023 , 17 （ 4 ）: 415 — 430 .

Tu Z , Ma Y , Li Z , et al . RGBT salient object detection: a large-scale dataset and benchmark [J ] . IEEE Transactions on Multimedia , 2022 , 25 : 4163 — 4176 .

Song K , Huang L , Gong A , et al . Multiple graph affinity interactive network and a variable illumination dataset for RGBT image salient object detection [J ] . IEEE Transactions on Circuits and Systems for Video Technology , 2023 , 33 （ 7 ）: 3104 — 3118 .

Pang Y , Zhao X , Zhang L , et al . CAVER: Cross-modal view-mixed transformer for bi-modal salient object detection [J ] . IEEE Transactions on Image Processing , 2023 , 32 : 892 — 904 .

Jin D , Shao F , Xie Z , et al . CAFCNet: Cross-modality asymmetric feature complement network for RGB-T salient object detection [J ] . Expert Systems with Applications , 2024 , 247 : 123222 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

No data

Related Author

No data

Related Institution

No data

⁰