基于分层融合与推理一致性的RGB-T显著目标检测方法

杨潞霞; 刘昊杰; 张红瑞; 马永杰

您当前的位置：

首页 >

文章列表页 >

基于分层融合与推理一致性的RGB-T显著目标检测方法

光学传感与测量 | 更新时间：2026-03-24

- 基于分层融合与推理一致性的RGB-T显著目标检测方法
- Hierarchical fusion with consistent reasoning for RGB-T saliency object detection
- 光学技术 2026年52卷第2期页码：236-244
- 作者机构：
  
  1.太原师范学院计算机科学与技术学院，山西晋中 030619
  2.智能优化计算与区块链技术山西省重点实验室，山西晋中 030619
  3.西北师范大学物理与电子工程学院，甘肃兰州 730070
- 作者简介：
  
  杨潞霞（1979— ），女，教授，博士，从事智能信息处理方面的研究。luxiayang@tynu.edu.cn
- 基金信息：
  
  国家自然科学基金项目(62066041);山西省重点研发计划(202102010101008);山西省高等学校科技创新项目(2024L295);山西省科技战略研究专项重点项目(202304031401011);山西省基础研究计划项目（自由探索类）(202403021222276;202503021212249);山西省研究生实践创新项目(2025SJ412);山西省研究生教育改革研究课题(2023JG163)
- DOI：
  中图分类号： TP394.1;TH691.9
- 收稿：2025-12-25，
  
  修回：2026-02-10，
  
  纸质出版：2026-03-20
- 稿件说明：
移动端阅览
杨潞霞,刘昊杰,张红瑞等.基于分层融合与推理一致性的RGB-T显著目标检测方法[J].光学技术,2026,52(02):236-244.

YANG Luxia,LIU Haojie,ZHANG Hongrui,et al.Hierarchical fusion with consistent reasoning for RGB-T saliency object detection[J].OPTICAL TECHNIQUE,2026,52(02):236-244.
杨潞霞,刘昊杰,张红瑞等.基于分层融合与推理一致性的RGB-T显著目标检测方法[J].光学技术,2026,52(02):236-244. DOI：

YANG Luxia,LIU Haojie,ZHANG Hongrui,et al.Hierarchical fusion with consistent reasoning for RGB-T saliency object detection[J].OPTICAL TECHNIQUE,2026,52(02):236-244. DOI：

摘要

针对跨模态显著目标检测中常忽视层级差异与边界恢复精度不足的问题，提出一种基于分层融合与推理一致性的网络（HFCR-Net）。该网络以Swin Transformer双分支编码器提取多尺度特征；然后，设计双门控融合模块（DGF），通过像素级空间门控与通道对齐机制实现深层特征的自适应加权融合，在中低层构建跨模态细化模块（CMR）通过交互式引导与可学习的注意力调控机制细化边缘与纹理；同时设计了推理一致性策略以提升模型的稳定性。实验结果表明，HFCR-Net在VT821数据集上

https://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=102517073&type=

https://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=102517072&type=

3.30200005

3.21733332

、

https://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=102517086&type=

https://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=102517085&type=

3.55599999

2.37066650

、

https://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=102517077&type=

https://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=102517087&type=

3.47133350

2.45533323

与

https://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=102517124&type=

https://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=102517089&type=

7.53533268

2.45533323

分别达到了0.942、0.977、0.985和0.010；在更具挑战性的VT1000与VT5000数据集上

https://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=102517138&type=

https://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=102517125&type=

3.30200005

3.21733332

分别达到了0.927与0.856，进一步验证了方法在不同复杂场景下的有效性与鲁棒性。

Abstract

To address the common issues of overlooked hierarchical differences and insufficient boundary restoration accuracy in cross modal salient object detection

a network based on hierarchical fusion and reasoning consistency （HFCR-Net） is proposed. The network employs a Swin Transformer dual branch encoder to extract multi-scale features. Subsequently

a Dual Gated Fusion module （DGF） is designed to achieve adaptive weighted fusion of deep features through pixel-level spatial gating and channel alignment mechanisms. At the middle and low levels

a Cross Modal Refinement module （CMR） refines edges and textures through interactive guidance and a learnable attention regulation mechanism. In addition

a reasoning-consistent inference strategy is introduced to improve model stability. Experimental results demonstrate that HFCR-Net achieves

https://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=102517116&type=

https://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=102517139&type=

3.30200005

3.21733332

https://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=102517118&type=

https://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=102517117&type=

3.55599999

2.37066650

https://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=102517144&type=

https://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=102517119&type=

3.47133350

2.45533323

and MAE of 0.942

0.977

0.985 and 0.010 on VT821

and attains

https://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=102517170&type=

https://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=102517145&type=

3.30200005

3.21733332

of 0.927 and 0.856 on the more challenging VT1000 and VT5000

respectively

further validating its effectiveness and robustness in diverse complex scenarios.

关键词

Keywords

references

Li Z X , Xie X M , Ling F , et al . Matching images and texts with multi-head attention network for cross-media hashing retrieval [J ] . Engineering Applications of Artificial Intelligence , 2021 , 106 : 104475 .

Zhou W J , Liu J F , Lei J S , et al . GMNet: graded-feature multilabel-learning network for RGB-thermal urban scene semantic segmentation [J ] . IEEE Transactions on Image Processing , 2021 , 30 : 7790 — 7802 .

Zhuge M , Gao D H , Fan D P , et al . Kaleido-BERT: Vision-language pre-training on fashion domain [C ] // Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition（CVPR） . Nashville, TN, USA : IEEE , 2021 : 12642 — 12652 .

Borji A , Frintrop S , Sihite D N , et al . Adaptive object tracking by learning background context [C ] // Proceedings of the 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops . Providence, RI, USA : IEEE , 2012 : 23 — 30 .

杨超 , 官铮 , 王学 , 等 . 显著性增强和全局感知的RGB-T显著性目标检测 [J ] . 计算机辅助设计与图形学学报 , 2025 , 37 （ 08 ）: 1359 — 1370 .

Yang Chao , Guan Zheng , Wang Xue , et al . Saliency enhancement and global perception for RGB-T salient object detection [J ] . Journal of Computer-Aided Design & Computer Graphics , 2025 , 37 （ 08 ）: 1359 — 1370 .

魏明军 , 魏帅 , 刘亚志 , 等 . 基于跨层级注意力学习的RGB-T显著目标检测 [J ] . 郑州大学学报（理学版） , 2025 , 57 （ 03 ）: 42 — 48 .

Wei Mingjun , Wei Shuai , Liu Yazhi , et al . RGB-T salient object detection based on cross-level attention learning [J ] . Journal of Zhengzhou University （Natural Science Edition） , 2025 , 57 （ 03 ）: 42 — 48 .

Lv C , Zhou X , Wan B , et al . Transformer-based cross-modal integration network for RGB-T salient object detection [J ] . IEEE Transactions on Consumer Electronics , 2024 , 70 （ 2 ）: 4741 — 4755 .

Liu Z , Tan Y , He Q , et al . SwinNet: Swin transformer drives edge-aware RGB-D and RGB-T salient object detection [J ] . IEEE Transactions on Circuits and Systems for Video Technology , 2022 , 32 （ 7 ）: 4486 — 4497 .

Tang B , Liu Z , Tan Y , et al . HRTransNet: HRFormer-driven two-modality salient object detection [J ] . IEEE Transactions on Circuits and Systems for Video Technology , 2023 , 33 （ 2 ）: 728 — 742 .

Hou Q , Cheng M M , Hu X , et al . Deeply supervised salient object detection with short connections [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2019 , 41 （ 4 ）: 815 — 828 .

Luo Z , Mishra A K , Achkar A , et al . Non-local deep features for salient object detection [C ] // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 2017 : 6593 — 6601 .

Zhang Q , Huang N C , Yao L , et al . RGB-T salient object detection via fusing multi-level CNN features [J ] . IEEE Transactions on Image Processing , 2020 , 29 : 3321 — 3335 .

Wei J , Wang S , Huang Q . F3Net: Fusion, feedback and focus for salient object detection [C ] // Proceedings of the AAAI Conference on Artificial Intelligence . 2020 , 34 （ 07 ）: 12321 — 12328 .

Tu Z , Li Z , Li C , et al . Multi-interactive dual-decoder for RGB-thermal salient object detection [J ] . IEEE Transactions on Image Processing , 2021 , 30 : 5678 — 5691 .

Bi H , Zhang J , Wu R , et al . RGB-T salient object detection via excavating and enhancing CNN features [J ] . Applied Intelligence , 2023 , 53 （ 21 ）: 25543 — 25561 .

Huo F , Zhu X , Zhang L , et al . Efficient context-guided stacked refinement network for RGB-T salient object detection [J ] . IEEE Transactions on Circuits and Systems for Video Technology , 2022 , 32 （ 5 ）: 3111 — 3124 .

Jiang X , Zhu L , Hou Y , et al . Mirror complementary transformer network for RGB-thermal salient object detection [J ] . IET Computer Vision , 2023 , 18 （ 1 ）: 15 — 32 .

Huo F , Zhu X , Zhang Q , et al . Real-time one-stream semantic-guided refinement network for RGB-thermal salient object detection [J ] . IEEE Transactions on Instrumentation and Measurement , 2022 , 71 : 1 — 12 .

Zhang P , Xu M N , Zhang Z Y , et al . Feature aggregation with transformer for RGB-T salient object detection [J ] . Neurocomputing , 2023 , 546 : 126329 .

Zhou W , Zhu Y , Lei J , et al . LSNet: Lightweight spatial boosting network for detecting salient objects in RGB-thermal images [J ] . IEEE Transactions on Image Processing , 2023 , 32 : 1329 — 1340 .

Liu C , Yang G , Wang Y . TANet: Transformer-based asymmetric network for RGB-D salient object detection [J ] . IET Computer Vision , 2023 , 17 （ 4 ）: 415 — 430 .

Tu Z , Ma Y , Li Z , et al . RGBT salient object detection: a large-scale dataset and benchmark [J ] . IEEE Transactions on Multimedia , 2022 , 25 : 4163 — 4176 .

Song K , Huang L , Gong A , et al . Multiple graph affinity interactive network and a variable illumination dataset for RGBT image salient object detection [J ] . IEEE Transactions on Circuits and Systems for Video Technology , 2023 , 33 （ 7 ）: 3104 — 3118 .

Pang Y , Zhao X , Zhang L , et al . CAVER: Cross-modal view-mixed transformer for bi-modal salient object detection [J ] . IEEE Transactions on Image Processing , 2023 , 32 : 892 — 904 .

Jin D , Shao F , Xie Z , et al . CAFCNet: Cross-modality asymmetric feature complement network for RGB-T salient object detection [J ] . Expert Systems with Applications , 2024 , 247 : 123222 .

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

暂无数据