Self-supervised gaze estimation method based on feature disentanglement

QIN Shanbin; CHEN Jing; ZHANG Tianqi; FU Shanfeng

您当前的位置：

首页 >

文章列表页 >

Self-supervised gaze estimation method based on feature disentanglement

Optical Image Processing | 更新时间：2026-07-23

- Self-supervised gaze estimation method based on feature disentanglement
- OPTICAL TECHNIQUE Vol. 52, Issue 4, Pages: 491-499(2026)
- 作者机构：
  
  北京理工大学光电学院，北京 100081
- 作者简介：
- 基金信息：
- DOI：
  CLC： TP391.41
- Received：11 February 2026，
  
  Revised：2026-04-20，
  
  Accepted：29 April 2026，
  
  Online First：23 July 2026，
  
  Published：20 July 2026
- 稿件说明：
移动端阅览
秦善彬,陈靖,张湉琪等.基于特征解耦的自监督视线估计方法[J].光学技术,2026,52(04):491-499.

QIN Shanbin,CHEN Jing,ZHANG Tianqi,et al.Self-supervised gaze estimation method based on feature disentanglement[J].OPTICAL TECHNIQUE,2026,52(04):491-499.
秦善彬,陈靖,张湉琪等.基于特征解耦的自监督视线估计方法[J].光学技术,2026,52(04):491-499. DOI：

QIN Shanbin,CHEN Jing,ZHANG Tianqi,et al.Self-supervised gaze estimation method based on feature disentanglement[J].OPTICAL TECHNIQUE,2026,52(04):491-499. DOI：

摘要

视线估计作为眼动追踪的关键技术，广泛应用于人机交互。现有方法一般依赖大量高精度注视标注面部图像，然而标注这些数据需要大量人力与物力。工作提出了一种基于特征解耦的自监督视线估计方法，根据视线估计预训练模型生成的伪标签构建注视方向相似图像对和身份与头姿相似图像对，并计算其注视相似度，采用注视解耦双编码器，分别从相似图像对中提取与注视方向相关的特征，以及头部姿态、身份信息等与注视无关的特征；随后，通过语义解耦判别模块预测不同类型特征间的语义相似性。同时使用交叉重构对该图像对的注视相似度迭代更新，进而完成与注视方向相关与无关的特征的解耦。实验表明，方法能有效将注视相关特征与无关特征解耦，相较于现有方法取得了更优异的性能，在MPIIFaceGaze数据集上的视线估计角度误差低至4.06°。

Abstract

Gaze estimation

as a key technology in eye movement tracking

is widely applied in human–computer interaction. Existing methods generally rely on a large number of facial images annotated with high-precision gaze directions

yet annotating such data incurs substantial human and material costs. This work proposes a self-supervised gaze estimation method based on feature disentanglement

which constructs gaze direction similar image pairs and identity & head pose similar image pairs using pseudo-labels generated by a pre-trained gaze estimation model

and computes their gaze similarity. A gaze disentanglement dual encoder is employed to extract gaze direction-related features and gaze-irrelevant features （such as head pose and identity information） from similar image pairs

respectively. A semantic disentanglement discriminative module then predicts the semantic similarity between different types of features. Meanwhile

cross-entropy reconstruction is used to iteratively update the gaze similarity of these image pairs

thereby achieving the disentanglement of gaze direction-related and gaze-irrelevant features. Experiments demonstrate that the proposed method can effectively disentangle gaze-related features from irrelevant features and achieves superior performance compared to state-of-the-art methods

with a gaze estimation angular error as low as 4.06° on the MPIIFaceGaze dataset.

关键词

Keywords

references

Clay V , König P , Koenig S . Eye tracking in virtual reality [J ] . Journal of Eye Movement Research , 2019 , 12 （ 1 ）: 3 .

Koulieris G A , Akşit K , Stengel M , et al . Near-eye display and tracking technologies for virtual and augmented reality [C ] // Computer Graphics Forum . Vienna, Austria : Wiley , 2019 : 493 — 519 .

Wang K , Gao Y , Song W , et al . Gaze pattern genius: gaze-driven vr interaction using unsupervised domain adaption [C ] // 2024 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops （VRW） . Wellington, New Zealand : IEEE , 2024 : 937 — 938 .K.

Swan G , Goldstein R B , Savage S W , et al . Automatic processing of gaze movements to quantify gaze scanning behaviors in a driving simulator [J ] . Behavior Research Methods , 2021 , 53 （ 2 ）: 487 — 506 .

Chuang M H , Chen C L , Ma J P . Eye tracking analysis of readers’ psychological interaction with marketing copy referencing life values [C ] // International Conference on Human-Computer Interaction . Los Angeles, CA, USA : Springer International Publishing , 2015 : 87 — 92 .

Yu Y , Odobez J M . Unsupervised representation learning for gaze estimation [C ] // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Seattle, Washington, USA : IEEE , 7314 — 7324 .

Gu D , Lv M , Liu J . Vicsgaze: a gaze estimation method using self-supervised contrastive learning [J ] . Multimedia Systems , 2024 , 30 （ 6 ）: 330 .

Farkhondeh A , Palmero C , Scardapane S , et al . Towards self-supervised gaze estimation [C ] // 33rd British Machine Vision Conference . London, UK : British Machine Vision Association , 2022 : 1 — 13 .

Sun Y , Zeng J , Shan S , et al . Cross-encoder for unsupervised gaze representation learning [C ] // Proceedings of the IEEE/CVF International Conference on Computer Vision . Montreal, QC, Canada : IEEE , 2021 : 3702 — 3711 .

Ni K , Chen J , Wang J , et al . Feature decomposition-based gaze estimation with auxiliary head pose regression [J ] . Pattern Recognition Letters , 2024 , 185 : 137 - 142 .

Zhang X , Sugano Y , Fritz M , et al . It's written all over your face: Full-face appearance-based gaze estimation [C ] // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops . Honolulu, Hawaii, USA : IEEE , 2017 : 51 — 60 .

Abdelrahman A A , Hempel T , Khalifa A , et al . L2cs-net: Fine-grained gaze estimation in unconstrained environments [C ] // 2023 8th International Conference on Frontiers of Signal Processing . Corfu, Greece : IEEE , 2023 : 98 — 102 .

Chen Z , Shi B E . Appearance-based gaze estimation using dilated-convolutions [C ] // 14th Asian Conference on Computer Vision . Perth, Australia : Springer International Publishing , 2018 : 309 — 324 .

Abdelrahman A A , Hempel T , Khalifa A , et al . Mobgazenet: robust gaze estimation mobile network based on progressive attention mechanisms [J ] . Machine Vision and Applications , 2025 , 36 （ 4 ）: 76 .

Huang G , Shi J , Xu J , et al . Gaze estimation by attention-induced hierarchical variational auto-encoder [J ] . IEEE Transactions on Cybernetics , 2023 , 54 （ 4 ）: 2592 — 2605 .

Chen T , Kornblith S , Norouzi M , et al . A simple framework for contrastive learning of visual representations [C ] // 37th International Conference on Machine Learning . Vienna, Austria : PMLR , 2020 : 1597 — 1607 .

Grill J B , Strub F , Altché F , et al . Bootstrap your own latent-a new approach to self-supervised learning [J ] . Advances in Neural Information Processing Systems , 2020 , 33 : 21271 — 21284 .

Mahmud Z , Hungler P , Etemad A . Gaze estimation with eye region segmentation and self-supervised multistream learning [J ] . arXiv preprint arXiv: 2112.07878 , 2021 .

Du L , Zhang X , Lan G . Unsupervised gaze-aware contrastive learning with subject-specific condition [J ] . arXiv preprint arXiv: 2309.04506 , 2023 .

Cheng Y , Lu F . GazeTR: gaze estimation with transformer [C ] // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . New Orleans, Louisiana, USA : IEEE , 2022 : 16347 — 16356 .

Hu M , Feng J , Hua J , et al . Online convolutional re-parameterization [C ] // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . New Orleans, Louisiana, USA : IEEE , 2022 : 568 — 577 .

Zhang X , Sugano Y , Fritz M , et al . Mpiigaze: Real-world dataset and deep appearance-based gaze estimation [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2017 , 41 （ 1 ）: 162 — 175 .

Funes Mora K A , Monay F , Odobez J M . Eyediap: A database for the development and evaluation of gaze estimation algorithms from rgb and rgb-d cameras [C ] // Proceedings of the Symposium on Eye Tracking Research and Applications . Safety Harbor, Florida, USA : ACM , 2014 : 255 — 258 .

Zhang X , Park S , Beeler T , et al . Eth-xgaze: A large scale dataset for gaze estimation under extreme head pose and gaze variation [C ] // 16th European Conference on Computer Vision . Glasgow, UK : Springer International Publishing , 2020 : 365 — 381 .

Cheng Y , Wang H , Bao Y , et al . Appearance-based gaze estimation with deep learning: A review and benchmark [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2024 , 46 （ 12 ）: 7509 — 7528 .

Noh H , Hong S , Han B . Learning deconvolution network for semantic segmentation [C ] // Proceedings of the IEEE International Conference on Computer Vision . Santiago, Chile : IEEE , 2015 : 1520 — 1528 .

Zhang C , Liu H , Deng Y , et al . Tokenhpe: Learning orientation tokens for efficient head pose estimation via transformers [C ] // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Vancouver, BC, Canada : IEEE , 2023 : 8897 — 8906 .

Kingma D P , Ba J . Adam: A method for stochastic optimization [J ] . arXiv preprint arXiv: 1412.6980 , 2014 .

Zhang X , Sugano Y , Fritz M , et al . Appearance-based gaze estimation in the wild [C ] // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . Boston, Massachusetts, USA : IEEE , 2015 : 4511 — 4520 .

Park S , Spurr A , Hilliges O . Deep pictorial gaze estimation [C ] // 15th European Conference on Computer Vision . Munich, Germany : Springer International Publishing , 2018 : 721 — 738 .

Cheng Y , Lu F , Zhang X . Appearance-based gaze estimation via evaluation-guided asymmetric regression [C ] // 15th European Conference on Computer Vision . Munich, Germany : Springer International Publishing , 2018 : 100 — 115 .

Y. Cheng , S. Huang , F. Wang , et al . A Coarse-to-fine adaptive network for appearance-based gaze estimation [C ] // 34th AAAI Conference on Artificial Intelligence . New York, NY, USA : AAAI , 2020 : 10623 — 10630 .

Zhang X , Sugano Y , Bulling A , et al . Learning-based region selection for end-to-end gaze estimation [C ] // 31st British Machine Vision Conference . Virtual : British Machine Vision Association , 2020 : 86 .

Van Der Maaten L , Hinton G . Visualizing data using t-sne [J ] . Journal of Machine Learning Research , 2008 , 9 （ 11 ）: 2579 — 2605 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Square-wave modulation thermoreflectance for metal film thickness measurement

Development of a high-resolution white-light interference and confocal microscopic sensor for three-dimensional surface topography measurement

Monocular-vision measurement of street-tree parameters by integrating depth-space correction and geometric anchors

Design of large field-of-view low-distortion aerial lens based on high-order aspheres

Design of Long-wave image square scanning optical system based on double optical wedges

Related Author

No data

Related Institution

No data

⁰