Domain Adaptation for Eye Segmentation

Domain adaptation (DA) has been widely investigated as a framework to alleviate the laborious task of data annotation for image segmentation. Most DA investigations operate under the unsupervised domain adaptation (UDA) setting, where the modeler has access to a large cohort of source domain labeled data and target domain data with no annotations. UDA techniques exhibit poor performance when the domain gap, i.e., the distribution overlap between the data in source and target domain is large. We hypothesize that the DA performance gap can be improved with the availability of a small subset of labeled target domain data. In this paper, we systematically investigate the impact of varying amounts of labeled target domain data on the performance gap for DA. We specifically focus on the problem of segmenting eye-regions from eye images collected using two different head mounted display systems. Source domain is comprised of 12,759 eye images with annotations and target domain is comprised of 4,629 images with varying amounts of annotations. Experiments are performed to compare the impact on DA performance gap under three schemes: unsupervised (UDA), supervised (SDA) and semi-supervised (SSDA) domain adaptation. We evaluate these schemes by measuring the mean intersection-over-union (mIoU) metric. Using only 200 samples of labeled target data under SDA and SSDA schemes, we show an improvement in mIoU of 5.4% and 6.6% respectively, over mIoU of 81.7% under UDA. By using all available labeled target data, models trained under SSDA achieve a competitive mIoU score of 89.8%. Overall, we conclude that availability of a small subset of target domain data with annotations can substantially improve DA performance.

[1]  Quinn Jones,et al.  Few-Shot Adversarial Domain Adaptation , 2017, NIPS.

[2]  Trevor Darrell,et al.  FCNs in the Wild: Pixel-level Adversarial and Constraint-based Adaptation , 2016, ArXiv.

[3]  Miguel Angel Ferrer-Ballester,et al.  SSERBC 2017: Sclera segmentation and eye recognition benchmarking competition , 2017, 2017 IEEE International Joint Conference on Biometrics (IJCB).

[4]  David Menotti,et al.  Fully Convolutional Networks and Generative Adversarial Networks Applied to Sclera Segmentation , 2018, 2018 IEEE 9th International Conference on Biometrics Theory, Applications and Systems (BTAS).

[5]  Taesung Park,et al.  CyCADA: Cycle-Consistent Adversarial Domain Adaptation , 2017, ICML.

[6]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Edward J. Delp,et al.  A New Human Identification Method: Sclera Recognition , 2012, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[8]  Maja Pantic,et al.  The iBUG Eye Segmentation Dataset , 2018, ICCSW.

[9]  Konstantinos Kamnitsas,et al.  Domain Adaptation for MRI Organ Segmentation using Reverse Classification Accuracy , 2018, ArXiv.

[10]  Trevor Darrell,et al.  Semi-Supervised Domain Adaptation via Minimax Entropy , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[11]  Mei Wang,et al.  Deep Visual Domain Adaptation: A Survey , 2018, Neurocomputing.

[12]  Jie Shen,et al.  Shape Constrained Network for Eye Segmentation in the Wild , 2019, 2020 IEEE Winter Conference on Applications of Computer Vision (WACV).

[13]  Ming-Hsuan Yang,et al.  Adversarial Learning for Semi-supervised Semantic Segmentation , 2018, BMVC.

[14]  Oleg V. Komogortsev,et al.  Dataset for Eye Tracking on a Virtual Reality Platform , 2020, ETRA.

[15]  Zhiming Luo,et al.  Attention guided U-Net for accurate iris segmentation , 2018, J. Vis. Commun. Image Represent..

[16]  Victor S. Lempitsky,et al.  Unsupervised Domain Adaptation by Backpropagation , 2014, ICML.

[17]  Nuno Vasconcelos,et al.  Bidirectional Learning for Domain Adaptation of Semantic Segmentation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  François Laviolette,et al.  Domain-Adversarial Training of Neural Networks , 2015, J. Mach. Learn. Res..

[19]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[20]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[21]  Yuxing Tang,et al.  Large Scale Semi-Supervised Object Detection Using Visual and Semantic Knowledge Transfer , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Trevor Darrell,et al.  Adversarial Discriminative Domain Adaptation , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[24]  Jian-Gang Wang,et al.  Estimating the eye gaze from one eye , 2005, Comput. Vis. Image Underst..

[25]  Yi-Hsuan Tsai,et al.  Domain Adaptation for Structured Output via Discriminative Patch Representations , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[26]  Chong-Wah Ngo,et al.  Semi-supervised Domain Adaptation with Subspace Learning for visual recognition , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Ming-Hsuan Yang,et al.  Learning to Adapt Structured Output Space for Semantic Segmentation , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[28]  Zhenan Sun,et al.  ScleraSegNet: an Improved U-Net Model with Attention for Accurate Sclera Segmentation , 2019, 2019 International Conference on Biometrics (ICB).

[29]  Jonathan Perry,et al.  MinENet: A Dilated CNN for Semantic Segmentation of Eye Features , 2019, 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW).

[30]  Dong Liu,et al.  Fully Convolutional Adaptation Networks for Semantic Segmentation , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[31]  K. Bowyer,et al.  Handbook of Iris Recognition , 2016, Advances in Computer Vision and Pattern Recognition.

[32]  Woong-Kee Loh,et al.  Sclera-Net: Accurate Sclera Segmentation in Various Sensor Images Based on Residual Encoder and Decoder Network , 2019, IEEE Access.

[33]  Soo-Hyung Kim,et al.  Eye Semantic Segmentation with A Lightweight Model , 2019, 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW).

[34]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[35]  Lizhuang Ma,et al.  Not All Areas Are Equal: Transfer Learning for Semantic Segmentation via Hierarchical Region Selection , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  Naser Damer,et al.  Eye-MMS: Miniature Multi-Scale Segmentation Network of Key Eye-Regions in Embedded Applications , 2019, 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW).

[37]  Mariusz Zubert,et al.  Reliable algorithm for iris segmentation in eye image , 2010, Image Vis. Comput..

[38]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.