头动眼动数据融合的视线方向算法研究与应用

头动眼动数据融合的视线方向算法研究与应用摘要个体眼睛的注视方向便是视线，视线能够体现个体的注意力。当前在计算机视觉研究中，视线估计扮演着十分重要的角色，在许多领域比如人机交互领域、心理学领域等得到了充分运用。如何适应不同的头部姿势，提高视线的精确度，是目前视线估计的难点。本文在分析国内外研究现状及其相关算法的基础上，提出一种基于三目八光源系统，利用深度卷积神经网络将头眼运动数据融合起来的新方法。主要做了以下几方面的工作：1)视线参数特征提取本文视线参数特征包括人眼图像、瞳孔中心和普尔软斑中心。人脸检测是定位人眼区域的前提，本文采用非穿戴式三目八光源系统，使用Haar特征和Adaboost算法实现人脸的识别：然后利用ASM算法对宽视野图像中的人脸实现特征点跟踪并截取人眼图像。为提高视线精度，本文利用卷积神经网络从三个方向随机采集的人脸图像中获取正视屏幕时所对应正脸，分类准确率达99%。在正脸对应的人眼中，借助双重Haar-like特征提取器来对瞳孔位置进行大致定位，接着选用一些手段比如椭圆拟合、形态学像素模型等来将瞳孔中心位置确定下来。将图像阈值处理后的普尔软斑变为高亮点，利用Canny边缘检测算子及查找算法得到普尔软斑位置，最后通过质心法得到普尔软斑中心坐标，视线参数提取成功。2)头部姿态特征提取本文中的头部姿态特征提取数据由横滚、俯仰、偏航角以及空间轴坐标中三轴的位置、速度等构成。此次将以惯性传感器为基础，基于图像的头部姿态估计相结合。首先，采用四阶龙格库塔法、毕卡逼近法和加速度插值积分法将头部姿态的旋转四元数转化为旋转角度，实现传感器姿态运动的跟踪。然后利用图像法根据人脸特征点通过三维人脸模型变换将三维特征映射到二维空间，并通过调整三维模型的深度参数来解算头部姿态角度问题。将这两种方法结合起来，可以提高头部运动测量的效率。3)头眼数据融合视线估计模型基于本实验室研究背景，为了得到以上数据信息，专门展开了数据采集实验，此次实验挑选了研究生志愿者八名，实时记录其在模拟操控期间的人脸、人眼和头动数据，并以此数据集为基础计算出视线落点。本课题研究了头眼的协调运动关系，头部运动能补偿眼部运动，故将头部数据与视线数据相融合。通过改进ResNet18的全连接层，将不同形态的人眼图像、瞳孔中心坐标、普尔钦斑坐标和头部姿态数据作为输入，进行多数据融合，将分类用的网络结构适用于本文的回归任务。本文通过采用改进的卷积神经网络将视线精度提高了20°。关键词：眼动特征：多目多光源：头部姿态：神经网络：数据融合本研究得到国家自然基金（编号：52072293）资助。AbstractThe gaze direction of individual eyes is the line of sight,which can reflect individualattention.At present,line of sight estimation plays a very important role in the research ofcomputer vision,and has been fully used in many fields,such as human-computer interaction,psychology and so on.How to adapt to different head postures and improve the accuracy of lineof sight is the difficulty of line of sight estimation.Based on the analysis of the research statusat home and abroad and its related algorithms,this paper proposes a new method of head eyemotion data fusion based on three eye eight light source system and deep convolution neuralnetwork.The following work has been done:1)Line of sight parameter feature extractionIn this paper,the characteristics of line of sight parameters include human eye image,pupilcenter and pulchin spot center.Face detection is the premise of locating the human eye area.Inthis paper,the non wearable three eye eight light source system is adopted,and the Haar featureand AdaBoost algorithm are used to realize face recognition;Then the ASM algorithm is usedto track the feature points of the face in the wide field image and intercept the human eye image.In order to improve the accuracy of line of sight,this paper uses convolutional neural networkto obtain the corresponding face when facing the screen from the face images randomlycollected in three directions,and the classification accuracy is 99%.In the eyes of the personcorresponding to the front face,the pupil position is roughly located with the help of dual Haarlike feature extractor,and then some means such as ellipse fitting and morphological pixelmodel are selected to determine the pupil center position.The purchin spot after imagethreshold processing is changed into highlight,and the position of purchin spot is obtained byCanny edge detection operator and search algorithm.Finally,the center