Combining Deep Learning Mechanisms to Predict Interests from Gaze

The paper proposes a model to predict users’ personal interests from their gaze. Current image processing technologies enable us to identify each user’s gaze and pupil diameter. However, they cannot identify user interests. The model uses deep learning technologies. It consists of mechanisms to handle the two kinds of gaze: one specific to individuals and the other common to many people. Its training method utilizes the human property that user interests in objects in vision affect their gaze. Since it is known that pupil diameters increase when users view what they are interested in, we can train the model to predict users’ personal interests using the pupil diameters as labels. Especially, the paper not only proposes a model structure to predict personal interests but also examines the process of the modification of parameters of the model to reveal requisites to predict personal interests from gaze. The method enables us to provide personalized information to each user, such as recommendations in VoD services, advertisements during e-shopping, plate annotations in restaurant menus, and so on.