Realtime Emotional Reflective User Interface Based on Deep Convolutional Neural Networks and Generative Adversarial Networks

Burrows, Holly and Zarrin, Javad and Babu Saheer, Lakshmi and Maktab-Dar-Oghaz, Mahdi (2021) Realtime Emotional Reflective User Interface Based on Deep Convolutional Neural Networks and Generative Adversarial Networks. Electronics, 11 (1). p. 118. ISSN 2079-9292

Published Version
Available under the following license: Creative Commons Attribution.

Download (4MB) | Preview
Official URL:


It is becoming increasingly apparent that a significant amount of the population suffers from mental health problems, such as stress, depression, and anxiety. These issues are a result of a vast range of factors, such as genetic conditions, social circumstances, and lifestyle influences. A key cause, or contributor, for many people is their work; poor mental state can be exacerbated by jobs and a person’s working environment. Additionally, as the information age continues to burgeon, people are increasingly sedentary in their working lives, spending more of their days seated, and less time moving around. It is a well-known fact that a decrease in physical activity is detrimental to mental well-being. Therefore, the need for innovative research and development to combat negativity early is required. Implementing solutions using Artificial Intelligence has great potential in this field of research. This work proposes a solution to this problem domain, utilising two concepts of Artificial Intelligence, namely, Convolutional Neural Networks and Generative Adversarial Networks. A CNN is trained to accurately predict when an individual is experiencing negative emotions, achieving a top accuracy of 80.38% with a loss of 0.42. A GAN is trained to synthesise images from an input domain that can be attributed to evoking position emotions. A Graphical User Interface is created to display the generated media to users in order to boost mood and reduce feelings of stress. The work demonstrates the capability for using Deep Learning to identify stress and negative mood, and the strategies that can be implemented to reduce them.

Item Type: Journal Article
Keywords: emotion recognition, deep learning, CNN, GAN, intelligent user interface, human computer interaction, mental health
Faculty: Faculty of Science & Engineering
Depositing User: Lisa Blanshard
Date Deposited: 10 Jan 2022 11:25
Last Modified: 17 Jan 2022 15:22

Actions (login required)

Edit Item Edit Item