Human Expression Recognition

Abstract

The Brain-Like Artificial Intelligence (BLAI) is pioneered by Prof.Nikola Kasabov and here it is applied to a specific application.

Understanding and measuring human emotions and behaviours have long been an interest of academic and commercial researchers. The proposed research methodology relies on the assumptions that there is a valid reflection of the human state of mind by automatically sensing facial and body expression which will provide a future prediction of a certain behaviour. The proposed research is a new AI, brain-inspired, facial, social and emotional analysis approach as part of a decision support system to measure human emotions and behaviours. The automated Affective analysis will lead to the development of Spiking Neural Network (SNN) model of Cognitive Affective-Quotient (SNN-CAQ) that will provide a better way to understand and recognise individual human behavioural propensity in the decision making. A SNN-CAQ is a temporal and multimodal pattern that evolves dynamically to measure the change of facial expression, body gesture and social cue that recognises the sentiment intensity of a human cognitive function. The significance of this research project will allow an affective-state to associate with a quotient that closely correlates to a behavioural tendency. Consumer behavioural preference towards a product and services is identified as one of the research case studies because the measurement of consumer’s perception, reaction and satisfaction continue to be an open problem in the product design, service provider and retail industries. Importantly, the automated analysis of using Affective Quotient can apply to a wide variety of domain-specific applications like analysing unsafe driving behaviour or aggressive crowd management.

FIGURE 1.  Schematic process flow of input from the emotional expression data into NeuCube.

FIGURE 2. Five Expressions labels – 01- Contempt, 02- Disgust, 03 – Joy, 04 – Surprise and 05 - Sadness.

FIGURE 3. Spikes connections after training the Cube and Classifier.

Related Papers and Benchmarking

The proposed methods and systems, when compared with traditional statistical and machine learning methods, showed superior results in the following aspects:

  1. Better data analysis and classification/regression accuracy (by 10 to 40%);
  2. Better visualisation of the created models, with a possible use of VR;
  3. Better understanding of the data and the processes that are measured;
  4. Enabling new information and knowledge discovery through meaningful interpretation of the models.

See also some of the related papers:

Kasabov, N. K. (2014). NeuCube: A spiking neural network architecture for mapping, learning and understanding of spatio-temporal brain dataNeural Networks52, 62-76.

Kasabov, N., Scott, N. M., Tu, E., Marks, S., Sengupta, N., Capecci, E., Othman, M., Gholoami Doborjeh, M., Murli, N., Hartono, R., Espinosa-Ramos, J. I., Zhou, L., Alvi, F., Wang, G., Taylor, D., Feigin, V., Gulyaev, S., Mahmoud, M., Hou, Z. G., Yang, J. (2016). Evolving spatio-temporal data machines based on the NeuCube neuromorphic framework: design methodology and selected applicationsNeural Networks78, 1-14.


R&D System

For this project, an R&D system has been developed based on NeuCube. The system can be obtained for R&D subject to licensing agreement.


Developer

The developers of this project are:

ClarenceTan