|
|
 |
| |
|
|
|
Multi-Modal Emotion Recognition Fusing Video and Audio |
|
|
|
PP: 455-462 |
|
|
|
Author(s) |
|
|
|
Chao Xu,
Pufeng Du,
Zhiyong Feng,
Zhaopeng Meng,
Tianyi Cao,
Caichao Dong,
|
|
|
|
Abstract |
|
|
| Emotion plays an important role in human communications. We construct a framework for multi-modal fusion emotion
recognition. Facial expression features and speech features are respectively extracted from image sequences and speech signals. In
order to locate and track facial feature points, we construct an Active Appearance Model for facial images with all kinds of expressions.
Facial Animation Parameters are calculated from motions of facial feature points as expression features. We extract short-term mean
energy, fundamental frequency and formant frequencies from each frame as speech features. An emotion classifier is designed to fuse
facial expression and speech based on Hidden Markov Models and Multi-layer Perceptron. Experiments indicate that multi-modal
fusion emotion recognition algorithm which is presented in this paper has relatively high recognition accuracy. The proposed approach
has better performance and robustness than methods using only video or audio separately. |
|
|
|
|
|
|
|
