Login New user?  
01-Applied Mathematics & Information Sciences
An International Journal
               
 
 
 
 
 
 
 
 
 
 
 

Content
 

Volumes > Volume 9 > No. 6

 
   

Multimodal Arabic Speech Recognition for Human-Robot Interaction Applications

PP: 2885-2897
Author(s)
Alaa Sagheer,
Abstract
By the earliest motivation of building humanoid robot to take care of human being in the daily life, the researches of robotics have been developed several systems over the recent decades. One of the challenges faces humanoid robots is its capability to achieve audio-visual speech communication with people, which is known as human-robot interaction (HRI). In this paper, we propose a novel multimodal speech recognition system can be used independently or to be combined with any humanoid robot. The system is multimodal since it includes audio speech module, visual speech module, face and mouth detection and user identification all in one framework runs on real time. In this framework, we use the Self Organizing Map (SOM) in feature extraction tasks and both the k-Nearest Neighbor and the Hidden Markov Model in feature recognition tasks. Results from experiments are undertaken on a novel Arabic database, developed by the author, includes 36 isolated words and 13 casual phrases gathered by 50 Arabic subjects. The experimental results show how the acoustic cue and the visual cue enhance each other to yield an effective audio-visual speech recognition (AVSR) system. The proposed AVSR system is simple, promising and effectively comparable with other reported systems.

  Home   About us   News   Journals   Conferences Contact us Copyright naturalspublishing.com. All Rights Reserved