About AMP Lab        Projects        Downloads        Publications        People        Links

Audio-Visual Speech Processing

Speech production and perception is inherently bimodal. Of late there has been increased interest in using the visual modality in combination with the normally used acoustic modality for improved speech processing. This field of study has gained the title of audio-visual speech processing (AVSP). Traditional acoustic based speech processing systems have attained a high level of performance in recent years, but the performance of these systems is heavily dependent on a match between train and test conditions. In the presence of mismatched conditions (i.e. acoustic noise) the performance of acoustic speech processing applications can degrade markedly. The visual speech modality is independent to most possible degradations in the acoustic modality. This independence, along with the bimodal nature of speech, naturally allows the visual speech modality to act in a complementary capacity to the acoustic speech modality. It is hoped that the integration of these two speech modalities will aid in the creation of more robust and effective speech processing applications in the future.

 

Our research effort is concentrated  on speech and speaker recognition. In particular we have been conducting active research for the visual speech modality concerning feature extraction and classifier design. The problem of audio-visual integration is also an active component of our effort.

(Integration strategies in AVSP)

Related papers:-