面向残疾人的汉语可视语音数据库

将人机交互领域中研究的唇读技术应用于康复工程之中,设计了一个基于视觉语言的语音合成系统。该系统特别针对后天致残,丧失语音能力的人设计,采用了一种特定条件下的汉语可视语音数据库。不同于现有的数据库,该数据库的设计具有以下特点：采用了非对称唇形轮廓模型,提取了嘴唇突出度的信息;针对汉语音节的特点,增强了汉字音节中信息变化过程;兼顾未来唇读技术的发展,以音节为基本元素,具有可扩充性。采用运动检测和数学形态学的办法提取唇动图像序列中的唇形区域,并从中提取非对称唇形轮廓模型特征参数,同时通过计算部分参数对时间的差分,来获得唇形轮廓的动态信息。基于隐马尔可夫模型的学习和识别实验表明,该数据库的设计方法合理,所选的唇动特征用能够将识别效果平均提高25%。

Mandarin Chinese Visual-speech Database for Speech-impaired People

A speech synthesis system based on lip-reading technique was investigated for rehabilitation;it presented a new communication approach for the speech-impaired people by using visual information only to synthesize their acoustic speech.In this system,a mandarin Chinese visual-speech database was designed for mute people.Different from existing visual-speech databases,our database was designed directly for disabled people.It has some specialties as follows:Unsymmetrical lip contour model was used to extract the information of pouting;Chinese pronunciation of vowel and consonant was enhanced to improve the dynamic process;considering the developing of lip-reading techniques,this database was fit for expanding.Movement detection and morphological processing were used to extract mouth area and parameters of lip contours from the image sequence.At the same time,the differential coefficients of some parameters were calculated to describe dynamic characteristic of the lip contours.Training and recognizing experiments based on Hidden Markov Model showed that the database was feasible;and the selected parameters improved the recognizing rate by more than 25%.