The Learning Data Management of the Utterance Learning System Using Lip Movements Recognition
- Learning Data Management Using SQLite and Feedback to a Learner -
Abstract
We previously proposed an utterance recognition system that uses the tracking and analysis of speakers' lip movements as words and sentences are uttered. Also, we compared the lip movements in the SQLite database with those of utterances produced by students learning the method. We accumulated lips movement history of the utterance learner in the conventional utterance learning system and added a function to manage the change of the lips movement of the learner. In this report, we describe development and the result of the system.
References
Wakamatsu E, Hoshino Y, Yamada M. Proposal for an utterance training method based on lip movements. IMQA2014 (The Seventh International Workshop on Image Media Quality and its Applications), September 2014, Chiba, Japan. pp. 44-47.
Hayamizu S, Takezawa T. Trends in research on multimodal information integration system. Trans Jpn Soc Artif Intell. 2006:13(2):206-211, Mar.1998. (in Japanese)
Nagata Y, Fujioka T, Abe M. Target signal detection system using two directional microphones. IEICE Trans A. 2000:J83-A(12):1445-1454. (in Japanese)
Yanagi T, Yamada M. Proposal and verification of lip movement model about utterance recognition interface without voice. HIS2008 (8th International Conference on Hybrid Intelligent Systems), Sept. 2008, p. 2422. (in Japanese)
Oda M, Ichinose S, Oda S. Development of a pronunciation practice CAI system based on lip reading techniques for deaf children. Technical Report of IEICE. 2007:107(179):53-58. (in Japanese)
Y.Hoshino, E.Wakamatsu, M.Yamada, "The Utility of the SQLite Database for An Utterance Training Method Based On Lip Movements", Information Engineering Express International Institute of Applied InformatiCSVol. 1, No. 1, pp.1-9, 2015
K.Sung, T.Poggio, "Example - based learning for view - based face detection", IEEE Patt. Anal. Mach. Intell., volume 20, pp.39-51, 1998.
H.Rowley, S.Baluja, T.kanade, "Neural network - based face detection", IEEE Patt. Anal. Mach. Intell., volume 20, pp.22-38, 1998.
H.Schneiderman, T.kanade, "A statistical method for 3D object detection applied to face and cars", International Conference on Computer Vision, 2000.
D.Roth, M.Yang, N.Ahuja, "A snowbased face detector",Neural Information Processing12, 2000.
Paul Viola, Mihael Jones, "Robust Real-Time Object Detection", Second Internatinal Workshop on Statistical and Computational Theories of Vision - Modeling, Learning, Computing and Sampling, pp.1-25, Vancouver, 2001 .
S.Saito, R.Konishi, "Lip Reading based on Trajectory Feature of Lip and Mouth Cavity Regions", the Institute of Electronics, Information and Communication Engineers Information and Systems Society, Human Communication, J90-D, 4, pp.1105-1114, 2007 (in Japanese)
M.Ishii, K.Sato, M.Nishida, Y.Kageyama, "Feature Extraction and Analysis for Lip Reading in Continuous Image", The transactions of the Institute of Electrical Engineers of Japan. D, A publication of Industry Applications Society 119(4), pp.465-472, 1999. (in Japanese)
T.Otsuki, T.Ohtomo, "Automatic Lipreading of Station Names Using Optical Flow and HMM", Institute of Electronics, Information, and Communication Engineers, Technical Report of IEICE PRMU, Vol.102, No.471, pp.25-30, 2002 (in Japanese)
T.Yoshida, K Fukushi, S.Hangai, "Development of lip movement sensor and its application to spoken word recognition", Institute of Electronics, Information, and Communication Engineers, FIT2006, pp.125-126, 2006. (in Japanese) .
Fukushima M. Easy training for a good voice. Seibido Publishing, Tokyo. 2006. in Japanese
Hirayama K. Dandelion. First Volume of Revised New National Language Two, Tokyo Shoseki, Jan. 1985. (in Japanese)
Three-month topic English-conversation. NHK publication, 2009. (in Japanese)
