Sign language animation using TVML

The deaf communities in Japan use a visual language called Japanese Sign Language (JSL) as their first language. Learning to read Japanese can be difficult for some deaf people, because JSL has a totally different grammar from Japanese. NHK broadcasts news and other programs in sign language (sign language broadcasts), but there is a strong demand from deaf communities for more sign language broadcasts. This motivated us to pursue technology for generating sign language animation. Our recent work has focused on adapting TVML (TV program Making Language) [Hayashi 1996; TVML Website] for use in producing sign language animation. TVML is a text-based computer language that produces computer graphics (CG) animated TV programs by simply writing a script in TVML. At first, we recorded body movements representing sign language words using an optical motion capture technology. In addition, we improved TVML to express manual movements of sign language and developed a Japanese-to-JSL dictionary with a collection of about 4,900 sign language words. Now in this latest work, we have developed a scheme for combining multiple captured motion data to generate sign language sentences using CG animation.