Dynamic sign synthesis from a virtual character to support language teaching to deaf students

Μέγεθος: px

Εμφάνιση ξεκινά από τη σελίδα:

Download "Dynamic sign synthesis from a virtual character to support language teaching to deaf students"

Μένθη Αγγελίδης
7 χρόνια πριν
Προβολές:

1 Dynamic sign synthesis from a virtual character to support language teaching to deaf students Eleni Efthimiou eleni_e@ilsp.gr Supporting Technologies Group Institute of Language and Speech Processing

2 Language Technology Integrates human knowledge on languages with IT systems Allows for and aims to human-computer interaction in human-human terms

3 Natural Language Processing Grammar approach Statistical approach Combination of both methods

4 precondition: Language resources: > Syntactic and semantic dictionaries > Electronic corpus

5 Supporting Technologies in the framework of Language Technology Research and applications in Speech Technology -Speech synthesis -Speech recognition Research and applications in Natural Language Analysis and Processing -Sign language applications

6 Sign language modeling requires: Analysis of sign language data Creation of digital language resources Production of representations of signed syntactic patterns Resource coding wrt the phonological characteristics of sign languages

7 e-learning for all Demand for universal access to Information Society technologies Traditional activities in HCI should be redesigned to allow access from all users

8 Designing accessible educational platforms An accessible platform must provide HCI means such as: Speech synthesis and recognition Virtual signers Procedures for simplified keyboards and switchbased navigation to support all possible users Our implementation: develop technologies to allow a virtual signer to mediate between a teacher and a student to present GSL educational material

9 e-learning in GSL Avatar animation technologies with GSL language resources to develop and refurbish educational material Application: teach GSL grammar to young elementary school students The same platform allows deaf students to access written language material in the form of signed summaries Harmonized with Universal Access and Design for All requirements, in the framework of the Information Society program

10 Development characteristics Scripting: STEP (Scripting Technology for Embodied Persona) Advantage: separate gesture and sign definition from avatar geometry and hierarchy definition Result: sign definitions can be modified without the need to re-model the virtual character h-anim compliant Avatars Web page deployment maximizes accessibility The web page includes embedded VRML objects (avatar), STEP engine references and JavaScript function calls

11 Input: written Greek text converted to GSL signs and displayed on screen

12 Extensions include Mouthed phonemes Facial expressions Body movement which usually declare non-verbal information in SLs Multi-sing units may be composed of a) free signs b) combinations of free and bound signs to create sign sequences, according to the rules of either inflectional or derivational morphology, or to construct sign phrases

13 Language resources: - vocabulary (HamNoSys notation) - computational grammar (ELAN notation) Combination of resources representation of any given signed phrases

14 Greek-to-GSL conversion: an expandable tool Future work: Library extension to cater for non-manual structural components that will support signing performance of the avatar closer to natural signing Enrichment of the system s linguistic knowledge will also allow for improvement of its generative capacity New written texts can be launched, so the e-learning platform may receive unlimited educational content besides primary school grammar units From a socio-economic point of view, the existence of such a tool will contribute towards ensuring equal opportunities for the deaf population in Greece

15 Related international projects: VISICAST, SignSynth, esign, Thetos,

Σχετικά έγγραφα

ΓΗΑΠΑΝΔΠΗΣΖΜΗΑΚΟ ΓΗΑΣΜΖΜΑΣΗΚΟ ΠΡΟΓΡΑΜΜΑ ΜΔΣΑΠΣΤΥΗΑΚΧΝ ΠΟΤΓΧΝ ΣΔΥΝΟΛΟΓΗΔ ΣΖ ΠΛΖΡΟΦΟΡΗΑ ΚΑΗ ΣΖ ΔΠΗΚΟΗΝΧΝΗΑ ΓΗΑ ΣΖΝ ΔΚΠΑΗΓΔΤΖ ΓΙΠΛΧΜΑΣΙΚΗ ΔΡΓΑΙΑ

ΓΗΑΠΑΝΔΠΗΣΖΜΗΑΚΟ ΓΗΑΣΜΖΜΑΣΗΚΟ ΠΡΟΓΡΑΜΜΑ ΜΔΣΑΠΣΤΥΗΑΚΧΝ ΠΟΤΓΧΝ ΣΔΥΝΟΛΟΓΗΔ ΣΖ ΠΛΖΡΟΦΟΡΗΑ ΚΑΗ ΣΖ ΔΠΗΚΟΗΝΧΝΗΑ ΓΗΑ ΣΖΝ ΔΚΠΑΗΓΔΤΖ ΓΙΠΛΧΜΑΣΙΚΗ ΔΡΓΑΙΑ ΔΘΝΙΚΟ ΚΑΙ ΚΑΠΟΓΙΣΡΙΑΚΟ ΠΑΝΔΠΙΣΗΜΙΟ ΑΘΗΝΧΝ ΣΜΖΜΑ ΔΚΠΑΗΓΔΤΖ ΚΑΗ ΑΓΧΓΖ ΣΖΝ ΠΡΟΥΟΛΗΚΖ ΖΛΗΚΗΑ ΣΜΖΜΑ ΔΠΗΚΟΗΝΧΝΗΑ ΚΑΗ ΜΔΧΝ ΜΑΕΗΚΖ ΔΝΖΜΔΡΧΖ ΠΑΝΔΠΙΣΗΜΙΟ ΘΔΑΛΙΑ ΣΜΖΜΑ ΑΡΥΗΣΔΚΣΟΝΧΝ ΜΖΥΑΝΗΚΧΝ ΣΔΥΝΟΛΟΓΙΚΟ ΔΚΠΑΙΓΔΤΣΙΚΟ