Εργαστήριο Υπολογιστικής Ευφυΐας. Παρουσίαση τρεχόντων έργων Ε&ΤΑ

Transcript

1 Εργαστήριο Υπολογιστικής Ευφυΐας Παρουσίαση τρεχόντων έργων Ε&ΤΑ

2 Τεχνολογίες : Δραστηριότητες Επεξεργασία και σημασιολογική επισημείωση εικόνας-vide Επεξεργασία ήχου και ομιλίας Επεξεργασία 3-D γραφικών Σύντηξη πολυμεσικών δεδομένων Μηχανική μάθηση Πρόσφατες εφαρμογές: Επεξεργασία και αναγνώριση εγγράφων Αναγνώριση ανθρώπινης συμπεριφοράς Αυτόματη ημερολογοποίηση ομιλητών Αναγνώριση συναισθήματος από εικόνα και ήχο Παρακολούθηση Φυσιολογικών Παραμέτρων από βίντεο (ambient assistive living) Εφαρμογές ανάλυσης εικόνων για remte sensing

3 : Ανθρώπινο δυναμικό Ερευνητές Δρ. Σ. Περαντώνης (Ερευνητής A ) Δρ. Β. Γάτος (Ερευνητής A ) Δρ. Ε. Χάρου (Ερευνήτρια Γ ) Συνεργαζόμενοι Δρ. Σ. Πετρίδης Δρ. Α. Κεσίδης Δρ. Γ. Λουλούδης Δρ. Θ. Γιαννακόπουλος Δρ. N. Σταματόπουλος Υποψήφιοι Διδάκτορες Αλ. Παπανδρέου Π. Δολιώτης Θ. Κονιδάρης Κ. Ντιρογιάννης Α. Μπολοβίνου

4 Dcument Image Prcessing at CIL

5 TRANSCRIPTORIUM

6 The TRANSCRIPTORIUM prject aims t develp innvative, efficient and cst effective slutins fr the indexing, search and full transcriptin f histrical handwritten dcument images, using mdern, hlistic Handwritten Text Recgnitin (HTR) technlgy. High quality handwritten text transcriptin will be apprached by means f interactive techniques which minimize user interventin. The resulting technlgies will be implemented bth in a cntent prvider prtal and in a specialized HTR web prtal fr crwd-surcing transcriptin.

7 Started: 1/1/2013 Ends: 31/12/2015

8 Wrk Packages WP1. Management (UPV) The verall bjective f this WP is t ensure the fulfillment f the prject bjectives and t ensure that the required prject deliverables are prvided t the Eurpean Cmmissin in time and within budget. This WP will prceed in clse cntact with WP7, respnsible fr the Disseminatin and Explitatin Plans. WP2. Data management (UIBK) The verall bjective f this WP is t define the data t be prcessed in transcriptrium, t cllect this data and t maintain the necessary infrastructure fr internal sharing f data. We will fcus mainly n dcuments with cursive handwriting and with available metadata r ther linguistic resurces that are useful fr the Handwritten Text Recgnitin (HTR) prcess. We will fcus n dcuments in fur languages: English, Dutch, German and Spanish. WP3. Fundamental research fr HTR and DIA (NCSR) In this WP, we will strive twards the develpment f innvative methdlgies in HTR and DIA in rder t prvide the cre elements f an HTR engine. Our aim is t g beynd state-f-the-art techniques in DIA in rder t efficiently enhance the quality and segment histrical handwritten dcuments in text line as well as t prepare the necessary HTR mdels which will be used in the subsequent recgnitin stages. Mrever, we will develp techniques fr indexing and searching in histrical handwritten dcuments based n KWS.

9 Wrk Packages (cnt.) WP4. Linguistic resurces fr HTR (INL) The aim f this wrk package will be t prvide a general apprach and wrkflw t acquisitin and integratin f linguistic resurces, t cllect linguistic resurces relevant t the manuscript cllectins tackled in the prject and t ptimize language mdeling fr HTR. The cntributin f the develpments f this WP fr imprving HTR results will be explicitly evaluated in WP3 and WP5. WP5. Integratin and interactin (UPV) This WP aims at integrating the tls develped in WP3 and the linguistic resurces generated in WP4 in a set f tls fr further develping web-based services. An API will be develped fr later use in the user interfaces that will be implemented in WP6. Interactive techniques will be studied fr imprving the recgnitin results f the DIA, HTR, and KWS engines and fr reducing the user effrt fr achieving accurate results. WP6.User interfaces (UCL) The gal f this WP is t demnstrate that it is feasible t utilise the DIA, HTR and KWS sftware prduced by the prject (a) fr crwdsurcing applicatins and (b) fr digital archives and e-research prtals. WP7. Disseminatin and explitatin (UPV) This WP is devted t the disseminatin f transcriptrium scientific and technlgical results in the areas f Cultural Resurces, as well as in the Digital Libraries. Als, a main bjective f this WP is t develp an explitatin plan fr the develpment f new services n the basis f transcriptrium results.

10

11 A cmplete grund truth generated frm supervised anntatins (layut analysis and transcripts) will be btained and maintained in this task fr training and evaluatin. Fr the cases that the transcripts f the handwritten dcuments are already available, efficient transcript mapping techniques will be invlved in rder t map the crrect text infrmatin t the segmentatin results prduced in the fllwing T3.2 task. The definitin f the type f anntatins and f standard grund truth frmats, used by the HTR platfrm fr evaluatin purpses, will be carried ut in this task. In additin, this task will set up a wrkflw hw already anntated material can be transfrmed int the previusly defined grund truth frmats. Sme f the anntatin prcesses will be subcntracted. Quality cntrl f the subcntracted anntatin will be defined in this task.

12 New rbust methds will be develped in rder t (i) prcess clr r grayscale histrical handwritten dcuments s that the fregrund (regins f handwritten text) is separated frm the backgrund (paper), (ii) eliminate the presence f unwanted and nisy regins as well as t enhance the quality f text regins, (iii) crrect dminant page skew and (iv) identify and crrect character slant. Fr the needs f this task, we will develp nvel techniques based n the estimatin f the backgrund surface f the handwritten dcument, n the adaptatin f efficient techniques (e.g. prjectin prfiles, crss-crrelatin) that have been already prpsed fr machine written dcuments as well as n the efficient cmbinatin f multiple techniques. Special fcus will be given n the challenging characteristics f the handwritten dcuments which include lw cntrast and uneven backgrund illuminatin, bleed thrugh and shining r shadw thrugh effects.

13

14 In rder t achieve accurate HTR perfrmance, a rbust and efficient segmentatin task must be invlved. In this task, we will build a hierarchical segmentatin mdel that cnsists f three levels. The first level is dedicated t the detectin and separatin f handwritten text blcks in the dcument. A dcument structure and layut analysis will be invlved in rder t detect the spatial psitin f text infrmatin. In the secnd level, text lines will be identified while the third level will invlve the segmentatin f each text line int wrds. We will adapt existing techniques such as the use f Hugh Transfrm r adaptive run length smthing fr the specific needs f the histrical handwritten dcuments. Challenges that will be met cncern the lcal skew and nise that culdn't be eliminated at the previus task, the existence f adjacent text lines tuching as well as the nn-unifrm spacing amng text areas. The text line segmentatin result f this task will be used as input t the fllwing T3.3 task while the wrd segmentatin result will be used fr the needs f task T3.4.

15

16 We will explre a nvel query-by-string KWS apprach which capitalizes n the wrd graphs btained frm a full-fledged HTR decding prcess. This apprach takes advantage f the reach wrd cntextual infrmatin derived frm the basic HTR prcess and can achieve accurate sptting results with very lw search cmputatinal requirements. Hwever, due t the great amunt f variability in handwriting, t the nisy backgrund, t the dense and arbitrary layut as well as t the existence f strngly cnnected characters, the applicatin f HTR techniques may nt be feasible fr sme histrical handwritten dcuments. Therefre, in this task we will als develp an alternative indexing technique based n sptting wrds directly n dcument images with the help f wrd matching, while aviding cnventinal HTR prcedures. In ur apprach, we will fllw three different paths which will be unified at the end. In particular, we will cnsider nt nly wrd sptting that take int accunt handwritten dcuments that have been previusly segmented int wrds and whle text lines but we will als cnsider a segmentatin-free apprach. The resulting ranked lists frm all three independent wrd sptting appraches will be treated as different mdalities fr which a late fusin algrithm will merge them int ne cherent final result.

17 Current Industrial Cntracts 1. University Innsbruck, Department fr German Language and Literature, Austria Παροχή εξειδικευμένων ερευνητικών υπηρεσιών για την δυαδική μετατροπή και την διόρθωση της στροφής ιστορικών εφημερίδων Προϋπολογισμός: Ευρώ, Διάρκεια: 1/11/ /12/ GLOBO Technlgies, Greece Σειρά βιβλιοθηκών λογισμικού επεξεργασίας των εγγράφων (Δυαδική μετατροπή, αυτόματος διαχωρισμός σελίδων εγγράφων, αφαίρεση περιθωρίου, διόρθωση της στροφής και της τοπικής καμπυλότητας, αφαίρεση θορύβου, διόρθωση ανάλυσης ) Προϋπολογισμός: Ευρώ πλέον ΦΠΑ, Διάρκεια: 1/1/ /2/ TPG Rewards Inc, USA Λογισμικό αναγνώρισης και εντοπισμού προϊόντος σε αποδείξεις. (σε διαδικασία υπογραφής συμφωνητικού ανάθεσης εργασίας)

18 USEFIL Unbtrusive Smart Envirnments fr Independent Living

19 USEFIL Outline 1. Overview Facts Aim and bjectives Supprting Infrastructure Research Challenges Privacy Issues Scial Awareness 1. Mnitring Functinal Status Mnitring Emtinal Status Mnitring Physilgical Status Mnitring i. Mnitring Face PPG ii. Mnitring Pupils

20 USEFIL Facts Unbstrusive Smart Envirnments Fr Independent Living Objective FP7-ICT Budget NCSR Budget 4,650,000 Eurs Eurs Start Nvember 2011 Duratin 3 years Crdinatr NCSR Demkrits

21 USEFIL Aim and Objectives elderly peple living independently fr a lnger time, feeling safe and scially cnnected Everyday silent mnitringf their functinal, emtinal and physilgical status enabling Scial Awarenessthrugh easy t use intuitive interfaces inter-cnnecting dctrs, carers and familythrugh web interfaces

22 USEFIL Supprting Infrastructure Interfaces a Slate-tablet PC a Web-TV (TP-Visin) in the living rm Sensrs a wrist watch a Depth camera (living rm) mic and camera n the Slatetablet PC mic and camera behind a mirrr Prcessing Pwer a nettp (e.g.zbx ID80)

23 USEFIL Research Challenges (NCSR) What Hw analysing uncnstrained audivisual infrmatin f activities within the huse. recgnise everyday activities thrugh audi and vide cues btain physilgical indexes thrugh face mnitring fuse infrmatin frm different sensrs (audi, vide, accelermeter) nn-btrusiveness: the huse shuld nt be verladed with sensrs lw-cst: USEFIL is targeting a wide ppulatin - devices and sensrs shuld nt be specialised, prcessing pwer is limited real-time: respecting privacy - mnitring indicatrs are evaluated directly - n riginal data is recrded use mstly unsupervised methds (calibratin pssible)

24 USEFIL Privacy issues New prblem: Users may be afraid that their everyday life as well as their health recrds is publicly "expsed" Apprach: Data stays inside the hme envirnment, whenever pssible Raw data is never stred (e.g. vide ftage) Establish a trustwrthy relatinship: user - health establishment. Clearly determine which peple are authrised t view which data. Use privacy-enabling technlgies Authenticatin (PKI), secure data strage (TrueCrypt) and secure cmmunicatins (SSL)

25 USEFIL Scial Awareness The elderly and relatives can cntinuusly check each thers state where they are what are they ding chatting is pssible examples: my father is nw sleeping / eating / bathing my daughter is nw driving

26 USEFIL Outline 1. Overview Facts Aim and bjectives Supprting Infrastructure Research Challenges Privacy Issues Scial Awareness 1. Mnitring Functinal Status Mnitring Emtinal Status Mnitring Physilgical Status Mnitring i. Mnitring Face PPG ii. Mnitring Pupils

27 USEFIL Functinal Status Mnitring (1) Activities f Daily Living 1. Mving 2. Eating 3. Bathing 4. Grming 5. Dressing 6. Using the WC + frailty / fatigue Depth Camera Task: Detect Dependent vs Independente.g. can brush his teeth alne Degree f independence [1..5] e.g. dressing ability: 2

28 USEFIL Mnitring Fci Three Directins f mnitring 1. Functinal Status Hw well can the elderly perfrm activities f daily living independently 1. Emtinal Status Hw des the elderly feel when living alne 1. Physilgical Status What are his health signs (e.g. bld pressure)

29 USEFIL Functinal Status Mnitring (2) The bathrm scenari Bathing Grming Using the WC Sensr: Micrphne Use audi cues t detect washing hands washing teeth bathing using the WC (flushing the water) On-ging develpment Machine-learning supervised algrithms fr classifying audi statinary sunds HMM mdels t capture user scenaris (e.g. clse the dr - using the WC - pen the dr)

30 USEFIL Emtinal Status Mnitring The Depressin scenari Fuse indirect cues t infer depressin audivisual cues

31 USEFIL Physilgical Status Mnitring Physilgical Indexes 1. Heart Rate + 2. Breathing Rate 3. Pupil size + 4. Face Clr +... (Ideally) Bld Pressure Task: Lng term mnitringfcus n trends rather than accuracy User's face is facing a camera tablet pc, camera behind the mirrr

32 USEFIL Mnitring Face PPG (1) Targets: heart rate, respiratin rate bld xygenatin and thers... stress / sedatin, gastric mtility respiratry sinus arrhythmia Cntact methds exist fr ver 50 years Phtplethysmgraphy Principle: using a light surce and a detectr and bserving skin absrbance r reflectance t light, we may measure physical cnditin related indexes

33 USEFIL Mnitring Face PPG (2) Unbtrusive Remte PPG Mnitring Main Issue: Nisy signal Light is nt cntrlled Camera is distant Subject is mving Apprach Adaptiveness t lighting cnditins Mtin tlerance Face tracking / nrmalisatin fast algrithms (GPGPU)

34 USEFIL Mnitring Face PPG (3) FACE PULSE Dataset Reference set f ~75 peple Pulse Oxymeter reference data Variance in cnditins recrding lighting subject mtin Onging Research: Dem Vide

35 USEFIL Mnitring Pupils (1) Pupil sizes relates t: emtinal state fatigue aging and pathlgical cases Research Challenge: Lw-cst pupilmetry using visible-spectrum, distant, lw-cst camera Aniscria Example

36 USEFIL Mnitring Pupils (2) CIL Pupils Dataset iris and pupil anntatin bth center and radius ~10 humans 3 anntatrs fr inter-anntatr agreement

37 USEFIL Mnitring Pupils (3) Irises and pupils are detected by maximizing a measure stemming frm a filtering prcedure with a set f disk masks Preliminary Results Precisi n Recal l F1 i-a.a methd

38 ORTHOEMAN A web-based e-training platfrm fr Extended Human Mtin Investigatin in Orthpedics

39 ORTHOEMAN Framewrk: LdV Transfer f Innvatin Bugdet: Eurs NCSR budget: Eurs Partners The prject cnsrtium is frmed by higher educatin institutins (2 universities, frm Rmania and Greece), 2 research centres (frm Greece and Spain), an emergency hspital (frm Rmania), all with great experience fr bth aspects (rthpedy and bimedical engineering), with demnstrated skills, recgnized expertise and cmpetence required t carry ut all aspects f the prpsed prject. P0 - UNIVERSITY OF CRAIOVA (ROMANIA) - UCV P1 - NATIONAL CENTER FOR SCIENTIFIC RESEARCH DEMOKRITOS (GREECE) - NCSR P2 - BIOMECHANICS INSTITUTE OF VALENCIA (SPAIN) IBV P3 - CLINICAL EMERGENCY HOSPITAL BUCHAREST (ROMANIA) SCUB P4 - DEMOCRITUS UNIVERSITY OF THRACE (GREECE) - DUTH

40 The prject aims t clse the gap between engineering and medicine, by develpment f a cmprehensive interdisciplinary virtual training system fr human mtin analysis and rthpedics (Orth-eMAN), by: - transferring and elabratin f a web-based platfrm that allws ubiquitus accessibility and platfrm independence; - ffering t rthpedic dctrs and engineers interested in medical field, a cmmn learning tl, with interdisciplinary appraches using learning methdlgies and experience frm previus EU prject EMEDI; - develpment f a strng curriculum that will allw efficient training; - the tagging f multimdal and interdisciplinary cntent apprpriate fr the cmpletin f realistic cases; - ffering a mre unified EU quality standard in biengineering and bimedicine. All these aspects will cntribute t acquisitin f key cmpetences in bimedical VET fr the target grup.

41 EMEDI Interactive e-training envirnment

42

43 AMINESS Ανάλυση Ναυτιλιακής Πληροφορίας για Περιβαλλοντικά Ασφαλή Ναυσιπλοΐα

44 Κεντρική ιδέα - Στόχοι Συνεχής παρακολούθηση από τις Ελληνικές Αρχές: μόνο σε συγκεκριμένες περιοχές προτεραιότητας Νομικοί περιορισμοί: μη επιβολή ναυτιλιακών διαδρομών Παραμέληση κινδύνων από τις ναυτιλιακές εταιρίες κατά την χάραξη πορείας Τρεις βασικοί στόχοι: 1. Εξαγωγή βέλτιστης διαδρομής με κριτήρια περιβαλλοντικης ασφάλειας και ασφάλειας πλεύσης 2. Εξαγωγή ειδοποιήσεων σε πραγματικό χρόνο για κινδύνους σχετικα με άλλα πλοία και σε συνάρτηση διαφόρων παραμέτρων (θέση, καιρός κτλ) 3. Εξαγωγή συμπερασμάτων που θα βοηθήσουν σε συστάσεις πολιτικής μέσω της βραχυπρόθεσμης και μακροπρόθεσμης ανάλυσης ιστορικών δεδομένων

45 Εταίροι

46 Εταίροι ΦΟΡΕΑΣ ΣΥΝΤΟΜ. ΕΠΩΝΥΜΙΑΣ ΕΙΔΟΣ ΦΟΡΕΑ[1] ΠΡΟΫΠΟΛΟΓΙΣΜΟΣ ( ) ΔΗΜΟΣΙΑ ΔΑΠΑΝΗ ( ) ΔΗΜΟΣΙΑ ΔΑΠΑΝΗ (%) 1 NCSRD ΕΦ ,00 2 UPRC ΕΦ ,00 3 UAEGEAN ΕΦ ,00 4 ARCHIPELAGOS ΛΦ ,00 5 IMIS ΕΠ ,71 6 DANAOS ΕΠ ,68 7 ASL ΕΠ ,80 ΣΥΝΟΛΟ

47 Μέθοδος

48 Απτά Αποτελέσματα Σχεδιασμός και υλοποίηση βάσης δεδομένων Εργαλείο επιλογής διαδρομής ελάχιστου κινδύνου Εργαλείο βραχυπρόθεσμης εκτίμησης αυξημένου κινδύνου Εργαλείο αυτόματης σύστασης πολιτικής Web prtal σχεδιασμός πορείας ειδοποιήσεις σε πραγματικό χρόνο ανάλυση δεδομένων και προσομοιώσεις συστάσεις