ΕΝΤΥΠΑ ΥΠΟΒΟΛΗΣ ΕΝΔΙΑΜΕΣΗΣ ΕΚΘΕΣΗΣ ΠΡΟΟΔΟΥ ΕΡΕΥΝΗΤΙΚΟΥ ΕΡΓΟΥ ΤΗΣ ΔΕΣΜΗΣ 2008

Transcript

1 1 ΕΝΤΥΠΑ ΥΠΟΒΟΛΗΣ ΕΝΔΙΑΜΕΣΗΣ ΕΚΘΕΣΗΣ ΠΡΟΟΔΟΥ ΕΡΕΥΝΗΤΙΚΟΥ ΕΡΓΟΥ ΤΗΣ ΔΕΣΜΗΣ 2008 Η Ενδιάμεση Έκθεση, υποβάλλεται σε δύο αντίγραφα, μέχρι και δύο μήνες μετά τη συμπλήρωση της μισής διάρκειας του χρόνου υλοποίησης του Έργου. Η Ενδιάμεση Έκθεση περιλαμβάνει δύο μέρη: ΜΕΡΟΣ Α - Ενδιάμεση Έκθεση Προόδου Α.1. «Γενικά Στοιχεία Έργου» Α.2. «Περίληψη» έκτασης 500 λέξεων για τη μέχρι στιγμής πορεία υλοποίησης του Έργου. Α.3. «Ενδιάμεση Έκθεση Υλοποίησης Έργου» Περιλαμβάνει τη συμπλήρωση ενός τυποποιημένου Δελτίου Αναφοράς για κάθε ΔΕ του Έργου. Α.4. «Συνοπτικός Πίνακας Δεσμών Εργασίας» όπου φαίνεται η πορεία υλοποίησης των Δεσμών Εργασίας και τα Παραδοτέα που προέκυψαν από κάθε μία απ αυτές. ΜΕΡΟΣ Β Παραρτήματα Β.1. Στο «Παράρτημα Β1» της έκθεσης επισυνάπτονται τα παραδοτέα του έργου που μπορούν να δοθούν σε έντυπη μορφή. Β.2. Στο «Παράρτημα Β2» της έκθεσης επισυνάπτονται οποιεσδήποτε άλλες πληροφορίες αναφορικά με το έργο κρίνονται απαραίτητες. Η Ενδιάμεση Έκθεση Οικονομικών Πεπραγμένων υποβάλλεται σε ξεχωριστά έντυπα που είναι διαθέσιμα στον Ιστοχώρο του Ιδρύματος Προώθησης Έρευνας σε μορφή αρχείου Excel.

2 2 Μ Ε Ρ Ο Σ Α Α.1. ΓΕΝΙΚΑ ΣΤΟΙΧΕΙΑ ΕΡΓΟΥ Επιχειρησιακό Πρόγραμμα Άξονας Προτεραιότητας Πρόγραμμα Δράση Αριθμός Πρωτοκόλλου Έργου Τίτλος Έργου Ανάδοχος Φορέας Συντονιστής Έργου Αειφόρος Ανάπτυξη και Ανταγωνιστικότητα Κοινωνία της Γνώσης και Καινοτομία ΤΕΧΝΟΛΟΓΙΕΣ ΠΛΗΡΟΦΟΡΙΚΗΣ ΚΑΙ ΕΠΙΚΟΙΝΩΝΙΩΝ ΤΕΧΝΟΛΟΓΙΕΣ ΠΛΗΡΟΦΟΡΙΚΗΣ ΤΠΕ/ΠΛΗΡΟ/0308(ΒΙΕ)/03 Synthesis of Dynamic Characters with Motion Capture Data for Human Figure Animation: Educating the Police Force Frederick Research Center (FRC) Dr. Stephania Loizidou Himona Ημερομηνία Έναρξης Έργου 01/12/2009 Ημερομηνία Λήξης Έργου 30/11/2011 Ημερομηνία Υποβολής Έκθεσης 30/11/2010 Εγκεκριμένη Επιχορήγηση: Ποσό που καταβλήθηκε από το ΙΠΕ: (μέχρι στιγμής) Ποσό που δαπανήθηκε: (μέχρι στιγμής) 123, ΕΥΡΩ 43, ΕΥΡΩ 48, ΕΥΡΩ Στοιχεία επικοινωνίας με τον Συντονιστή Έργου: Διεύθυνση: 7-9 Filokiprou, Palouriotissa, 1036 Nicosia Τηλέφωνα: , Τηλεομοιότυπο: Ηλεκτρονικό ταχυδρομείο:

3 3 Α.2. «ΠΕΡΙΛΗΨΗ» (500 λέξεις - 2 σελίδες) Having completed one year towards the running of the research project Synthesis of Dynamic Characters with Motion Capture Human Figure Animation: Educating the Cyprus Police Force we are in the content situation to claim that the work progresses according to the initial frameworked plan. This claim is based firstly on the time schedule that has been met almost completely, that is, the deliverables that have been completed on time as well as the general progress which resulted from the smooth running and coordination of the project. The proposed work s major objective is the development of a software tool that will help the police force in training its officers on how to deal with situations of their everyday patrol excursions. The tool will enable the police officers to practice how to engage suspects or criminals simulated within a 3D virtual environment, without putting themselves in danger. For the development of the project the first thing that has been done was the creation of a website portal that helps not only for disseminating the results achieved but also for communication purposes between the project s partners. Since the commencement of the project our major objective has been the development of the Hybrid animation method that is, the combination of the Motion Capture data together with the Dynamics of Motion as originally specified in the proposal. Extensive study in the state of the art work has been performed mainly by an exhaustive review of the latest literature e.g. research articles. These let us identify the scientific advances in the area of computer animation in recent years, but also enabled us to recognize which research topics are currently being sought by the specific scientific community. In particular, we can now utilize both methods involved (motion capture and dynamics of motion) in isolation and we have started working towards devising of the Hybrid method, which will enable us to bring them together. We have also proceeded with drafting out the software design that will be used for implementing the software components that will comprise the training platform. The platform in its current state reflects our knowledge of the initial main user requirements as these have been identified through extensive communication and consultation with the police force. In addition, it allows for the implementation of desirable features that should be included, it enables the integration and support of any necessary hardware devices and it also sets the foundation for all of its future development. Having described all of the above and taking into consideration the initial project s proposal we are now ready to proceed towards the second, final stage of the work, that is, the completion of the training of the police officers tool using the latest scientific advances.

4 4 Α.3. «ΕΝΔΙΑΜΕΣΗ ΕΚΘΕΣΗ ΥΛΟΠΟΙΗΣΗΣ ΕΡΓΟΥ» (μέχρι 1500 λέξεις - 5 σελίδες ανά ΔΕ) Τίτλος Δέσμης Εργασίας ΔΕ1: Project Management Κωδικός Φορέα ΑΦ ΣΦ1 ΣΦ2 ΣΦ3 ΣΦ4 Ανθρωπομήνες για κάθε Φορέα (με βάση το Συμβόλαιο) Ανθρωπομήνες για κάθε Φορέα (δεδουλευμένοι) Στόχοι Δέσμης Εργασίας Αναφέρονται επιγραμματικά οι στόχοι της παρούσας Δέσμης Εργασίας. This work package is mostly concerned with the coordination of the project. That is, the monitoring of the smooth running of the program as well as the checking of the delivery of the expected work. Additionally, it deals with the management of the financial resource (budget) and its proper allocation. WP1 is performed throughout the duration of the proposed project, mainly by the host organisation with extra support from PA1. Περιγραφή Εργασίας Καταγράφονται οι δραστηριότητες που αφορούν στη Διαχείριση του Έργου, τυχόν προβλήματα που προέκυψαν στα πλαίσια του Συντονισμού του Δικτύου Συνεργασίας και πως αυτά επιλύθηκαν. This work package is responsible for the smooth coordination and management of the project. The coordination involves the communication between all the team members through both physical weekly meetings as well as the use of the technology for fast communication (e.g. through the projects website, and teleconferencing). The management involves setting up the criteria and standards of the work, the supervision of the team and the management of the crises. Specifically, it is responsible for the preparation as well as the dissemination of the deliverables as expected and on time. Analytically, the management team so far was responsible for the preparation of the 6-month progress report, the preparation of the current interim progress report, as well as the monitoring of the development of the software (D6); the hybrid system for motion control. This Hybrid system involves the utilization of the Dynamics of Motion together with the Motion Capture data through the use of the Motion Graphs (see WP2). Παραδοτέα

5 5 Αναφέρονται τα Παραδοτέα που προέκυψαν μέσα από τη συγκεκριμένη Δέσμη Εργασίας. Στα πλαίσια της Δέσμης Εργασίας 1, ως παραδοτέα περιλαμβάνονται οι εκθέσεις προόδου (τριμηνιαίες / εξαμηνιαίες, Ενδιάμεση και Τελική Έκθεση Προόδου, συναντήσεις ομάδας εργασίας κοκ) που θα πρέπει να υποβληθούν στο ΙΠΕ κατά τη διάρκεια του Έργου. 1. D1 6-month progress report (already submitted to the RPF on the 31 st May 2010) 2. D2 Interim progress report (current) 3. Meeting minutes - sample (see Appendix 2 - Παράρτημα Β2) 4. Figures (see Appendix 2 - Παράρτημα Β2) 5. References (see Appendix 2 - Παράρτημα Β2) 6. Timesheets Τίτλος Δέσμης Εργασίας ΔΕ2: Dissemination and Exploitation of Results Κωδικός Φορέα ΑΦ ΣΦ1 ΣΦ2 ΣΦ3 ΣΦ4 Ανθρωπομήνες για κάθε Φορέα (με βάση το Συμβόλαιο) Ανθρωπομήνες για κάθε Φορέα (δεδουλευμένοι) Στόχοι Δέσμης Εργασίας (όπως περιγράφονται στο Συμβόλαιο) Αναφέρονται επιγραμματικά οι στόχοι της παρούσας Δέσμης Εργασίας The main aim of this work package is the dissemination of the results that emerge from the proposed research project. Περιγραφή Εργασίας- Βαθμός Υλοποίησης των στόχων της Δέσμης Εργασίας Καταγράφονται οι δραστηριότητες που εντάσσονται στη Διάχυση και Εκμετάλλευση Αποτελεσμάτων. Γίνεται εκτενής αναφορά στο βαθμό υλοποίησής τους, σε πιθανά προβλήματα που προέκυψαν και σε τυχόν αποκλίσεις από τους αρχικούς στόχους. Όπου εφαρμόζεται, γίνονται αριθμητικές και ποσοτικές αναφορές στα επί μέρους στάδια της ΔΕ και γίνεται, επίσης, σαφής προσδιορισμός στο Φορέα που ανάλαβε και διεκπεραίωσε την κάθε δραστηριότητα. A website portal for the project has been implemented: ( This has been used not only for informing the general public of the project s progress so far, but also as a means of communication between all the parties in the consortium involved. The website is continuously updated to reflect the current status of the project at any given time. This website has been developed mainly by an independent freelancer (web-designer) who has been paid for his work. PA3 took over the tasks of maintaining and updating the website as required. Moreover, a URL related to the host organisation: is also linked to the website s current location that makes it easier to be found / accessed by the academic community (HO) via popular search engines in the WWW.

6 6 Παραδοτέα Αναφέρονται επιγραμματικά τα Παραδοτέα που προέκυψαν 0μέσα από τη συγκεκριμένη Δέσμη Εργασίας. Στα πλαίσια της Δέσμης Εργασίας 2, ως παραδοτέα περιλαμβάνονται οι δημοσιεύσεις σε επιστημονικά περιοδικά, η διοργάνωση τοπικών ημερίδων για παρουσίαση των αποτελεσμάτων του Έργου, οι παρουσιάσεις των αποτελεσμάτων του Έργου σε συνέδρια του εξωτερικού, οι αιτήσεις για κατοχύρωση πνευματικών δικαιωμάτων, κα. As originally specified in the proposal, the only deliverable that should have been emerged from this specific work package at the 12 th month of the project is the D6. Deliverable D6 is mainly concerned with the preparation of the software that utilizes and optimizes the integration of the Dynamics of Motion together with the Motion Capture Data. Although all the necessary background work has been accomplished (HO) and we are in the content position to specify the exact method that we are going to use for its implementation, a full implementation, installation and demonstration of the system will have to wait a bit (one or maximum two more months) mainly due to the fact that the tracking system has only been delivered last October 2010 and therefore its full installation has not been completed yet. Furthermore, proper installation requires room adjustments to be made in the VR lab of PA1 (see Figure 1). These have been approved (by PA1) and the preparation of the room to fit the complete tracking system with its 12 cameras has started. Τίτλος Δέσμης Εργασίας ΔΕ3: Platform Design Κωδικός Φορέα ΑΦ ΣΦ1 ΣΦ2 ΣΦ3 ΣΦ4 Ανθρωπομήνες για κάθε Φορέα (με βάση το Συμβόλαιο) Ανθρωπομήνες για κάθε Φορέα (δεδουλευμένοι) Στόχοι Δέσμης Εργασίας (όπως περιγράφονται στο Συμβόλαιο) Αναφέρονται επιγραμματικά οι στόχοι της παρούσας Δέσμης Εργασίας The main objective of work package 3 is the design of the software platform that will be used for implementing the software application components to be delivered later in the project (as deliverable D5). Περιγραφή Εργασίας- Βαθμός Υλοποίησης των στόχων της Δέσμης Εργασίας Καταγράφονται οι δραστηριότητες που εντάσσονται στη συγκεκριμένη Δέσμη Εργασίας (ΔΕ). Γίνεται εκτενής αναφορά στο βαθμό υλοποίησής τους, σε πιθανά προβλήματα που προέκυψαν και σε τυχόν αποκλίσεις από τους αρχικούς στόχους. Όπου εφαρμόζεται, γίνονται αριθμητικές και ποσοτικές αναφορές στα επί μέρους στάδια της ΔΕ και γίνεται, επίσης, σαφής προσδιορισμός στο Φορέα που ανάλαβε και διεκπεραίωσε την κάθε δραστηριότητα. In this work package we have proceeded with drafting out the software design that will be used for implementing the software components that will comprise the platform.

7 7 The design of the platform currently: reflects our knowledge of the initial user requirements. allows for the implementation of desirable features in the new training platform. enables integration and support of necessary hardware devices. sets the foundation of delivering an extensible and reusable software platform that can be potentially turned into a fully featured software product in the future. Prior to its development, as well as throughout the work (whenever necessary) communication with the police force needed to take place. The communication between the technical experts (HO, PA1 and PA3) and the police trainers (PA4) was a bit slow at the beginning. This was bidirectional, as the technical partners did not have experience with the particular procedures and special protocols used in police officers training activities. Respectively, the police trainers lacked the technical expertise to immediately understand the capabilities and potential of the technologies the consortium possesses. Nevertheless, careful study of the existing video-based training platform the police currently operates, together with demonstration of similar systems to PA4 by the technical experts, has helped us identify and formulate the police force needs to fit in our currently developed 3D VR training platform. In addition, we have invited police force officers (trainers) in several occasions in our facilities (VR lab of PA1) and held informal meetings regarding the technology and the possibilities. To summarise, a number of meetings between the police officers involved and the technical partners took place, both in the premises of the police department (to actually see their existing system in action) and in the VR lab of PA1 (for PA4 to understand the capabilities of the proposed platform). Μεθοδολογία και Αποτελέσματα Περιγράφεται αναλυτικά η μεθοδολογική προσέγγιση που ακολουθήθηκε και αναλύονται τα αποτελέσματα που προέκυψαν μέσα από τη συγκεκριμένη Δέσμη Εργασίας. To establish the initial platform design we have first identified the main user requirements for the platform. The formulation of the user requirements was under the responsibility of the HO, PA1, PA3 and PA4. This was partly driven by the need to reproduce the functionality of the existing video-based training system into the new platform where applicable. It has also been motivated by the additional functionality the officers would expect the new platform to have; mainly tackling of fundamental shortcomings in the existing system that were an important reason for their participation in the project. Finally, user requirements have also been formed due to the capability of the alternative technology (Virtual Reality) we will be utilizing in the project. The extraction of user requirements based on the evaluation of the existing video-based software was performed by a live presentation of the system by PA4 to the rest of the project partners. There the technical experts had the opportunity to consult with the police trainers and establish a list of advantages and disadvantages the existing platform has, as well as understand what the officers would ideally want to see in the new platform. The HO, PA1 and PA3 have then used this

8 8 list and their technical expertise to form a second list that enumerated the potential pros and cons of the proposed new platform. This list was then discussed over with the police (PA4) and the technical partners have formed the initial user requirements and software features that were then turned into a feasible initial software design. This software design is being utilized in the work of WP7, which is concerned with the implementation of the platform. We anticipate that in the second part of the project we will be in the position to refine and adapt our software design to include research results produced in the other work packages in conjunction with the implemented software of the platform. Pros Photorealistic content (filmed videos) Easy to play back Easy to use interface for non-technical staff Logging of trainees performance and historical data Replay of trainees sessions for assessment Low computational footprint Video-based Platform (existing training system of PA4) Cons Very tedious and expensive to create new content Limited authoring capabilities Limited control of training scenario in real-time Platform unaware of most trainees parameters (e.g., location, etc.) Potential of plagiarism between trainees (same scenarios for all) Habituation of trainees due to repetitive training Non-immersive training scenarios (2D viewing/physical separation) Expensive and proprietary system cannot be modified by 3 rd parties Unsuitable for simulating certain critical incidents (e.g. injury, death, etc.) Pros Reuse of content (i.e. 3D models) to author new scenarios Authoring of new scenarios even by non-technical staff Fully controllable training scenarios Easy to use interface for non-technical staff Logging of trainees performance and historical data Replay of trainees sessions for assessment Dynamic scenario execution (easy to differentiate a scenario among different trainees) Immersive 3D stereoscopic viewing Full body motion tracking data of trainee Open-source, modular, expandable software platform Reduced overall cost, mainly due to content reusability Wireless and remotely controlled training capability VR-based Platform (proposed and developed by the project) Cons Photorealism cannot be fully reproduced (technological limitation) Platform software improvements require technical staff Cost of motion tracking systems still relatively high Requirement for specially setup training facility

9 9 Ability to simulate critical incidents (e.g. death, injury, etc.) OS platform-independent The software design of our platform takes into consideration the following characteristics that were dimmed the most desirable features of the software platform in order to be able to replace the existing video-based solution used by the police: Usability Extensibility Modularity Compatibility The user interfaces provided to the users must be fairly easy to use and assume no extensive technical background from the point of view of the users. The user interfaces shall differ for trainers and trainees and ensure they are intuitive and fairly self-explanatory, providing welldefined mechanisms to carry out their individual tasks (e.g. scenario authoring (trainer), scenario execution (trainer), scenario training (trainee), etc.) The platform software should allow with moderate effort by technical staff to enhance the platform by adding new features, without changes required at the underlying core architecture. This is important, as additional features may be identified later in the project period that have to be integrated with the existing software. The design shall allow for new features to be developed as modules of the platform that can be tested and integrated independently with little direct interoperation, if any. Module support not only enhances extensibility, but can also allow in the future for 3 rd parties to implement functionality not previously considered. It is also particularly important for the consortium to be able to build their research results into modules that are attached to the core of the platform. The agile nature of hardware advances commands that the platform shall not be based on a specific operating system (e.g. Microsoft s Windows OS), neither being exclusively dependent on specific hardware devices (e.g. our motion tracking system). The software architecture must provide an abstraction layer that separates the hardware specifics from the software modules of the platform, especially those that are relevant to the user interface. In this way hardware can be upgraded independently of the software, with minimal development effort. To fulfill these software design requirements we have chosen to use a client/server infrastructure that utilizes the Model-View-Controller (MVC) software design pattern in its implementation. In the MVC, the Model is responsible for maintaining the data used by our platform (e.g. training scenarios, 3D content, audio, user data, etc.). The Model abstracts the data store for the rest of the application enabling different storage devices to be used as required. For instance, the Model may allow storing the data in an external USB hard drive or on a remote internet server. Views within the architecture are responsible of requesting, accessing and using the data for the purpose of serving the user. Views are typically interactive user interfaces the trainer and trainee will use. The key point about considering each interface as a View, is that the underlying data are not changed, it is the individual View that is responsible for utilizing the data in a specific way. The platform is anticipated to have two distinct Views, the trainer s interface and the trainee s interface. The trainer s interface should encapsulate the functionality to maintain the data (content), through commands send to the Model via the Controller, to maintain the trainee user database, to author and execute scenarios to train officers. This View will essentially have full read/write access to the data enabling trainers to perform a variety of tasks both offline and while training officers (e.g. send commands to the active training scenario to modify it dynamically). On the other hand, the trainee s user interface will mainly be a real-time stereoscopic 3D application that will take input by the trainee (e.g. position of the trainee, gun firing, etc.), but will

10 10 have no ability to modify the data content, apart from the data collected for the training session as part of the trainee s performance. Finally, the Controller will be a single component that will handle all command traffic (events) between the Views, as well as the Model. The Model and Controller will be developed as a networked server application running on the same computer, while the Views will be developed as network clients that will be possible to run from different machines. It must be noted, that this design allows for all components in the system to be run on the same computer also, without any change of the software. Figure 2 shows the conceptual software design of the individual components and their interoperation. In Figure 3, a more detailed diagram proposes a possible implementation of this software design that is particularly useful for WP7. Παραδοτέα Αναφέρονται επιγραμματικά τα Παραδοτέα που προέκυψαν μέσα από τη συγκεκριμένη Δέσμη Εργασίας. No deliverables are there up until this 12-month progress report. Τίτλος Δέσμης Εργασίας ΔΕ4: Realistic Motion of Avatars Κωδικός Φορέα ΑΦ ΣΦ1 ΣΦ2 ΣΦ3 ΣΦ4 Ανθρωπομήνες για κάθε Φορέα (με βάση το Συμβόλαιο) Ανθρωπομήνες για κάθε Φορέα (δεδουλευμένοι) Στόχοι Δέσμης Εργασίας (όπως περιγράφονται στο Συμβόλαιο) Αναφέρονται επιγραμματικά οι στόχοι της παρούσας Δέσμης Εργασίας. The combination of the motion capture data together with the dynamics of motion for the realistic movement of the anthropomorphic figures has been the major research work of this work package. Περιγραφή Εργασίας- Βαθμός Υλοποίησης των στόχων της Δέσμης Εργασίας Καταγράφονται οι δραστηριότητες που εντάσσονται στη συγκεκριμένη Δέσμη Εργασίας (ΔΕ). Γίνεται εκτενής αναφορά στο βαθμό υλοποίησής τους, σε πιθανά προβλήματα που προέκυψαν και σε τυχόν αποκλίσεις από τους αρχικούς στόχους. Όπου εφαρμόζεται, γίνονται αριθμητικές και ποσοτικές αναφορές στα επί μέρους στάδια της ΔΕ και γίνεται, επίσης, σαφής προσδιορισμός στο Φορέα που ανάλαβε και διεκπεραίωσε την κάθε δραστηριότητα. In the first part of this work package, the extended study of the latest research bibliography has taken place. This has allowed the researchers (and particularly the new researcher, namely Dr. Efstathios Stavrakis under the HO) to first identify the state of the art in the area of motion synthesis with particular emphasis on techniques with potential of combining motion capture

11 11 data and dynamics. It has also enabled us to identify two research issues that are particularly important in the area of 3D computer animation and are particularly suitable for the current training platform. In the second part of the work package, we have identified the requirements for animating virtual characters using both motion capture and physics. We have therefore implemented one of the standard algorithms for generating long animation sequences using smaller clips of motion captured data, namely the Motion Graphs technique, Kovar02 [1]. Subsequently, we have experimented with various software libraries used by the community for simulating rigid body dynamics (i.e. the Open Dynamics Engine, Havok, PhysX) and build a demo application to gain experience with the simulation software and its capabilities. Μεθοδολογία και Αποτελέσματα Περιγράφεται αναλυτικά η μεθοδολογική προσέγγιση που ακολουθήθηκε και αναλύονται τα αποτελέσματα που προέκυψαν μέσα από τη συγκεκριμένη Δέσμη Εργασίας. Research-wise, we have reviewed the most recent state-of-the-art scientific papers relevant to the realistic motion of the avatars, for the implementation of the project. Specifically, the idea of combining motion capture data together with the dynamics of motion is considered an important and difficult to solve problem in the computer animation community. While a small number of solutions exist, the problem of combining the two methods is not fully tackled. We have identified that motion captured data is best-suited for commonly performed actions, e.g. walking, running, jumping, etc. However, motion captured data are fixed prerecorded sequences and cannot be directly used in arbitrary synthetic conditions, such as introduction of obstacles in the 3D environment. Physics may be applied to the motion captured data in order to synthesize new motions that take into consideration novel or extreme conditions that did not existed in the recorded data. For instance, along the walk path of a 3D character an object may be introduced in real-time, when motion captured data are used for the walking motion, a physically based simulation can assist in generating the impact with the injected object by applying forces to the object and the character. Physically-based animation is important in situations of simulating motions that are very difficult and sometimes impossible to record, e.g. an impact of an object to a character that causes serious injury or death. In such cases, a physically-based simulation can be used. We have chosen to use the Motion Graphs, Kovar02 [1], technique for tackling the realistic motion of characters using motion capture data. Despite its very wide adoption by the computer animation community there have not been available software implementations of the technique that we could use, therefore we have developed it from scratch. Alongside with the core Motion Graphs software library that we have implemented in C++, we have developed auxiliary libraries for parsing and handling motion capture data files of Biovision (BVH) file format. This choice of supporting BVH files allows all our software to take advantage of existing and freely available motion capture databases, such as the Carnegie Mellon University Motion Capture Database ( which counts for over 2500 captured motions. Looking ahead, we anticipate that the implemented BVH software library will be able, with small modifications, to support data captured with our own optical motion capture system that will be operational shortly in the premises of PA1. In addition, we have developed a graphical

12 12 application in C++ using OpenGL that allows simultaneous viewing and playback of multiple BVH motion files in real-time. This software allows applying the Motion Graphs algorithm on motions the user loads and has been used to debug and preview the results. See Figure 4 and Figure 5 for sample output of the motion control application. To tackle character animation using dynamics of motion we have utilized the Open Dynamic Engine ( which is one of the standard libraries used by the community for simulating rigid body dynamics. To gain experience with the libraries, we have built simple 3D applications where physics are applied to objects of our testing 3D environment. In the platform both these methods (motion captured data and dynamics) can co-exist and a solution to the problem of transitioning from one to another will be tackled in WP5. Switching between the two methods is currently done manually. In parallel to the research on the combination of motion capture and dynamics, we have identified two very important topics that the computer animation community has very recently turned its attention to and they are tightly related to the objectives of this project. Specifically, the first problem is the lack of tools that can assist content developers to generate believable character animations using motion capture data and dynamics. Current algorithms may be too difficult to expose to a user through conventional software applications, and their parameters maybe difficult to translate meaningfully for non-expert content developers. This problem was identified as we worked in the design and development of the platform (WP3 and WP7). We found that even very competent users would face a problem leveraging the capabilities of motion graphs and dynamics simulation, not because of the software itself, but rather because of the complexity of the underlying algorithms and the large space of different possibilities it offers to users. For instance, the user may find it difficult to identify which motion capture data would be appropriate for a walking motion, using a database that has hundreds of different walk sequences. A solution to this problem could help improve the usability of the platform. Another problem that is particularly challenging is two-character interaction as seen from the viewpoint of motion synthesis, which is extremely important in computer games (e.g. hand-tohand combat games). A potential solution to this challenge, apart from being very important in the research community as there are not many techniques available yet, it would allow the platform to feature character interactions, thus allowing richer training scenarios to be authored. Παραδοτέα Αναφέρονται επιγραμματικά τα Παραδοτέα που προέκυψαν μέσα από τη συγκεκριμένη Δέσμη Εργασίας. D6 is the deliverable of this work package. This is the Hybrid system for motion control. The software implemented, by the HO, demonstrates the capabilities of using motion captured data and physics in isolation. Τίτλος Δέσμης Εργασίας ΔΕ5: Automatic Transition within the Hybrid method

13 13 Κωδικός Φορέα ΑΦ ΣΦ1 ΣΦ2 ΣΦ3 ΣΦ4 Ανθρωπομήνες για κάθε Φορέα (με βάση το Συμβόλαιο) Ανθρωπομήνες για κάθε Φορέα (δεδουλευμένοι) Στόχοι Δέσμης Εργασίας (όπως περιγράφονται στο Συμβόλαιο) Αναφέρονται επιγραμματικά οι στόχοι της παρούσας Δέσμης Εργασίας. Believable, automatic transitions between the motion capture and the dynamics method, as well as the clear understanding when each one is more appropriate has been the main objective of work package 5. Καταγράφονται οι δραστηριότητες που εντάσσονται στη συγκεκριμένη Δέσμη Εργασίας (ΔΕ). Γίνεται εκτενής αναφορά στο βαθμό υλοποίησής τους, σε πιθανά προβλήματα που προέκυψαν και σε τυχόν αποκλίσεις από τους αρχικούς στόχους. Όπου εφαρμόζεται, γίνονται αριθμητικές και ποσοτικές αναφορές στα επί μέρους στάδια της ΔΕ και γίνεται, επίσης, σαφής προσδιορισμός στο Φορέα που ανάλαβε και διεκπεραίωσε την κάθε δραστηριότητα. As we are in the content position to identify exactly the Hybrid method to be employed for the animation of the avatars, the automatic transition from one method to the other has started being our next concern. Since we have identified when it is best (or even possible) to use the motion capture data and when the dynamics of motion control our next concern has just started being the switch between the two with the minimum user-intervention possible, or automatically if possible (HO). Μεθοδολογία και Αποτελέσματα Περιγράφεται αναλυτικά η μεθοδολογική προσέγγιση που ακολουθήθηκε και αναλύονται τα αποτελέσματα που προέκυψαν μέσα από τη συγκεκριμένη Δέσμη Εργασίας. It is a bit early to express ideas regarding the automatic transition between the two methods of the Hybrid system and therefore this has to wait until its completion. Up to now, we can only identify clearly when each method could be employed for achieving its best utilization and thus the best results in terms of realism and motion control. Extended study in the area as well as experimentation with each technique separately, together with their efficient combination resulted in identifying when to use each method (using the outcomes of WP4) and therefore, we can now proceed to set up the automatic transition from one to the other and perhaps back again. Παραδοτέα

14 14 Αναφέρονται επιγραμματικά τα Παραδοτέα που προέκυψαν μέσα από τη συγκεκριμένη Δέσμη Εργασίας. No deliverables yet. Τίτλος Δέσμης Εργασίας ΔΕ6: Body and Face Integration Κωδικός Φορέα ΑΦ ΣΦ1 ΣΦ2 ΣΦ3 ΣΦ4 Ανθρωπομήνες για κάθε Φορέα (με βάση το Συμβόλαιο) Ανθρωπομήνες για κάθε Φορέα (δεδουλευμένοι) Στόχοι Δέσμης Εργασίας (όπως περιγράφονται στο Συμβόλαιο) Αναφέρονται επιγραμματικά οι στόχοι της παρούσας Δέσμης Εργασίας The integration of the system with the realistic full body models of the human beings together with the integration with emotional face expressions are the main objectives of this work package. Περιγραφή Εργασίας- Βαθμός Υλοποίησης των στόχων της Δέσμης Εργασίας Καταγράφονται οι δραστηριότητες που εντάσσονται στη συγκεκριμένη Δέσμη Εργασίας (ΔΕ). Γίνεται εκτενής αναφορά στο βαθμό υλοποίησής τους, σε πιθανά προβλήματα που προέκυψαν και σε τυχόν αποκλίσεις από τους αρχικούς στόχους. Όπου εφαρμόζεται, γίνονται αριθμητικές και ποσοτικές αναφορές στα επί μέρους στάδια της ΔΕ και γίνεται, επίσης, σαφής προσδιορισμός στο Φορέα που ανάλαβε και διεκπεραίωσε την κάθε δραστηριότητα. Once the capabilities of the Hybrid method are fully examined, the smooth automatic transition between its methods has started being the major concern and the design of the platform to be used in the application has been set up, the project is on the track of being integrated with fullbody models for the realistic representation of the avatars (PA2) together with the accumulation of realistic facial expressions (PA3). As indicated in the Work Package Time Schedule, the Body and Face Integration Work Package will be starting with the submission of the 12-month interim report. However, a comprehensive literature review is being carried out by the researchers involved (PA2 and PA3), and the main outcomes can be found in the next section. Μεθοδολογία και Αποτελέσματα Περιγράφεται αναλυτικά η μεθοδολογική προσέγγιση που ακολουθήθηκε και αναλύονται τα αποτελέσματα που προέκυψαν μέσα από τη συγκεκριμένη Δέσμη Εργασίας. The problems associated with defining the surface of a complex deformable object such as the human figure are considerable, as yet, nobody has devised a system which is flexible enough to cater for the whole range of possible surface shapes of such a figure, while retaining the

15 15 capabilities of displaying fine detail on the surface, and being sufficiently economical in use for it to be suitable for producing animations in a real-time system. Complex figures, such as the human body, are defined by an arbitrary topology of vertices and they consist of a number of branched surfaces. The most popular methods for handling branched surfaces based on a non-rectangular topology of points are subdivision splines [2, 3, 4, 5, 6]. An investigation of such methods is being carried out in order to find the one best fitted in the current project (by PA2). This method will be used on existing human figures to smooth the animated human joints and especially the face, after face animation methods are applied, along with the rest of the surface. Current technologies in the area of facial animation are still unable to synthesize human expressions in a realistic and efficient manner. Given the current application, it is vital for suspect emotions to be synthesized as realistically as possible, given that trainees will use the information provided to try to interpret the suspects emotions and subsequently take the required action. An in depth investigation of the existing technologies on synthesizing emotional expressions is under way (by PA3). The two methods that prevail from the literature are described below. The first method, MPEG-4 [7, 8, 9, 10], indicates a way of modeling facial expressions and the underlying emotions, which is strongly influenced by neurophysiologic and psychological studies. The Facial Animation Parameters (FAPs) that are utilized in the framework of MPEG-4 for facial animation purposes are strongly related to the Action Units (AUs) which makes up the core of the Facial Action Coding System (FACS) [11, 12]. The FACS system has now become the de facto standard in characterizing facial expressions. In psychological studies it is generally believed that the six basic expressions (happiness, sadness, anger, disgust, fear and surprise) can be decomposed into culture and ethnic independent facial AUs. FAPs are adequate to define the measurement of muscular actions relevant to AUs. In fact, FAPs can be placed on any synthetic facial model in a consistent manner with little influence by the inter-personal variations between the models. The second method is based on Geoface [13], a system originally developed by Keith Waters for animating 3-dimensional computer generated faces and further developed (Geoface2) by one of the partners [14, 15] of the project (PA3). The model creates expressions upon the face by using muscle-vectors, which produce geometric distortions in the face, simulating skin movement. The extension implemented to Geoface allows for an independent set of muscles to be created, that can be applied to arbitrary face meshes. The set of linear muscles can be transformed, scaled and/or rotated as well as the system allows for the creation of new muscles to accommodate for face model variations. The method that produces the more realistic facial expressions in the most efficient manner will be integrated to the rest of the platform. Παραδοτέα

16 16 Αναφέρονται επιγραμματικά τα Παραδοτέα που προέκυψαν μέσα από τη συγκεκριμένη Δέσμη Εργασίας. No deliverables yet. Τίτλος Δέσμης Εργασίας ΔΕ7: Platform Development and Testing Κωδικός Φορέα ΑΦ ΣΦ1 ΣΦ2 ΣΦ3 ΣΦ4 Ανθρωπομήνες για κάθε Φορέα (με βάση το Συμβόλαιο) Ανθρωπομήνες για κάθε Φορέα (δεδουλευμένοι) Στόχοι Δέσμης Εργασίας (όπως περιγράφονται στο Συμβόλαιο) Αναφέρονται επιγραμματικά οι στόχοι της παρούσας Δέσμης Εργασίας The application of the proposed work, i.e. the implementation of the pilot tool that will help the training of the Cyprus Police Force, is the aim of this work package. Περιγραφή Εργασίας- Βαθμός Υλοποίησης των στόχων της Δέσμης Εργασίας Καταγράφονται οι δραστηριότητες που εντάσσονται στη συγκεκριμένη Δέσμη Εργασίας (ΔΕ). Γίνεται εκτενής αναφορά στο βαθμό υλοποίησής τους, σε πιθανά προβλήματα που προέκυψαν και σε τυχόν αποκλίσεις από τους αρχικούς στόχους. Όπου εφαρμόζεται, γίνονται αριθμητικές και ποσοτικές αναφορές στα επί μέρους στάδια της ΔΕ και γίνεται, επίσης, σαφής προσδιορισμός στο Φορέα που ανάλαβε και διεκπεραίωσε την κάθε δραστηριότητα. The development and testing of the training platform has been initiated according to the project s plan in this work package. In this work package the activities have so far involved transferring the software platform designs, as these have been identified in WP3, into a software implementation. The technical partners have identified the software tools, libraries and environment the training tool will be developed in, and have already build several of the components present in the design. Furthermore, the HO and PA1 have worked with PA4 (the Cyprus Police Force) to assess the usability of the early prototype and understand the needs of the implemented system, from a users point of view. To this end, PA4 has provided a few potential training scenarios that have been used by the technical partners as guidance while engineering the training platform prototype.

17 17 Μεθοδολογία και Αποτελέσματα Περιγράφεται αναλυτικά η μεθοδολογική προσέγγιση που ακολουθήθηκε και αναλύονται τα αποτελέσματα που προέκυψαν μέσα από τη συγκεκριμένη Δέσμη Εργασίας. Based on the software design of WP3 the HO and mainly PA1 have advanced on the implementation of 4 different parts of the platform: (a) The Scenario Server, which implements the Model of the software design. The Scenario Server has been implemented in the Python programming language and offers the ability to manage at a low-level the digital assets, such as 3D models, metadata descriptions (using XML), etc., that are useful for the training scenarios. The server utilizes the UDP networking protocol and is accessible over a network port, offering asynchronous, multi-client communication, thus allowing several clients to connect and interact with it. The Scenario Server is a core system component that is aimed to be used by the other platform components, and therefore requires no direct user interaction; hence it has no graphical user interface and is the primary platform component to be launched in a training environment. The scenario server can be run independent of other system components and does not require to be restarted between training sessions, making it easy and efficient to use. The Scenario Server will be developed further, alongside the other platform components to fulfil their needs, but its existing features are already fully functional. A sample execution of the Scenario Server is shown in Figure 6. (b) The Hardware Abstraction Layer, which is composed of low-level components that are used by the other platform components to manage data between the diverse hardware used. One of the libraries handles the input from a haptic glove, which provides the data for the trigger of the weapon of trainee officers. Another is handling the positional and rotational data provided by the magnetic tracker, which is monitoring the hand of the trainee, allowing us to know where the officer is aiming at from the real world to the virtual world. The tracker also takes the position and orientation of the head of the trainee, so that the virtual camera can be adjusted both for navigation and viewpoint adjustment. This tracker will be replaced with the more versatile full body tracker that is currently being installed, allowing more robust tracking and additional interactivity. (c) The Trainee Client, is a component that has been build to satisfy the need for an interface between the platform and a trainee police officer. The Trainee Client implements one of the possible arbitrary Views in the software design and has a number of distinct features that set it apart from other Views in the platform. The Trainee Client connects the real world, where the officers are being trained, with the 3D virtual world using an immersive 3D stereoscopic projection system (operated by PA1). It tracks the position of the trainee officer using a Polhemus magnetic tracking device (which will soon be replaced by the more versatile full body motion tracking acquired) and integrates interaction data from a 5DT haptic glove that officers wear in order to detect triggering of their physical training hand gun. The client is a 3D application that executes training scenarios fetched from the Scenario Server and it is implemented in the Panda3D engine. The hardware integration has been done by developing wrapper Python libraries that form the Hardware Abstraction Layer. While the magnetic motion tracker used is very limited in functionality compared to the one currently installed at

18 18 the premises of PA1, it has allowed us to carry on with the development of all software components uninterrupted. A small amount of effort is anticipated to provide support for the new motion tracking device (i.e. by developing another wrapper library in the Hardware Abstraction Layer). In Figure 9, two photographs of a trainee using the platform are shown. (d) The Trainer Client, is the other main component that has been developed to implement another View of the software design. This client enables trainer officers to connect to the Scenario Server and execute a training session in the user space of the trainee officers; they can use it to author scenarios and to monitor the performance and progress of trainees. The Trainer Client is in general a component that can modify the data assets, inserting new content and removing old content, in contrast to the Trainee Client that can only read content and only writes data relevant to the performance of the trainee. The capabilities of the Trainer Client are still limited and we anticipate advancing on that front in the second part of the project, as we identify what functionality will be both useful and easy to use for the trainer officers of PA4. A first implementation of the Trainer Client is shown in Figure 7, which has been subsequently improved with a graphical user interface and additional functionality, shown in Figure 8. Παραδοτέα Αναφέρονται επιγραμματικά τα Παραδοτέα που προέκυψαν μέσα από τη συγκεκριμένη Δέσμη Εργασίας. No deliverables up until this 12-month report. Α.4. «ΠΙΝΑΚΑΣ ΔΕΣΜΩΝ ΕΡΓΑΣΙΑΣ» Καταγράφονται οι ΔΕ, ο μήνας έναρξης και υλοποίησης και τα παραδοτέα που προέκυψαν μέχρι στιγμής.

19 19 Αριθμός Δέσμης Εργασίας Τίτλος Δέσμης Εργασίας Έναρξη (μήνας) Ολοκλήρωση (μήνας) Παραδοτέα ΔΕ1 Project Management 1 24 D1, D2 ΔΕ2 Dissemination and Exploitation of Results 3 24 None ΔΕ3 Platform Design 3 18 None ΔΕ4 Realistic Motion of Avatars 1 12 D6 ΔΕ5 Automatic Transition with the Hybrid method None ΔΕ6 Body and Face Integration None ΔΕ7 Platform Development and Testing 4 22 None

20 20 Μ Ε Ρ Ο Σ Β ΠΑΡΑΡΤΗΜΑ Β1 Επισυνάπτονται τα παραδοτέα του έργου που μπορούν να δοθούν σε έντυπη μορφή. Apart from the two reports (D1, D2), no other deliverables in printed format could be delivered up until now. The only other deliverable at this stage is D6, which involves a software implementation. ΠΑΡΑΡΤΗΜΑ Β2 Επισυνάπτονται οποιεσδήποτε επιπλέον πληροφορίες αναφορικά με το ερευνητικό έργο, οι οποίες θεωρούνται απαραίτητες. Sample of meeting minutes For the smooth running of the work we had plenary technical meetings at least once a month. In some cases where the need emerged we had even in-between meetings (sometimes on a weekly basis) to solve ad-hoc problems. Choosing some of the meetings, the following gives a sample of the minutes taken. Sample 1 Meeting Date: 12 th February 2010 Present: Dr. Stephania Loizidou Himona (HO), Dr. Efstathios Stavrakis (HO), Dr. Yiorgos Chrysanthou (PA1), Dr. Andreas Savva (PA2) and Dr. Andreas Loizides (PA3). Topics Discussed: 1. The project so far did not have an acronym therefore the importance of it was raised by Dr. G. Chrysanthou. After some discussion of possible options the short name Simpol VR was given to it to be used throughout the work when referring to it. 2. The immediate need for a website creation was raised by Dr. A. Loizides. Necessary inclusions in it have been discussed thoroughly and it was agreed that an independent webdesigner would have to be employed for its development. Dr. A. Loizides is going to be responsible to find a suitable person for its immediate construction. 3. The timetable was agreed with regards also to the timesheets that need to be filled out every month. 4. Special care was given to the research part of the project by Dr. S. Loizidou and was agreed that the relevant publications, state of the art papers, regarding the realistic motion and the possible combination of the two methods of the hybrid approach as specified in the proposal would have to be of immediate concern.

21 21 Sample 2 Meeting Date: 5 th March 2010 Present: Dr. Stephania Loizidou Himona (HO), Dr. Efstathios Stavrakis (HO), Dr. Yiorgos Chrysanthou (PA1), Dr. Andreas Savva (PA2), Dr. Andreas Loizides (PA3), Mr. Aristos Chrysanthou (Cyprus Police Force), Costas Constantinou (Cyprus Police Force) and Marios Atteshlis (Cyprus Police Force). Topics Discussed: 1. An informal workshop with the police representatives took place at the PA1 VR lab. Its major aim has been to demonstrate to the policemen an early prototype of the platform to be developed. Valuable feedback was received that helped in identifying their needs and their expectations. As an outcome it was agreed exactly how the end result is going to be and how it will differ from the existing system that the police force is currently using. The major difference would be the interactive scenario creation. 2. We have arranged a follow-up meeting on the 12 th March at the police s premises so that they can show to us their current system. Sample 3 Meeting Date: 23rd April 2010 Present: Dr. Stephania Loizidou Himona (HO), Dr. Efstathios Stavrakis (HO), Dr. Yiorgos Chrysanthou (PA1), Dr. Andreas Savva (PA2), Dr. Andreas Loizides (PA3) and Dr. Christos Gatzoulis (Visiting Lecturer, PA1) Topics Discussed: 1. Dr. C. Gatzoulis who is a visiting lecturer at PA1 and his interest lie in the field of Computer Graphics raised the importance of looking up at special graphical packages that might help in our work. Specifically, he suggested Endorphin and HAVOK. It was agreed that all of us are going to have a look at them, identifying their strengths (and their weaknesses) and how they might help in the project. 2. The blending of Motion Capture Data with the Dynamics of Motion was discussed thoroughly. Dr. S. Loizidou and Dr. E. Stavrakis expressed the advantages of how these two methods can be combined in a Hybrid system and how to proceed in utilising them. 3. The possibility of including realistic faces to our avatars was discussed. Specifically, Dr. A. Loizides who is the expert on facial animation explained how this integration could become possible, although this has to wait until a later stage of the project. 4. The integration with full-body models was also raised by the expert in the field, namely Dr. A. Savva. What it needs to be done, and in what format the files have to be in order to be incorporated with our work was identified. This also has to wait until later in the project.

22 22 Sample 4 Meeting Date: 29 th June 2010 Present: Stephania Loizidou Himona (HO), Dr. Efstathios Stavrakis (HO), Dr. Yiorgos Chrysanthou (PA1), Dr. Andreas Savva (PA2) and Dr. Andreas Loizides (PA3). Topics Discussed: 1. Dr. E. Stavrakis has raised the importance on focusing our work on motion graphs, a rather new idea on solving the difficult problem of motion control. 2. It was agreed that a specific novel paper on motion graphs was going to be studied from all the team members and in a follow-up meeting to discuss their advantages, if any, and the possibility of include them in our system. 3. The timesheets were analysed once more and agreed how we are all going to fill them up. 4. The CMU Motion capture data and its utilisation was raised. 5. Crowd simulation as an option was identified. Figures (1-9) Figure 1 A 3D impression of the VR Lab that will host the stereoscopic projection and motion tracking system at the premises of PA1.

23 23 Figure 2 - SimPol - Software Architecture concept.

24 24 Figure 3 - Possible software implementation of the design.

25 25 Figure 4 Motion Control 3D application. Two motion captured animation sequences are shown. Figure 5 Candidate transition points (marked with red) between the two motions shown in Figure 4, as calculated by the Motion Graphs algorithm.

26 26 Figure 6 - Scenario Server. A console application handling network communication between the platform components. Figure 7 - Trainer application that allows the trainer to control a training session.