Opendata, web and dolomites


Automated Measurement of Engagement Level of Children with Autism Spectrum Conditions during Human-robot Interaction

Total Cost €


EC-Contrib. €






Project "EngageME" data sheet

The following table provides information about the project.


Organization address
address: INNSTRASSE 41
city: PASSAU
postcode: 94032

contact info
title: n.a.
name: n.a.
surname: n.a.
function: n.a.
email: n.a.
telephone: n.a.
fax: n.a.

 Coordinator Country Germany [DE]
 Total cost 239˙860 €
 EC max contribution 239˙860 € (100%)
 Programme 1. H2020-EU.1.3.2. (Nurturing excellence by means of cross-border and cross-sector mobility)
 Code Call H2020-MSCA-IF-2015
 Funding Scheme MSCA-IF-GF
 Starting year 2016
 Duration (year-month-day) from 2016-10-01   to  2019-09-30


Take a look of project's partnership.

# participants  country  role  EC contrib. [€] 
1    UNIVERSITAT PASSAU DE (PASSAU) coordinator 239˙860.00


 Project objective

Engaging children with ASC (Autism Spectrum Conditions) in communication centred activities during educational therapy is one of the cardinal challenges by ASC and contributes to its poor outcome. To this end, therapists recently started using humanoid robots (e.g., NAO) as assistive tools. However, this technology lacks the ability to autonomously engage with children, which is the key for improving the therapy and, thus, learning opportunities. Existing approaches typically use machine learning algorithms to estimate the engagement of children with ASC from their head-pose or eye-gaze inferred from face-videos. These approaches are rather limited for modeling atypical behavioral displays of engagement of children with ASC, which can vary considerably across the children. The first objective of EngageME is to bring novel machine learning models that can for the first time effectively leverage multi-modal behavioural cues, including facial expressions, head pose, vocal and physiological cues, to realize fully automated context-sensitive estimation of engagement levels of children with ASC. These models build upon dynamic graph models for multi-modal ordinal data, based on state-of-the-art machine learning approaches to sequence classification and domain adaptation, which can adapt to each child, while still being able to generalize across children and cultures. To realize this, the second objective of EngageME is to provide the candidate with the cutting-edge training aimed at expanding his current expertise in visual processing with expertise in wearable/physiological, and audio technologies, from leading experts in these fields. EngageME is expected to bring novel technology/models for endowing assistive robots with ability to accurately ‘sense’ engagement levels of children with ASC during robot-assisted therapy, while providing the candidate with a set of skills needed to become one of the frontiers in the emerging field of affect-sensitive assistive technology.


year authors and title journal last update
List of publications.
2019 Rudovic, O., Zhang, M, Schuller, B., Picard, R.
Multi-modal Active Learning From Human Data: A Deep Reinforcement Learning Approach
published pages: , ISSN: , DOI:
International Conference on Multimodal Interaction (ICMI ’19) 2019-12-16
2019 Rudovic, O., Utsumi, Y., Guerrero, R., Peterson, K., Rueckert, D., Picard, R. W.
Meta-Weighted Gaussian Process Experts for Personalized Forecasting of AD Cognitive Changes
published pages: , ISSN: , DOI:
Machine Learning for Healthcare Conference 2019-12-16
2019 Rudovic, O., Park, H-W., Busche, J., Schuller, B. , Breazeal, C., Picard, R. W.
Personalized Estimation of Engagement from Videos Using Active Learning with Deep Reinforcement Learning
published pages: , ISSN: , DOI:
IEEE CVPR -AMFG W 2019-12-16
2018 M. Feffer, O. Rudovic, R. W. Picard
A Mixture of Personalized Experts for Human Affect Estimation
published pages: , ISSN: , DOI:
in International Conference on Machine Learning and Data Mining in Pattern Recognition 2019-04-18
2018 E. C. Ferrer, O. Rudovic, T. Hardjono, A. Pentland
RoboChain: A Secure Data-Sharing Framework for Human-Robot Interaction
published pages: , ISSN: , DOI:
The 10th International Conference on eHealth, Telemedicine, and Social Medicine (eTELEMED 2018) 2019-04-18
2017 K. Peterson, O. Rudovic, R. Guerrero, R. W. Picard
Personalized Gaussian Processes for Future Prediction of Alzheimer\'s Disease Progression
published pages: , ISSN: , DOI:
NIPS\'W on Machine Learning for Health 2019-04-18
2017 T.D. Linh, R. Walecki, O. Rudovic, S. Eleftheriadis, B. Schuller, M Pantic
DeepCoder: Semi-parametric Variational Autoencoders for Automatic Facial Action Coding.
published pages: , ISSN: , DOI:
ICCV 2019-05-03
2017 W. Chen, O. Rudovic, R. W. Picard
GIFGIF+: Collecting Emotional Animated GIFs with Clustered Multi-Task Learning
published pages: , ISSN: , DOI:
The 7th International Conference on Affective Computing and Intelligent Interaction (ACII) 2019-05-03
2017 Liu, Dianbo; Peng, Fengjiao; Shea, Andrew; Ognjen; Rudovic; Picard, Rosalind
DeepFaceLIFT: Interpretable Personalized Models for Automatic Estimation of Self-Reported Pain
published pages: , ISSN: 1532-4435, DOI:
Journal of Machine Learning Research, IJCAI AComp 3 2019-05-03
2017 M. Sra, P. Vijayaraghavan, O. Rudovic, P. Maes , D. Roy
DeepSpace: Mood-Based Image Texture Generation for Virtual Reality from Music
published pages: , ISSN: , DOI:
CVPR\'W 2019-05-03
2018 Ognjen Rudovic, Jaeryoung Lee, Miles Dai, Björn Schuller, Rosalind W. Picard
Personalized machine learning for robot perception of affect and engagement in autism therapy
published pages: eaao6760, ISSN: 2470-9476, DOI: 10.1126/scirobotics.aao6760
Science Robotics 3/19 2019-05-03
2017 Robert Walecki, Ognjen Rudovic, Vladimir Pavlovic, Maja Pantic
A Copula Ordinal Regression Framework for Joint Estimation of Facial Action Unit Intensity
published pages: 1-1, ISSN: 1949-3045, DOI: 10.1109/TAFFC.2017.2728534
IEEE Transactions on Affective Computing 2019-05-03
2017 Predicting Tomorrow\'s Mood, Health, and Stress Level using Personalized Multitask Learning and Domain Adaptation
N. Jaques, O. Rudovic, S. Taylor, A. Sano, R.W. Picard
published pages: , ISSN: 1532-4435, DOI:
Journal of Machine Learning Research, IJCAI AComp. 2019-05-03
2017 R. Suzuki, Lee J., O. Rudovic
Nao-dance therapy for children with ASD
published pages: , ISSN: , DOI:
Proceedings of the Companion of the 2017 ACM/IEEE International Conference on Human- Robot Interaction 2019-05-03
2017 B. Gholami, O. Rudovic, V. Pavlovic
PUnDA: Probabilistic Unsupervised Domain Adaptation for Knowledge Transfer Across Visual Categories
published pages: , ISSN: , DOI:
ICCV 2019-05-03
2017 Ognjen Rudovic, Jaeryoung Lee, Lea Mascarell-Maricic, Björn W. Schuller, Rosalind W. Picard
Measuring Engagement in Robot-Assisted Autism Therapy: A Cross-Cultural Study
published pages: , ISSN: 2296-9144, DOI: 10.3389/frobt.2017.00036
Frontiers in Robotics and AI 4 2019-05-03
2017 R. Walecki, O. Rudovic, B. Schuller, V. Pavlovic, M. Pantic
Deep Structured Learning for Facial Action Unit Intensity Estimation
published pages: , ISSN: , DOI:
Proceedings of IEEE Int\'l Conf. Computer Vision and Pattern Recognition (CVPR\'17) 2019-05-03
2018 Adria Ruiz, Ognjen Rudovic, Xavier Binefa, Maja Pantic
Multi-Instance Dynamic Ordinal Random Fields for Weakly Supervised Facial Behavior Analysis
published pages: 3969-3982, ISSN: 1057-7149, DOI: 10.1109/TIP.2018.2830189
IEEE Transactions on Image Processing 27/8 2019-05-03
2017 Stefanos Eleftheriadis, Ognjen Rudovic, Marc Peter Deisenroth, Maja Pantic
Gaussian Process Domain Experts for Modeling of Facial Affect
published pages: 4697-4711, ISSN: 1057-7149, DOI: 10.1109/TIP.2017.2721114
IEEE Transactions on Image Processing 26/10 2019-05-03
2017 C.D. Tran, O. Rudovic, V. Pavlovic
Unsupervised domain adaptation with copula models
published pages: , ISSN: , DOI:
Machine Learning for Signal Processing (MLSP) 2019-05-03
2017 D. L. Martinez, O. Rudovic, R. W. Picard
Personalized Automatic Estimation of Self-reported Pain Intensity from Facial Expressions,
published pages: , ISSN: , DOI:
CVPR\'W on Deep Affective Learning and Context Modeling. 2019-05-03
2017 D. L. Martinez, O. Rudovic, R. W. Picard
Physiological and behavioral profiling for nociceptive pain estimation using personalized multitask learning
published pages: , ISSN: , DOI:
NIPS\'W on Machine Learning for Health 2019-04-18
2018 Y. Utsumi, O. Rudovic, K. Peterson, R. Guerrero, R. W. Picard
Personalized Gaussian Processes for Forecasting of Alzheimer\'s Disease Assessment Scale-CognitionSub-Scale (ADAS-Cog13)
published pages: , ISSN: , DOI:
The 40th International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) 2019-04-18

Are you the coordinator (or a participant) of this project? Plaese send me more information about the "ENGAGEME" project.

For instance: the website url (it has not provided by EU-opendata yet), the logo, a more detailed description of the project (in plain text as a rtf file or a word file), some pictures (as picture files, not embedded into any word file), twitter account, linkedin page, etc.

Send me an  email ( and I put them in your project's page as son as possible.

Thanks. And then put a link of this page into your project's website.

The information about "ENGAGEME" are provided by the European Opendata Portal: CORDIS opendata.

More projects from the same programme (H2020-EU.1.3.2.)

GWFP (2019)

Geometric study of Wasserstein spaces and free probability

Read More  

VINCI (2020)

The Value of Information and Choice to Improve Control.

Read More  


Ribosomal frameshifts as a novel mechanism to control RNA turnover in stress

Read More