Opendata, web and dolomites


Analysis and Representation of Complex Activities in Videos

Total Cost €


EC-Contrib. €






Project "ARCA" data sheet

The following table provides information about the project.


Organization address
city: BONN
postcode: 53113

contact info
title: n.a.
name: n.a.
surname: n.a.
function: n.a.
email: n.a.
telephone: n.a.
fax: n.a.

 Coordinator Country Germany [DE]
 Project website
 Total cost 1˙499˙875 €
 EC max contribution 1˙499˙875 € (100%)
 Programme 1. H2020-EU.1.1. (EXCELLENT SCIENCE - European Research Council (ERC))
 Code Call ERC-2015-STG
 Funding Scheme ERC-STG
 Starting year 2016
 Duration (year-month-day) from 2016-06-01   to  2021-05-31


Take a look of project's partnership.

# participants  country  role  EC contrib. [€] 


 Project objective

The goal of the project is to automatically analyse human activities observed in videos. Any solution to this problem will allow the development of novel applications. It could be used to create short videos that summarize daily activities to support patients suffering from Alzheimer's disease. It could also be used for education, e.g., by providing a video analysis for a trainee in the hospital that shows if the tasks have been correctly executed.

The analysis of complex activities in videos, however, is very challenging since activities vary in temporal duration between minutes and hours, involve interactions with several objects that change their appearance and shape, e.g., food during cooking, and are composed of many sub-activities, which can happen at the same time or in various orders.

While the majority of recent works in action recognition focuses on developing better feature encoding techniques for classifying sub-activities in short video clips of a few seconds, this project moves forward and aims to develop a higher level representation of complex activities to overcome the limitations of current approaches. This includes the handling of large time variations and the ability to recognize and locate complex activities in videos. To this end, we aim to develop a unified model that provides detailed information about the activities and sub-activities in terms of time and spatial location, as well as involved pose motion, objects and their transformations.

Another aspect of the project is to learn a representation from videos that is not tied to a specific source of videos or limited to a specific application. Instead we aim to learn a representation that is invariant to a perspective change, e.g., from a third-person perspective to an egocentric perspective, and can be applied to various modalities like videos or depth data without the need of collecting massive training data for all modalities. In other words, we aim to learn the essence of activities.


year authors and title journal last update
List of publications.
2019 Pau Panareda Busto, Ahsan Iqbal, Juergen Gall
Open Set Domain Adaptation for Image and Action Recognition
published pages: 1-1, ISSN: 0162-8828, DOI: 10.1109/tpami.2018.2880750
IEEE Transactions on Pattern Analysis and Machine Intelligence 2019-07-30
2019 Hilde Kuehne, Alexander Richard, Juergen Gall
A Hybrid RNN-HMM Approach for Weakly Supervised Temporal Action Segmentation
published pages: 1-1, ISSN: 0162-8828, DOI: 10.1109/tpami.2018.2884469
IEEE Transactions on Pattern Analysis and Machine Intelligence 2019-07-30
2017 Hilde Kuehne, Alexander Richard, Juergen Gall
Weakly supervised learning of actions from transcripts
published pages: 78-89, ISSN: 1077-3142, DOI: 10.1016/j.cviu.2017.06.004
Computer Vision and Image Understanding 163 2019-06-19
2017 Alexander Richard, Juergen Gall
A bag-of-words equivalent recurrent neural network for action recognition
published pages: 79-91, ISSN: 1077-3142, DOI: 10.1016/j.cviu.2016.10.014
Computer Vision and Image Understanding 156 2019-06-19
2017 Ahsan Iqbal, Alexander Richard, Hilde Kuehne, Juergen Gall
Recurrent Residual Learning for Action Recognition
published pages: 126-137, ISSN: , DOI: 10.1007/978-3-319-66709-6_11
German Conference on Pattern Recognition 2019-06-19
2016 Martin Garbade, Juergen Gall
Handcrafting vs Deep Learning: An Evaluation of NTraj+ Features for Pose Based Action Recognition
published pages: , ISSN: , DOI:
Workshop New Challenges in Neural Computation 2019-06-19
2016 Umer Rafi, Ilya Kostrikov, Juergen Gall, Bastian Leibe
An Efficient Convolutional Network for Human Pose Estimation
published pages: , ISSN: , DOI:
British Machine Vision Conference 2019-06-19
2018 Umer Rafi, Juergen Gall, Bastian Leibe
Direct Shot Correspondence Matching
published pages: , ISSN: , DOI:
British Machine Vision Conference 2019-05-09
2018 Umar Iqbal, Andreas Doering, Hashim Yasin, Björn Krüger, Andreas Weber, Juergen Gall
A dual-source approach for 3D human pose estimation from single images
published pages: 37-49, ISSN: 1077-3142, DOI: 10.1016/j.cviu.2018.03.007
Computer Vision and Image Understanding 172 2019-05-09
2018 Rania Briq, Michael Moeller, Juergen Gall
Convolutional Simplex Projection Network for Weakly Supervised Semantic Segmentation
published pages: , ISSN: , DOI:
British Machine Vision Conference 2019-05-09
2018 Andreas Doering, Umar Iqbal, Juergen Gall
JointFlow: Temporal Flow Fields for Multi Person Pose Estimation
published pages: , ISSN: , DOI:
British Machine Vision Conference 2019-05-09

Are you the coordinator (or a participant) of this project? Plaese send me more information about the "ARCA" project.

For instance: the website url (it has not provided by EU-opendata yet), the logo, a more detailed description of the project (in plain text as a rtf file or a word file), some pictures (as picture files, not embedded into any word file), twitter account, linkedin page, etc.

Send me an  email ( and I put them in your project's page as son as possible.

Thanks. And then put a link of this page into your project's website.

The information about "ARCA" are provided by the European Opendata Portal: CORDIS opendata.

More projects from the same programme (H2020-EU.1.1.)


The Mass Politics of Disintegration

Read More  

SUExp (2018)

Strategic Uncertainty: An Experimental Investigation

Read More  


The Enemy of the Good: Towards a Theory of Moral Progress

Read More