Opendata, web and dolomites

SecondHands SIGNED

SecondHands: A Robot Assistant For Industrial Maintenance Tasks

Total Cost €


EC-Contrib. €






Project "SecondHands" data sheet

The following table provides information about the project.


Organization address
postcode: AL10 9UL
website: n.a.

contact info
title: n.a.
name: n.a.
surname: n.a.
function: n.a.
email: n.a.
telephone: n.a.
fax: n.a.

 Coordinator Country United Kingdom [UK]
 Project website
 Total cost 6˙930˙000 €
 EC max contribution 5˙994˙000 € (86%)
 Programme 1. H2020-EU. (Advanced interfaces and robots: Robotics and smart spaces)
 Code Call H2020-ICT-2014-1
 Funding Scheme RIA
 Starting year 2015
 Duration (year-month-day) from 2015-05-01   to  2020-04-30


Take a look of project's partnership.

# participants  country  role  EC contrib. [€] 
1    OCADO GROUP PLC UK (HATFIELD) coordinator 1˙768˙250.00
4    UNIVERSITY COLLEGE LONDON UK (LONDON) participant 982˙250.00


 Project objective

The subject of this proposal is a robot assistant that is trained to understand maintenance tasks so that it can either pro-actively or as a result of prompting, offer assistance to maintenance technicians performing routine and preventative maintenance. Conceptually the robot's task is to provide a second pair of hands to the maintenance engineer, such that once the robot has been trained it can predict when and how it can usefully provide help.

The robot's behavioural repertoire is learnt in a training phase that includes the monitoring of maintenance technician activity, the construction of a knowledge base that describes the context of a task, and a theory of action that enables dynamic behaviour generation. The result is a set of competencies coupled with an ability to recognise the state of a task and an understanding of how these competencies can be usefully deployed given the state.

The scope of work includes the construction of a robot assistant, the systems that facilitate the training, the actual training on a number of representative tasks, perceptual systems that facilitate activity recognition, and validation of the system's ability to usefully contribute to tasks in collaboration with a maintenance engineer. Assessment of the system will test its ability to recognise when it doesn't know something as well as its ability to generalise its knowledge to previously unseen tasks.


List of deliverables.
Task generation and task switching preliminary implementation Other 2019-08-29 13:28:51
3D object recognition for a small number of object categories and action recognition for a simple vocabulary of gestures. Interface with other work packages Other 2019-08-29 13:28:55
Preliminary Knowledge learning model in completely known scenario; planning and execution of a pre-generated maintenance task Other 2019-04-29 08:15:04
3D Human pose estimation in uncluttered lab-based background. Interface with other work packages Other 2019-04-29 08:15:04
Dense 3D geometry acquisition and segmentation in a simplified lab environment (no articulated or non-rigid objects). Interface with other work packages Other 2019-04-29 08:15:04

Take a look to the deliverables list in detail:  detailed list of SecondHands deliverables.


year authors and title journal last update
List of publications.
2016 Jan Niehues, Thai Son Nguyen, Eunah Cho, Thanh-Le Ha, Kevin Kilgour, Markus Müller, Matthias Sperber, Sebastian Stüker, and Alex Waibel
Dynamic transcription for low-latency speech translation
published pages: , ISSN: , DOI:
Proceedings of INTERSPEECH Annual 2019-05-29
2015 M. Sanzari, F. Natola, F. Nardi, V. Ntouskos, M. Qudseya, and F. Pirri
Rigid tool affordance matching points of regard
published pages: , ISSN: , DOI:
\"Proceedings of the IROS 2015 Workshop \"\"Learning object affordances: a fundamental step to allow prediction, planning and tool use?\"\"\" 2019-05-29
2016 F. Natola, V. Ntouskos, F. Pirri, and M. Sanzari
Single image object modeling based on BRDF and r-surfaces learning
published pages: , ISSN: , DOI:
IEEE conference on Computer Vision and Pattern Recognition Annual 2019-05-29
2016 Mirko Wächter, Simon Ottenhaus, Manfred Kröhnert, Nikolaus Vahrenkamp, Tamim Asfour
The ArmarX Statechart Concept: Graphical Programing of Robot Behavior
published pages: , ISSN: 2296-9144, DOI: 10.3389/frobt.2016.00033
Frontiers in Robotics and AI 3 2019-05-29
2018 Penate-Sanchez, Adrian; Agapito, Lourdes
3D Pick & Mix: Object Part Blending in Joint Shape and Image Manifolds
published pages: , ISSN: , DOI:
Proceedings of the Asian Conference on Computer Vision (ACCV’18) 2 2019-06-06
2019 Manuel A. Ruiz Garcia, Rafael A. Rojas, Fiora Pirri
Object-Centered Teleoperation of Mobile Manipulators With Remote Center of Motion Constraint
published pages: 1745-1752, ISSN: 2377-3766, DOI: 10.1109/LRA.2019.2897172
IEEE Robotics and Automation Letters 4/2 2019-06-06
2019 Camil Demetrescu, Francesco Lupia, Angelo Mendicelli, Andrea Ribichini, Francesco Scarcello, Marco Schaerf
On the Shapley value and its application to the Italian VQR research assessment exercise
published pages: 87-104, ISSN: 1751-1577, DOI: 10.1016/j.joi.2018.11.008
Journal of Informetrics 13/1 2019-06-06
2019 Constantin, Stefan; Niehues, Jan; Waibel, Alex
Multi-task learning to improve natural language understanding
published pages: , ISSN: , DOI:
Proceedings of the Tenth International Workshop on Spoken Dialogue Systems (IWSDS) 10 2019-06-06
2018 Camil Demetrescu, Andrea Ribichini, Marco Schaerf
Accuracy of author names in bibliographic data sources: an Italian case study
published pages: 1777-1791, ISSN: 0138-9130, DOI: 10.1007/s11192-018-2945-x
Scientometrics 117/3 2019-06-06
2019 Mauro, Lorenzo; Puja, Francesco; Grazioso, Simone; Ntouskos, Valsamis; Sanzari, Marta; Alati, Edoardo; Pirri, Fiora
Visual search and recognition for robot task execution and monitoring
published pages: 94-109, ISSN: , DOI: 10.3233/978-1-61499-929-4-94
Frontiers in Artificial Intelligence and Applications 2 2019-06-06
2016 Ntouskos, Valsamis; Pirri, Fiora
Confidence driven TGV fusion
published pages: , ISSN: , DOI:
5 2019-05-27
2019 Marta Sanzari, Valsamis Ntouskos, Fiora Pirri
Discovery and recognition of motion primitives in human activities
published pages: e0214499, ISSN: 1932-6203, DOI: 10.1371/journal.pone.0214499
PLOS ONE 14/4 2019-05-27
2018 Nikolaus Vahrenkamp, Eduard Koch, Mirko Wachter, Tamim Asfour
Planning High-Quality Grasps Using Mean Curvature Object Skeletons
published pages: 911-918, ISSN: 2377-3766, DOI: 10.1109/lra.2018.2792694
IEEE Robotics and Automation Letters 3/2 2019-05-27
2017 Puja, Francesco; Grazioso, Simone; Tammaro, Antonio; Ntouskos, Valsmis; Sanzari, Marta; Pirri, Fiora
Vision-based deep execution monitoring
published pages: , ISSN: , DOI:
CoRR 2017 1 2019-05-27
2018 Constantin, Stefan; Niehues, Jan; Waibel, Alex
An End-to-End Goal-Oriented Dialog System with a Generative Natural Language Response Generation
published pages: , ISSN: , DOI:
IWSDS 2018 1 2019-05-27
2018 Rünz, Martin; Buffier, Maud; Agapito, Lourdes
MaskFusion: Real-Time Recognition, Tracking and Reconstruction of Multiple Moving Objects
published pages: , ISSN: , DOI:
IEEE International Symposium on Mixed and Augmented Reality (ISMAR) 2018 1 2019-05-27
2017 Lorenzo Porzi, Samuel Rota Bulo, Adrian Penate-Sanchez, Elisa Ricci, Francesc Moreno-Noguer
Learning Depth-Aware Deep Representations for Robotic Perception
published pages: 468-475, ISSN: 2377-3766, DOI: 10.1109/LRA.2016.2637444
IEEE Robotics and Automation Letters 2/2 2019-05-27
2016 Liu-Yin, Qi; Yu, Rui; Agapito, Lourdes; Fitzgibbon, Andrew; Russell, Chris
Better Together: Joint Reasoning for Non-rigid 3D Reconstruction with Specularities and Shading
published pages: , ISSN: , DOI:
British Machine Vision Conference (BMVC) 2016 1 2019-05-27
2017 Ruede, Robin; Müller, Markus; Stüker, Sebastian; Waibel, Alex
Yeah, Right, Uh-Huh: A Deep Learning Backchannel Predictor
published pages: , ISSN: , DOI:
Proceedings of the Eight International Workshop on Spoken Dialogue System 3 2019-05-27
2016 Matthias Plappert, Christian Mandery, Tamim Asfour
The KIT Motion-Language Dataset
published pages: 236-252, ISSN: 2167-6461, DOI: 10.1089/big.2016.0028
Big Data 4/4 2019-05-27
2016 José Medina, Klas Kronander, and Aude Billard
Risk-sensitive Model Predictive Control for Reproducing Uncertain Dynamical Systems
published pages: , ISSN: , DOI:
Workshop on Dynamic Locomotion and Manipulation 2019-05-30

Are you the coordinator (or a participant) of this project? Plaese send me more information about the "SECONDHANDS" project.

For instance: the website url (it has not provided by EU-opendata yet), the logo, a more detailed description of the project (in plain text as a rtf file or a word file), some pictures (as picture files, not embedded into any word file), twitter account, linkedin page, etc.

Send me an  email ( and I put them in your project's page as son as possible.

Thanks. And then put a link of this page into your project's website.

The information about "SECONDHANDS" are provided by the European Opendata Portal: CORDIS opendata.

More projects from the same programme (H2020-EU.


REaching and grasping Training based on Robotic hybrid AssIstance for Neurological patients: End users Real life evaluation

Read More  

RoMaNS (2015)

Robotic Manipulation for Nuclear Sort and Segregation

Read More  

SecondHands (2015)

SecondHands: A Robot Assistant For Industrial Maintenance Tasks

Read More