Opendata, web and dolomites

BeyondBlackbox SIGNED

Data-Driven Methods for Modelling and Optimizing the Empirical Performance of Deep Neural Networks

Total Cost €


EC-Contrib. €






Project "BeyondBlackbox" data sheet

The following table provides information about the project.


Organization address
postcode: 79098

contact info
title: n.a.
name: n.a.
surname: n.a.
function: n.a.
email: n.a.
telephone: n.a.
fax: n.a.

 Coordinator Country Germany [DE]
 Total cost 1˙495˙000 €
 EC max contribution 1˙495˙000 € (100%)
 Programme 1. H2020-EU.1.1. (EXCELLENT SCIENCE - European Research Council (ERC))
 Code Call ERC-2016-STG
 Funding Scheme ERC-STG
 Starting year 2017
 Duration (year-month-day) from 2017-01-01   to  2021-12-31


Take a look of project's partnership.

# participants  country  role  EC contrib. [€] 


 Project objective

Deep neural networks (DNNs) have led to dramatic improvements of the state-of-the-art for many important classification problems, such as object recognition from images or speech recognition from audio data. However, DNNs are also notoriously dependent on the tuning of their hyperparameters. Since their manual tuning is time-consuming and requires expert knowledge, recent years have seen the rise of Bayesian optimization methods for automating this task. While these methods have had substantial successes, their treatment of DNN performance as a black box poses fundamental limitations, allowing manual tuning to be more effective for large and computationally expensive data sets: humans can (1) exploit prior knowledge and extrapolate performance from data subsets, (2) monitor the DNN's internal weight optimization by stochastic gradient descent over time, and (3) reactively change hyperparameters at runtime. We therefore propose to model DNN performance beyond a blackbox level and to use these models to develop for the first time:

1. Next-generation Bayesian optimization methods that exploit data-driven priors to optimize performance orders of magnitude faster than currently possible; 2. Graybox Bayesian optimization methods that have access to -- and exploit -- performance and state information of algorithm runs over time; and 3. Hyperparameter control strategies that learn across different datasets to adapt hyperparameters reactively to the characteristics of any given situation.

DNNs play into our project in two ways. First, in all our methods we will use (Bayesian) DNNs to model and exploit the large amounts of performance data we will collect on various datasets. Second, our application goal is to optimize and control DNN hyperparameters far better than human experts and to obtain:

4. Computationally inexpensive auto-tuned deep neural networks, even for large datasets, enabling the widespread use of deep learning by non-experts.


year authors and title journal last update
List of publications.
2018 Matthias Feurer Katharina Eggensperger Stefan Falkner Marius Lindauer Frank Hutter
Practical Automated Machine Learning for the AutoML Challenge 2018
published pages: , ISSN: , DOI:
ICML 2018 2020-04-15
2019 Jör Franke, Jörg Gregor Köhler Noor Awad Frank Hutter
Neural Architecture Evolution in Deep Reinforcement Learning for Continuous Control
published pages: , ISSN: , DOI:
NeurIPS 2019 2020-04-15
2018 Matthias Feurer Frank Hutter
Towards Further Automation in AutoML
published pages: , ISSN: , DOI:
ICML 2018 2020-04-15
2019 Hutter, Frank Elsken, Thomas Metzen, Jan Hendrik
Efficient Multi-Objective Neural Architecture Search via Lamarckian Evolution - Published as a conference paper at ICLR 2019
published pages: , ISSN: , DOI:
2017 Bischl, Bernd; Casalicchio, Giuseppe; Feurer, Matthias; Hutter, Frank; Lang, Michel; Mantovani, Rafael G.; van Rijn, Jan N.; Vanschoren, Joaquin
OpenML Benchmarking Suites and the OpenML100
published pages: , ISSN: , DOI:
1 2019-08-29
2019 Runge, Frederic; Stoll, Danny; Falkner, Stefan; Hutter, Frank
Learning to Design RNA
published pages: , ISSN: , DOI:
ICLR 2019 5 2019-08-29
2018 Chrabaszcz, Patryk; Loshchilov, Ilya; Hutter, Frank
Back to Basics: Benchmarking Canonical Evolution Strategies for Playing Atari
published pages: , ISSN: , DOI:
IJCAI 2018 1 2019-08-29
2019 Hutter, Frank, Kotthoff, Lars, Vanschoren, Joaquin (Eds.)
Automated Machine Learning: Methods, Systems, Challenges
published pages: , ISSN: , DOI:
The Springer Series on Challenges in Machine Learning 2019-08-29
2018 Zela, Arber; Klein, Aaron; Falkner, Stefan; Hutter, Frank
Towards Automated Deep Learning: Efficient Joint Neural Architecture and Hyperparameter Search
published pages: , ISSN: , DOI:
AutoML Workshop 1 2019-08-29
2018 Falkner, Stefan; Klein, Aaron; Hutter, Frank
BOHB: Robust and Efficient Hyperparameter Optimization at Scale
published pages: , ISSN: , DOI:
ICML 2018 5 2019-08-29
2019 Loshchilov, Ilya; Hutter, Frank
Decoupled Weight Decay Regularization
published pages: , ISSN: , DOI:
ICLR 2019 5 2019-08-29
2017 Chrabaszcz, Patryk; Loshchilov, Ilya; Hutter, Frank
A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasets
published pages: , ISSN: , DOI:
arXiv 5 2019-08-29
2019 Ying, Chris; Klein, Aaron; Real, Esteban; Christiansen, Eric; Murphy, Kevin; Hutter, Frank
NAS-Bench-101: Towards Reproducible Neural Architecture Search
published pages: , ISSN: , DOI:
ICML 2019 5 2019-08-29
2018 Wilson, James T.; Hutter, Frank; Deisenroth, Marc Peter
Maximizing acquisition functions for Bayesian optimization
published pages: , ISSN: , DOI:
NeurIPS 2018 1 2019-08-29
2019 Elsken, Thomas; Metzen, Jan Hendrik; Hutter, Frank
Neural Architecture Search: A Survey
published pages: , ISSN: 1533-7928, DOI:
JMLR 5 2019-08-29
2017 Aaron Klein, Stefan Falkner, Jost Tobias Springenberg, Frank Hutter
Learning curve predictionwith bayesian neural networks
published pages: , ISSN: , DOI:
proceedings of ICLR 2019-06-13
2017 Aaron Klein, Stefan Falkner, Numair Mansur, Frank Hutter
RoBO: A Flexible and Robust Bayesian OptimizationFramework in Python
published pages: , ISSN: , DOI:
Proceedings of BayesOpt 2017 2019-06-13
2017 Jan N. van Rijn, Frank Hutter
An Empirical Study of Hyperparameter Importance Across Datasets
published pages: , ISSN: , DOI:
proceedings of AutoML 2019-06-13
2017 Klaus Greff, Aaron Klein, Martin Chovanec, Frank Hutter, Jürgen Schmidhuber
The Sacred Infrastructure for ComputationalResearch
published pages: , ISSN: , DOI:
proceedings of the 15th python in science conference 2019-06-13
2017 James T. Wilson, Riccardo Moriconi, Frank Hutter, Marc Peter Deisenroth
The reparameterization trick for acquisition functions
published pages: , ISSN: , DOI:
Proceedings of BayesOpt 2017 2019-06-13
2017 Aaron Klein, Stefan Falkner, Simon Bartels, Philipp Hennig, Frank Hutter
Fast Bayesian hyperparameter optimization on large datasets
published pages: , ISSN: 1935-7524, DOI:
Electronic Journal of Statistics 2019-06-13
2018 Jan N. van Rijn, Frank Hutter
Hyperparameter Importance Across Datasets
published pages: , ISSN: , DOI:
Proceedings of KDD 2018 2019-06-13
2017 Stefan Falkner, Aaron Klein, Frank Hutter
Combining Hyperband and Bayesian Optimization
published pages: , ISSN: , DOI:
Proceedings of BayesOpt 2017 2019-06-13

Are you the coordinator (or a participant) of this project? Plaese send me more information about the "BEYONDBLACKBOX" project.

For instance: the website url (it has not provided by EU-opendata yet), the logo, a more detailed description of the project (in plain text as a rtf file or a word file), some pictures (as picture files, not embedded into any word file), twitter account, linkedin page, etc.

Send me an  email ( and I put them in your project's page as son as possible.

Thanks. And then put a link of this page into your project's website.

The information about "BEYONDBLACKBOX" are provided by the European Opendata Portal: CORDIS opendata.

More projects from the same programme (H2020-EU.1.1.)


The Mass Politics of Disintegration

Read More  

InsideChromatin (2019)

Towards Realistic Modelling of Nucleosome Organization Inside Functional Chromatin Domains

Read More  

ECOLBEH (2020)

The Ecology of Collective Behaviour

Read More