Opendata, web and dolomites

Report

Teaser, summary, work performed and final results

Periodic Reporting for period 2 - U-Test (Testing Cyber-Physical Systems under Uncertainty: Systematic, Extensible, and Configurable Model-based and Search-based Testing Methodologies)

Teaser

The U-Test project aims at ensuring that CPSs are tested adequately under uncertainty using systematic and automated techniques such as model and search-based testing to guarantee their dependable operation in real environment.The overall objective of the U-Test project is to:...

Summary

The U-Test project aims at ensuring that CPSs are tested adequately under uncertainty using systematic and automated techniques such as model and search-based testing to guarantee their dependable operation in real environment.
The overall objective of the U-Test project is to:
“Improve the dependability of CPSs, via cost-effective model-based and search-based testing of CPSs under uncertainty, by defining an Uncertainty Taxonomy and holistic modelling and testing frameworks with considerable reliance on standards.”
The overall objective will be met by the following project objectives:
1. O1: Provide unified and precise definitions of uncertainties in CPSs and systematically classify and characterize them, by developing a comprehensive Uncertainty Taxonomy (U-Taxonomy) with the ultimate aim of enabling the reuse, potentially via standardization, of the taxonomy across a wide range of diverse domains.
2. O2: Enable systematic, holistic and standard-based modelling of uncertainties in CPSs by developing a configurable and extensible Uncertainty Modelling Framework (UMF) utilizing existing standards.
3. O3: Developing a smart, systematic and automated approach of discovering realistic unknown uncertainties relying on existing search algorithms to support testing.
4. O4: Develop a standard-based and configurable Uncertainty Testing Framework (UTF) to generate cost effective test cases (e.g., achieving high coverage and having high chances of catching faults with lowest possible cost (in terms of e.g., test case execution time, test case generation time) possible by utilizing search algorithms (e.g., genetic algorithms).

Work performed

As discussed in Section 1.1, there are four project objectives (O1 to O4).
During the first reporting year (2015), the first objective was achieved, i.e., O1: Providing a comprehensive and extensible taxonomy of uncertainties, classifying uncertainties, their properties, and their relationships. We delivered an uncertainty taxonomy—U-Taxonomy that was validated with the pilots use cases.
With respect to O2, during 2016, we have delivered a second version of Uncertainty Modeling Framework (UMF) with the final version due in April 2017. We provided implementation of U-Taxonomy as a UML profile covering the three levels of CPS using UML structural and behavioral models. The UML Testing Profile (UTP) is also integrated in the UMF. Furthermore, we implemented an additional set of model libraries in the UMF. At its current state, we have created pilot models using the UMF for the following percentages: 1) Application Level: ULMA 55%, FPX: 66%, 2) Infrastructure: ULMA/FPX: 60%, 3) Integration: ULMA/FPX: 66%. During the delivery of the final version of UMF, we expect to deliver test ready models of 100% use cases at all the three levels.
With respect to O3 and O4, we have delivered an initial version of Uncertainty Testing Framework (UTF) in this reporting period. The second and final versions are due in 2017. During 2016, we have developed in total 14 test strategies that explicitly take into consideration uncertainty (O4) and two initial versions of model evolution algorithms to evolve test ready models (O3). A subset of these test strategies is already implemented in the tools, i.e., FOKUS!MBT and CertifyIt. Using the tools, test cases were generated with the implemented strategies for a subset of use cases for the both pilots. The generated test cases were executed on both the pilots and results of test execution were delivered to the pilots for further analysis, such as discovery of new uncertainties. Moreover, test strategies are also prototyped by researcher partners in their tools for experimentation.

Final results

The project exploitation activities aim to ensure that the project results will effectively be exploited in the market. Hence, they bridge between the RTD activities and the project expected impact objectives. A parallel exploitation goal is to deal with the intellectual property rights’ (IPRs) background and foreground issues during the project, in order to establish a fair and successful collaborative exploitation strategy. The four key objectives for this work package are: (1) Identify U-Test’s potential value opportunities; (2) Define business oriented exploitation approaches for the value opportunities; business models and business plans basics; (3) Bring the project’s exploitable results’ paths progressively closer to the impact objectives through product-market contrasts, and; (4) Establish consortium’s collaborative exploitation strategies and foreground IPRs agreements to facilitate partners’ exploitation initiatives.
Standardization plays an important role in a modern society, which is being increasingly globe. To be competitive internationally, many industries are encouraged or forced to conform to various standards. In other words, standards make an important contribution to national and international competitiveness, which is now a well-recognized fact, as one can see from the list of membership of OMG (http://www.omg.org/cgi-bin/apps/membersearch.pl), where organizations such as NASA, THALES, Airbus Group play roles in OMG standardization activities. To be more specific, in the context of the project, our aim of conforming our solutions to existing and widely adopted standards is an important strategy to improve the competitiveness of the consortium.
During 2016, the U-Test project team has focused on consolidating not only the results of research work but also actively promoting and communicating the U-Test’s solution to the CPS market. The National Science Foundation of the U.S. predicted that the deployment of CPS is going to revolutionize the way people interact with engineered systems like how the Internet has transformed the way people interact with information . However, the inherent complexity of CPS as well as the constraints regarding safety, privacy, and security bring huge challenges to the fast growing application of CPS. In this context, U-Test project addresses an essential aspect of CPS i.e. uncertainty, and looks into improving the dependability of CPS, via cost-effective model-based and search-based testing. The values that U-Test could contribute to the CPS market must be actively promoted. In particular, our dissemination and communication activities have purposefully reached the core groups of the U-Test’s community and beyond. The core groups of the U-Test’s community are composed of end user community, academia, and industry, which develop and deploy CPS in real life applications. First and foremost, the U-Test’s research results have been significantly disseminated to the academic world in terms of research articles written by U-Test project team. As can be seen on the U-Test’s website, the number of published research articles had increased significantly in the second project year. The acceptance of our research work for publication in top publication venues with rigorous peer review processes has shown the recognition of the academic world to the U-Test’s research results.

Website & more info

More info: http://www.u-test.eu.