Report

Teaser, summary, work performed and final results

Periodic Reporting for period 2 - SeaDataCloud (SeaDataCloud - Further developing the pan-European infrastructure for marine and ocean data management)

Teaser

Oceanographic and marine data are collected by over a thousand research institutes, governmental organisations and private companies in the countries bordering the European seas by various heterogeneous observing sensors and platforms. These data are collected at a very...

Summary

Oceanographic and marine data are collected by over a thousand research institutes, governmental organisations and private companies in the countries bordering the European seas by various heterogeneous observing sensors and platforms. These data are collected at a very considerable cost and can be of prime value for many applications, if well managed and available.
SeaDataNet operates an essential pan-European infrastructure for managing, indexing and providing access to marine data sets and products, from research cruises and other observations, worldwide. Core partners are NODCs and major institutes from 34 states bordering European seas. The network is well embedded in the marine community, with links to EuroGOOS, CMEMS, EMODnet, EurOBIS, EU research projects, observing networks, Euro-ARGO, MSFD, , and internationally via ICES, IOC-IODE, IHO, WMO, GEOSS, and others.
The SeaDataCloud project has the following main objectives:
1. To enhance and innovate the SeaDataNet standards, products and services offered to an expanded multi-disciplinary community:
2. To promote the adoption of the protocols and standards developed with a perspective to expand the communities of data providers and users and exploiting synergies with relevant ESFRI infrastructures:
3. To bring together, document, maintain, and give access to large collections of marine environmental data sets for physics, chemistry, biology, geophysics, and bathymetry from European data originators, to serve many user communities, including major initiatives such as CMEMS, EMODnet, MSFD, INSPIRE, and others
4. To present a long-term sustainable arrangement for the integrated SeaDataNet infrastructure and its network.
The project aims at adopting cloud solutions and to build upon the state of the art in ICT and e-infrastructures for data, computing and networking. Therefore the SeaDataNet network has initiated a strategic cooperation with the EUDAT network of computing infrastructures, who are also well engaged in the EOSC development.

Work performed

SeaDataNet publishes European directories for marine organisations in Europe. Their population has increased considerably in numbers and content. For instance, the CDI catalogue has increased from 1.87 million to 2.28 million entries, while number of connected data centres has expanded from 103 to 113, providing harmonised access to data for physics, chemistry, geology, geophysics, and biology from 725 data originators.
The cloud environment is operational providing a data cache that hosts all public data (87% of total CDI). The new Replication Manager (RM) software, for the replication of the public data in the EUDAT cloud, has been deployed in all SDN data centres. The RM manages the exchanges between SeaDataNet data centres and the EUDAT cloud through the Import Manager (also a new software).
The new CDI user interface has been launched. It uses a combination of elastic search and relational database technologies and is very powerful and fast, offering plenty of new features such as: combination of search criteria from pull down lists and search by facets, free text search on all metadata fields, full screen mapping, My SeaDataNet for customized services...

A series of technical developments have been undertaken, next to the CDI service upgrading, and each have made considerable progress:
• Upgrading and expansion of the SeaDataNet common vocabularies, supporting a variety of groups
• Successfully analysing the feasibility of transforming SeaDataNet output formats to INSPIRE metadata and data implementation rules
• Delivering upgraded versions of the SeaDataNet data management tools NEMO, MIKADO, and OCTOPUS
• Public opening of the online SWE ingestion service with a new ingestion workflow using the online SensorML editor smle and new SOS Viewing Services
• Developing and publishing with other projects standards for handling High Frequency Radar (HFR) and Flow Cytometer data
• Launching the prototype SDC Virtual Research Environment (VRE) in the EUDAT cloud for a collaborative and individual research: the VRE prototype and its advanced services are operational (using Docker containers)
• Launching a SeaDataNet DOI minting service which invites and facilitates scientists to publish their research data in the field of marine sciences as citable resources
• Releasing 6 new regional and 1 global ocean data collections of temperature and salinity in 6 regional sea areas and their corresponding climatologies, as well as the corresponding documentation (PIDoc)
Two training workshops have been organised in order to train the data centres for the uptake and deployment of the new tools and software.

Final results

The ambition is to advance the SeaDataNet infrastructure in such a way that it will give researchers easy access to available marine data resources, provided by many data centres. It must be attractive and easy for data centres to connect and to disclose their harmonised data. The realisation of this ambition is well underway as considerable progress is made with standards, services, and tools. Some examples:
• New the import process for new and updated CDI metadata and associated data sets, introducing successfully cloud technology and giving data centres more self-control and services for managing the import process
• Deployment of the upgraded CDI with a new optimised GUI providing users much improved performance for deliveries of ordered data and a more sophisticated user interface for discovering and ordering data sets
• Enhancement of the NVS vocabulary with deprecation management, exposition of previous versions and version history of vocabularies; GitHub set up as the platform for governance of vocabulary content
• Opening of the cloud based VRE: each of the advanced services already operational and tested by the regional product leaders of the project for the preparation of the 2nd version of the SeaDataCloud products
• Opening of the online SWE ingestion service to support sensor operators and scientists in streaming and publishing observation data from sensors at platforms
• Production of standards and guidelines for handling different data types such as High Frequency Radar, Flow cytometer and Glider data.

SeaDataCloud is promoted through the networks of partners and associations and by a range of media:
• 94 presentations in different conferences, workshops, and meetings
• 24 posters
• 3 videos
• 1 SeaDataCloud workshop at the EUDAT Conference
• 3 newsletters communicated to more than 5000 potential readers
• Organisation of the 6th edition of the International conference on Marine Data and Information Systems (IMDIS), in Barcelona in November 2018 (184 participants from 34 countries worldwide)

A MoU between SeaDataCloud and the Copernicus Marine Environmental Monitoring Service (CMEMS) was signed.

The last year of the project will be devoted to:
• improve the robustness of all the new components,
• finalise the missing components of the VRE which concerns the on-line biological quality checks and sub-setting tool,
• complete the Brokerage Service to offer SeaDataNet users also easy discovery of data collections of leading international marine data portals,
• develop the INSPIRE data transformation service,
• populate new data types such as from gliders and HFR in the CDI catalogue,
• organise two workshops: for user and for product leaders,
• organise the 7th edition of IMDIS conference

Website & more info

More info: http://www.seadatanet.org.