#	Pagina
attuale pagina	/open-h2020/projects/208283/results.html

Report

Teaser, summary, work performed and final results

Periodic Reporting for period 1 - So2Sat (Big Data for 4D Global Urban Mapping â€“ 10^16 Bytes from Social Media to EO Satellites)

Teaser

Summary

By 2050, around three quarters of the worldâ€™s population will live in cities. Despite of increasing efforts, global urban mapping still drags behind the geometric, thematic and temporal resolutions of geo-information needed to address these challenges. Nowadays diverse sets of incomplete data exist. For example, Earth observation (EO) satellites reliably provide geodetically accurate large scale geo-information of the cities on a routine basis from space. But the data availability is limited by resolutions and acquisition geometries of the sensors. Complementary, massive amounts of imagery, text messages and GIS data from open sources and social media provide a temporally quasi-seamless, spatially multi-perspective information basis, but with unknown and diverse qualities. So2Sat aims at a joint exploitation of big data from social media and satellite observations for global urban mapping, and aims at breakthroughs in 3D/4D urban modelling, infrastructure occupancy classification, and very high resolution population density mapping on a global scale for revolutionizing urban geographic research. In detail, the following methodological and application objectives will be addressed: improving urban-related information retrieval from EO satellite data, mining urban imagery and text messages from social media data, fusion of heterogeneous data sources, big data processing, as well as pilot application research regarding informal settlements classification and global population density estimation. The outcome of So2Sat will be the first and unique global and consistent spatial data set on urban morphology (3D/4D) of settlements, and a multidisciplinary application derivative assessing population density.

Work performed

The work performed during the first financial reporting period (months 1-18 of the project duration) can be structured into four major work packages:
In the field of 3D/4D urban mapping, a prototypical algorithm based on compressive sensing, nonlocal filtering, and deep learning was developed to generate 3D city models on a global scale, exploiting a minimum of interferometric SAR and auxiliary data. A first showcase to demonstrate the feasibility of the algorithm was achieved for the study city of Munich, Germany. Validation with respect to highly accurate LiDAR reference data covering about 30,000 buildings showed a mean height reconstruction accuracy of less than 2m. In addition, preparations for global processing were successfully started: big data pipelines for steps such as data ordering, InSAR pre-processing and the parallelization of the information extraction algorithms were developed. The project team will thus be able to start a global 3D urban analysis in the next period.
In the field of 2D urban classification, thorough investigations were carried out about how to segment urban areas semantically into meaningful areal classes based on diverse satellite data inputs: Sentinel-1 C-band SAR data, Sentinel-2 multi-spectral imagery, or TanDEM-X X-band InSAR data. The backbone of the research was a convolutional neural network, adapted to deal with the different input data peculiarities and the local climate zone (LCZ) target classification scheme. The outcomes of the investigations have shown that up to about 60% of average accuracy can be achieved for 17 target classes on completely unseen data. To foster additional research and public interest in local climate zone classification, we have published an open access dataset consisting of Sentinel-1 and Sentinel-2 image patches as well as the corresponding LCZ labels, which is currently used in the frame of a deep learning competition on the Tianchi platform operated by Alibaba Cloud. Last, but not least, a prototype for global semantic mapping of urban areas was developed and applied to the 1,692 largest cities of the world to produce local climate zone maps for them. Thus, the project team is ready to carry out fully global semantic urban mapping in the next reporting period.
In the field of social media data analysis, a workflow for unstructured geospatial data management was developed and implemented. Using it as a backbone, first results regarding the classification of human settlements based on social media images and texts (i.e. tweets) were achieved. The results highlight the promising perspective of social media analysis for semantic urban mapping and directly link to the last work package on anthropogeographic application research: both satellite and social media data have been used to analyze both so-called digital deserts and slums in several case studies.

Final results

Triggered by the need for methods from the field of artificial intelligence (AI) in the project, the project group has become a leading group with regard to the application of deep learning in Earth observation and has started to define the corresponding state-of-the-art in the remote sensing community. By the end of the project, the group will have consolidated this position, and several innovative algorithms fine-tuned to both semantic (2D) and topographic (3D/4D) urban analysis will have been developed and made available to the public in the form of open access publications.