SANSA And SARAO To Commission The Digital Earth South Africa Data Cube

Conference Banner

Data analysts have long encountered challenges in the interpretation of large datasets for use in decision-making. With the advent of data cubes in recent years, the burden on analysts has been alleviated through reliance on multidimensional arrays of values represented in the form of a data cube. In light of these developments, discussions are underway for a collaboration between the South African National Space Agency (SANSA) and the South African Radio Astronomy Observatory (SARAO) to commission the Digital Earth South Africa (DESA) platform.

“The Digital Earth South Africa platform is a data cube for ingesting analysis-ready satellite imagery from the national satellite archive, which will facilitate the development of earth observation products and services,” said SANSA CEO Valanathan Munsami.

SANSA has a legislative mandate to drive the promotion and use of space and cooperation in space-related activities and to this end, has partnered with The South African Radio Astronomy Observatory (SARAO), a facility of the National Research Foundation, to launch the DESA platform. SARAO is responsible for the administration of all radio astronomy initiatives and facilities in South Africa, including amongst others, the MeerKAT Radio Telescope in the Karoo, and the Geodesy and VLBI activities at the HartRao facility.

Space agencies and big data repositories globally are responding to the world’s challenges by leveraging Fourth Industrial Revolution (4IR) technologies such as cloud computing services, machine learning and analysis-ready data standards to convert raw data into actionable insights for decision-making products, suited for local challenges and needs.

Tailoring its services to the South African and African market, SANSA and SARAO aim to leverage their expertise in these emerging technologies in the development of DESA.

According to SANSA records, “DESA will build on data cube technology and analysis-ready data standards to deliver a unique capability to store, manage, process, interrogate, and present Earth observation data in formats that further promote its usefulness to support information-based decision-making processes.” It will facilitate this process by translating over 30 years of Earth observation satellite imagery stored in SANSA’s archives, into actionable information and insights, with reliable, standardised and easily accessible products and services.

SANSA plans to integrate on-site measurements such as ground-based, airborne and ship observations and measurements as well as demographic data for comprehensive multi-level analysis.

The DESA solution will be based at the Earth Observation Data Centre (EODC) located at the Hartebeesthoek ground station, which is the primary location where storage and processing servers are hosted.

How would DESA work?

The data cube is a series of data structures and tools which organise and enable the analysis of large Earth Observation satellite data collections and uses the open-source code and application development provided by the Open Data Cube community.

Australia, Columbia, Switzerland, Vietnam and the United Kingdom have implemented this technology at scale with measurable success. Vital to this implementation is the standardisation and calibration of data, which has a positive impact on the value of the Earth Observation data and other large datasets. This is due to the fact that this process allows for the rapid development of information products that enable informed decision-making across government and private industry.

In the past, satellite imagery and geospatial datasets were downloaded, analysed, and provided to users on a custom basis. This was neither time nor cost-efficient, particularly for single-use purposes. By calibrating the entire data stream to the same standard in advance as well as making the data accessible in a High-Performance Data (HPD) structure co-located within a High-Performance Computing (HPC) facility, DESA provides enabling infrastructure for data-intensive science.

DESA will then organise this calibrated data into stacks of consistent, time-stamped geographic ‘tiles’ that can be rapidly manipulated in an HPC environment. For this purpose, a database will be used to track the data. Despite the platform containing trillions of individual observations, the database will have the capacity to track every observation back to the point of collection. Accordingly, DESA will synthesize satellite images collected over a 30-year basis over a continuous period, including future images, providing these images and derived products in a platform that is accessible to any user, and will deliver a unique capability to process, interrogate and present this data in response to specific issues, for example, crop yield, water quality, land use and forest cover.