Overview

Network Effects of Combining Data 

Today we are working with more disparate data sources than ever before. When we talk about Big Data solutions, external data sources, as well as your own internal data, are important in order to perform advanced analytics to drive competitive advantage.

More organisations are realising the importance of external data which is accessible through APIs and that can be downloaded into cloud storage solutions where it can be combined with their own internal data.

The process of collecting data is called ingestion and there are a number of cloud managed services that can help with this process.

ELT over ETL 

Typically, we want to ingest the data in native format and store in the Data Warehouse of our choice so that we can return to the entirety of the data at a later date to perform particular transformations and analysis for a particular business objective

Therefore, in cloud and advanced analytic scenarios you could be more inclined to adopt an Extract, Load and Transform (ELT) model where you’re purely extracting and loading the data in native form.

The following sections detail how to perform Extract and Load jobs from DoordaHost to your target Cloud Provider environment using the applicable services available.