How to Upload Any Data Into Sap System in Real Time

Overview

This is the 1st of 7 blog posts which are part of the SAP Business Engineering Platform Showcase series of blogs and videos. We invite you to check this overall blog, so you can understand the full stop-to-stop story and the context involving multiple SAP BTP solutions.

Here you volition see how to easily eat data from systems of records (east.one thousand. SAP ERP), cloud and on-premise databases (east.grand. SAP HANA, SQLServer, Oracle, Athena, Redshift, BigQuery, etc.), oData Services, csv/text files available in your own local computer, or whatever File/Object shop (eastward.thou. Amazon S3). Nosotros will leverage SAP Data Warehouse Deject's Replication and Information Menstruation capabilities, also every bit demonstrate how to access remote sources using data virtualization.

Beneath yous can see this is the 1st step of the "Solution Map" prepared for the journey on the referred overall blog:

SAP BTP Showcase – Overall Technical Architecture

As a recap, SAP Data Warehouse Deject is an analytic and persona-driven data warehouse-as-a-service tailored for concern and IT users. It provides instant access to data via pre-built business content and adapters to integrate data from various sources. It is scalable, rubberband, open up and flexible, making it suitable for any company of any size.

Introduction

In this blog we will handle the specific "Connection" service, used to connect more 20 types of datasources. You tin cheque all available technologies in in the SAP Help Portal – Connection Types.

Every bit an example, we have logged on SAP Information Warehouse Cloud, and created a new Space, named "DASC SHOWCASE" (DASC stands for SAP HANA Database & Analytics Solutions in the Cloud).

Once created, nosotros have to access the "DASC SHOWCASE" Space, and hit the "plus" (+) push button in order to create a new "Connection":

Permit's now work on multiple scenarios for loading data into SAP Data Warehouse Cloud.

Scenarios

Consume information from systems of records

In this showcase, we will show that SAP Data Warehouse Cloud provides multiple connections for SAP applications, as per the below examples:

For instance, permit'southward create a connectedness to an SAP ERP. We can employ the SAP ABAP or ECC Connectedness Types in this case:

After informing the credentials to the ABAP system (e.g. ECC, BW, S4/HANA, or BW4/HANA, etc.), you can see a message showing that the connectedness is valid.

And so, if you go to "Data Architect" and create whatever new artifact, like a "Graphical View", you lot will be able to choose a organization in the "Sources/Connection" tab, for case, "ABAP". Then y'all tin choose the object'south type, selection-upwards a source and drag&drop it to the white expanse in the graphical builder. Past clicking in "Preview Information", you can hands check the contents of the ERP/BW object, as per the figure below.

Consume data from databases

Now let's see how to connect to multiple database technologies, for example.

When you select a "Connection Type", you lot will get a screen to fill with connections backdrop for each of the technologies:

Allow'south connect to an SAP HANA Cloud instance for example:

Then, if you go to "Data Builder" and create whatsoever new antiquity, similar a "Graphical View", you will be able to choose a organization in the "Sources/Connexion" tab, for instance "HANA Deject". Then you can cull the object's type, pick-up a source and drag&drib information technology to the white area in the graphical builder, every bit per the picture beneath.

Eat data via oData Services

Now let's create a new "Connexion" for consuming oData service data.

Let'south say we will consume an oData service, provided by an SAP HANA Cloud Native application, as presented in Web log v: Develop a SAP HANA Cloud native application. You tin point to the oData URL generated past the application, as the example below:

You tin "right click" the "/production-program-values" and copy the link. Then you lot tin can paste it on the Connection holding, as per the screenshot below:

Then, if you get to "Data Builder" and create any new artifact, like a "Graphical View", you lot will be able to choose a arrangement in the "Sources/Connexion" tab, for instance "oData". So you can choose the object's type, pick-up a structure and drag&drop it to the white area in the graphical builder, as per the picture below.

Consume csv/text files from your own local computer

Now let's upload a local .csv case file, containing data virtually weekday's names.

In "Data Builder", just click on "Import CSV File", and select the .csv file that you desire to upload and persist in SAP Data Warehouse Cloud.

And so y'all can adjust the information structure appropriately, for example, renaming columns, irresolute datatypes, and fifty-fifty applying more sophisticated transformation rules.

Then yous take your data uploaded into SAP Data Warehouse Cloud, and you can come across the content of the table, equally per the figure below:

Replicate data from hyperscaler's file/object store using the "Data Flow" functionality

At present nosotros are going to create a new "Connection" for "Amazon Simple Storage Service", also known as "Amazon S3", so we can replicate this data, persisting information technology on SAP Data Warehouse Cloud. You just take to inform S3's Endpoint, Credentials and Root Path (which is the saucepan itself).

Then, if yous get to "Data Architect" and create a new "Data Flow", you will be able to choose a system in the "Sources/Connection" tab, for example, "S3-Filesystem". Then you tin choose the directory, pick-upwardly a source file and drag&drib it to the white expanse in the graphical builder. Past clicking in "Preview Data", you can easily check the contents of the file object.

So, you need to click in "Add table", equally presented in the figure below, so you can create a new table on SAP Data Warehouse Cloud'south persistence area (which is an SAP HANA Cloud database instance).

Now you inform the target table name, and click in "Create and Deploy Table", to effectively deploy this new table.

After that, you need to choose the load beliefs, "APPEND" or "TRUNCATE" mode. And then, you lot hit the salvage button in the left top corner of the screen, as demonstrated in the picture below:

Later on saving your "Data Menstruation", you are ready to execute it. Only click in the "Execute" button in the elevation left corner of your screen. The execution starts and you can see the Status as "Running", on the right middle of your screen. Then you can click in the symbol highlighted in cherry-red, in order to analyze the execution in the "Data Flow Monitor".

In the screenshot below, yous can run across that the execution was completed successfully, and a re-create of your dataset seating in Amazon S3 is set up to utilise, replicated and persisted in SAP Data Warehouse Deject.

Replicate data using "Tabular array/View Replication"

You can rely on "Information Integration Monitor" to replicate tables/views' contents from remote systems, persisting a copy of this data straight in SAP Information Warehouse Cloud. Just click on the object y'all want to replicate, then click in the "Table Replication" link, on the superlative right of your screen, like presented in the screenshot below:

If you need a ane-shot synchronization, then you can choose "Load New Snapshot", which volition replicate the entire object's contents into SAP Information Warehouse Deject's persistence area. That works for tables and views.

In example you lot need a real-fourth dimension synchronization betwixt the source system and SAP Data Warehouse Cloud, then you should choose "Enable Real-Fourth dimension Access". It first loads a snapshot of the source information into SAP Data Warehouse Cloud's persistence area, then keeps the replication synchronized in existent-fourth dimension. This strategy avoids intense read operations in source systems. For applying this technique, the database table in the source organisation has to be enabled for change data capture (CDC). For example, SAP HANA technology is CDC-enabled.

You can find detailed information for this feature in the SAP Assist Portal – Loading and Monitoring Remote Tables.

Once the replication of the snapshot is finished, then you can see an updated Status "Available".

So, if y'all go to "Data Builder" and open the replicated artifact, you will see that the Data Access is "Replicated", as highlighted in the screenshot below.

Summary

In this blog you could larn how to leverage SAP Data Warehouse Cloud'due south persistence area, consuming or replicating data from different source systems & technologies. This data can now be modeled by business users and consumed by analytic applications, as presented on the post-obit ii linked blogs:

Blog six: Provide governed business semantics with SAP Data Warehouse Cloud, and

Blog 7: Consume SAP Data Warehouse Cloud'southward assets using SAP Analytics Cloud.

All of your feedback is appreciated. Enjoy!

nordfouldlairity1967.blogspot.com

Source: https://blogs.sap.com/2021/01/29/sap-btp-showcase-load-data-into-sap-data-warehouse-cloud/