Skip to content
Rel
HOW-TO GUIDES
Data I/O: Accessing the Cloud

Data I/O: Accessing the Cloud

This guide demonstrates how to interact with data using the supported cloud providers.

Goal

By following this guide, you will be able to access data from the cloud in the RKGMS database and vice versa using the different available options.

This guide complements the CSV Import/CSV Export and JSON Import/Export guides, where you can find all the relevant information for importing and exporting specific data types.

Cloud Storage Providers

Currently, Amazon S3 and Azure Blob Storage are supported.

🔎

Note that S3 data must be in buckets in the (N. Virginia) us-east-1 region.

Cloud Parameters

To interact with the cloud storage service, you need to define a config module that specifies the data configuration. There are two relevant options for describing how to access the cloud:

OptionDescription
pathA string that specifies the location and name of the file you want to import/export. Currently, this can point to s3://... (Amazon S3) or azure://... (Microsoft Azure) URLs.
integrationCredentials needed to access the data.

Public Data

If the cloud storage location is publicly available (e.g.: this file), you only need to provide the path in the configuration option path.

module config
    def path = "s3://relationalai-documentation-public/csv-import/simple-import-4cols.csv"
end
🔎

For security reasons, "http://" and "https://" URL addresses are not supported.

Private Data

In order to access private data, you need to specify cloud credentials in integration.

Currently, only private Azure Blob Storage is supported. To access it, you must provide a valid SAS token and URL address, corresponding to a valid Azure account.

The URL is provided via the path option. Within the integration submodule, the following information needs to be provided:

  • the cloud storage provider in the provider field, and
  • the access token as a string in the credentials field, along with the token name (:azure_sas_token).
module config
    def path = "azure://myaccount.blob.core.windows.net/sascontainer/myfile.csv"

    module integration
        def provider = "azure"
        def credentials = (:azure_sas_token, "sv=2014-02-14&sr=b&si=TestPolicy&sig=o%2B5%2F0C%2BLm7tWWftNKvQEGKHlSt%2Bfs8No7FZkUk5T%2Bv0%3D")
    end
end

See Also

Now that you know how to access a service cloud provider, you can check our I/O how-to guides to learn how to interact with data. Check CSV Import and CSV Export for CSV data. If you are interested in JSON, go to JSON Import/Export.