Skip to content
  • REL
  • HOW-TO GUIDES
  • Data I/O: Accessing the Cloud

Data I/O: Accessing the Cloud

This guide demonstrates how to interact with data using the supported cloud providers.

Goal

By following this guide, you will be able to access data from the cloud in the RKGMS database and vice versa using the different available options.

This guide complements the CSV Import/CSV Export and JSON Import/Export guides, where you can find all the relevant information for importing and exporting specific data types.

Cloud Storage Providers

Currently, Azure Blob Storage and Amazon S3 are supported.

🔎

S3 data must be in a bucket with public read access. For private data, use Azure Blob Storage.

ProviderURI prefixPublic/private DataRead-only AccessWrite AccessSupported Regions
Azure Blobazure://public & private(N. Virginia) us-east-1
AWS S3s3://public only(N. Virginia) us-east-1

Cloud Parameters

To interact with the cloud storage service, you need to define a module that specifies the data configuration. There are two relevant options for describing how to access the cloud:

OptionDescription
pathA string that specifies the location and name of the file you want to import/export. Currently, this can point to azure://... (Microsoft Azure) or s3://... (Amazon S3) URLs.
integrationCredentials needed to access the data.

Public Data

If the cloud storage container provides public read access — for example, this file — you only need to provide the path in the configuration option path.

module my_config
    def path = "s3://relationalai-documentation-public/csv-import/simple-import-4cols.csv"
end

For security reasons, "http://" and "https://" URL addresses are not supported.

Private Data

To access private data, you need to specify cloud credentials in integration.

Currently, only private Azure Blob Storage is supported. To access it, you must provide a valid SAS token and URL address.

The URL is provided via the path option. Within the integration submodule, the following information needs to be provided:

  • the cloud storage provider in the provider field, and
  • the access token as a string in the credentials field, along with the token name (:azure_sas_token).
module config
    def path = "azure://myaccount.blob.core.windows.net/sascontainer/myfile.csv"
 
    module integration
        def provider = "azure"
        def credentials = (:azure_sas_token, raw"sv=2014-02-14&sr=b&si=TestPolicy&sig=o%2B5%2F0C%2BLm7tWWftNKvQEGKHlSt%2Bfs8No7FZkUk5T%2Bv0%3D")
    end
end

Note that the unescaped percent sign, %, is used for string interpolation within a Rel string. This means you need to store URL and cloud credentials that contain % as raw strings. See the example above.

See Also

Now that you know how to access a service cloud provider, you can check the I/O how-to guides to learn how to interact with data. Check CSV Import and CSV Export for CSV data. If you are interested in JSON, see JSON Import/Export.

Was this doc helpful?