Data I/O: Accessing the Cloud
This guide demonstrates how to interact with data using the supported cloud providers.
By following this guide, you will be able to access data from the cloud in the RKGMS database and vice versa using the different available options.
S3 data must be in a bucket with public read access. For private data, use Azure Blob Storage.
|Provider||URI prefix||Public/private Data||Read-only Access||Write Access||Supported Regions|
|Azure Blob||azure://||public & private||✅||✅||(N. Virginia) us-east-1|
|AWS S3||s3://||public only||✅||❌||(N. Virginia) us-east-1|
To interact with the cloud storage service, you need to define a module that specifies the data configuration. There are two relevant options for describing how to access the cloud:
|A string that specifies the location and name of the file you want to import/export. Currently, this can point to |
|Credentials needed to access the data.|
If the cloud storage container provides public read access — for example, this file — you only need to provide the path in the configuration option
module my_config def path = "s3://relationalai-documentation-public/csv-import/simple-import-4cols.csv" end
For security reasons,
"https://" URL addresses are not supported.
To access private data, you need to specify cloud credentials in
Currently, only private Azure Blob Storage is supported. To access it, you must provide a valid SAS token and URL address.
The URL is provided via the
integration submodule, the following information needs to be provided:
- the cloud storage provider in the
- the access token as a string in the
credentialsfield, along with the token name (
module config def path = "azure://myaccount.blob.core.windows.net/sascontainer/myfile.csv" module integration def provider = "azure" def credentials = (:azure_sas_token, raw"sv=2014-02-14&sr=b&si=TestPolicy&sig=o%2B5%2F0C%2BLm7tWWftNKvQEGKHlSt%2Bfs8No7FZkUk5T%2Bv0%3D") end end
Now that you know how to access a service cloud provider, you can check the I/O how-to guides to learn how to interact with data. Check CSV Import and CSV Export for CSV data. If you are interested in JSON, see JSON Import/Export.