Supported Data Sources

This guide describes the supported data sources for importing data into the Relational Knowledge Graph System (RKGS).

Goal

By following this guide, you’ll learn which data sources are supported by the system, allowing you to import data into the RKGS.

This guide complements the other Data Import and Export guides.

Supported Data Sources

These are the different sources you can use to import data into the RKGS:

The Configuration Module

When importing data from the cloud or strings, loading is controlled via a configuration module. This module is passed as an argument to the loading relations:

module config
    // ...
end
 
def my_json = load_json[config]

See Data Types for more details on supported data types and the available loading relations.

The following configuration options are common to all Rel data-loading relations and control where the data are loaded from.

The option path is a string that specifies the location and name of the file you want to import. It can be combined with integration for specifying credentials when accessing private data in cloud storage. See Cloud for more details.

The option data is a string specifying the actual data to be imported. You use this to load data from strings.

Strings

You can import data as a formatted String in Rel source code. This option is convenient for experimental examples and small datasets.

Here’s an example of loading some JSON data:

// read query
 
module config
    def data = 
    """
      {"name": "John", "phone": [{"no.":"1234"}, {"no.":"4567"}]}
    """
end
 
def output = load_json[config]

Local Files

You can load files directly from your computer using:

Here’s an example of loading a CSV file using the Python SDK:

file_name = "my_data_file.csv"
with open(file_name) as fp:
    data = fp.read()
 
api.load_csv(ctx, database, engine, "my_csv", data)

Cloud

You can load data from several cloud providers. The system supports importing both public and private data. Importing private data requires setting up certain credentials within the loading configuration module.

Here’s an example of loading an Iceberg table from AWS S3:

module config
    def path = "s3://my-s3-bucket/path/to/table_iceberg"
end
 
def my_iceberg = load_iceberg[config]

See Accesing the Cloud for more details on supported providers, regions, and examples.

Snowflake

The RAI Integration Services are a collection of features designed to enable RAI’s advanced modeling and querying capabilities to analyze your Snowflake data in new ways, including graph analytics.

A RAI data stream synchronizes your data from a Snowflake database to a RAI database. You can think of a RAI data stream as a materialized view that connects your Snowflake data with RAI. To use it, you need access to a RAI integration.

Here’s an example that creates a RAI data stream between the Snowflake table product and the RAI relation rai_product:

CREATE OR REPLACE TABLE product (ID INT, Name STRING, Category STRING) 
    AS SELECT * FROM VALUES
    (11, 'Laptop', 'Electronics'),
    (27, 'Shirt', 'Clothing');
 
CALL RAI.create_data_stream('my_sf_db.my_sf_schema.product', 'my_rai_db', 'rai_product');

See RAI Data Streams for more details.

See Also

Now that you know the several data sources you can use to import data into RAI, check out the other Data Import and Export guides to learn how to interact with your data.