RelationalAI SDK for Julia

This guide presents the main features of the RelationalAI SDK for Julia, which can be used to interact with RelationalAI’s Relational Knowledge Graph System (RKGS).

The rai-sdk-julia package is open source and is available in this GitHub repository:

RelationalAI/rai-sdk-julia

It includes self-contained examples (opens in a new tab) of the main API functionality. Contributions and pull requests are welcome.

Note: This guide applies to rai-sdk-julia, the latest iteration of the RelationalAI SDK for Julia. The relationalai-sdk package is deprecated.

Requirements

You can check the rai-sdk-julia (opens in a new tab) repository for the latest version requirements to interact with the RKGS using the RelationalAI SDK for Julia.

Installation

The RelationalAI SDK for Julia is a stand-alone package. It can be installed using the Julia REPL:

using Pkg; Pkg.add("RAI")

Configuration

The RelationalAI SDK for Julia can access your RAI Server credentials using a configuration file. See SDK Configuration for more details.

The Julia API load_config() function takes the configuration file and the profile name as optional arguments:

using RAI: load_config
cfg = load_config(fname="~/.rai/config", profile = "default")

To load a different configuration, you can replace "default" with a different profile name.

Creating a Context

Most API operations use a context struct that contains the necessary settings for making requests against the RelationalAI REST APIs. To create a context using the default profile in your ~/.rai/config file, you can use:

using RAI: Context, load_config
cfg = load_config()
# to specify a non-default profile use:
# cfg = load_config(profile = "myprofile")
ctx = Context(cfg)

The remaining code examples in this document assume that you have a valid context in the ctx Julia variable and that you have brought the RAI module into the current namespace:

using RAI

You can test your configuration and context by running:

list_databases(ctx)

This should return a list with database info, assuming your keys have the corresponding permissions. See Listing Databases below.

Additionally, most of the Julia API calls throw an HTTPError exception when there is an issue. Therefore you can typically wrap the API calls discussed here in a try ... catch block similar to:

try
    list_databases(ctx)
catch e
    e isa HTTPError ? show(e) : rethrow()
end

You can find the full test example here (opens in a new tab).

Managing Users

A client with the right permissions can create, disable, and list the users under the account.

Creating a User

You can create a user as follows:

create_user(ctx, email, roles)

Here, email is a string, identifying the user, and roles is a list of roles. The roles currently supported are user and admin, with user being the default role.

Deleting a User

You can delete a user through:

delete_user(ctx, id)

In this case, id is a string representing a given user’s ID.

Disabling and Enabling a User

You can disable a user through:

disable_user(ctx, id)

Again, id is a string representing a given user’s ID. You can reenable the user as follows:

enable_user(ctx, id)

Listing Users

list_users(ctx)

Retrieving User Details

get_user(ctx, user)

Here, user is a string ID, for example, "auth0|XXXXXXXXXXXXXXXXXX".

Finding Users Using Email

You can look up a user’s details by specifying their email:

find_user(ctx, email)

In this case, email is a string.

Managing OAuth Clients

OAuth clients can be managed with the following functions, provided you have the corresponding permissions:

create_oauth_client(ctx, name, permissions)

name is a string identifying the client. permissions is a list of permissions from the following supported permissions:

• create:accesskey
• create:compute
• create:oauth_client
• create:user
• delete:compute
• delete:database
• delete:oauth_client
• list:accesskey
• list:compute
• list:database
• list:oauth_client
• list:permission
• list:role
• list:user
• read:compute
• read:credits_usage
• read:oauth_client
• read:role
• read:user
• rotate:oauth_client_secret
• run:transaction
• update:database
• update:oauth_client
• update:user

This is how to get a list of OAuth clients:

list_oauth_clients(ctx)

This is how to get details for a specific OAuth client, identified by the string id:

get_oauth_client(ctx, id)

Here’s how to delete the OAuth client identified by the string id:

delete_oauth_client(ctx, id)

Managing Engines

To query and update RelationalAI databases, you will need a running engine. The following API calls create and manage them.

Creating an Engine

You can create a new engine as follows. Note that the default size is XS:

engine = "julia_sdk_engine"
size = "XS"
 
rsp = create_engine(ctx, engine, size=size)
println(rsp)

API requests return a JSON value. Here is a sample result for create_engine:

{'engine': {
    'account_name': '#########',
    'created_by': '#########',
    'id': '#########',
    'name': 'julia_sdk_engine',
    'region': '#########',
    'requested_on': '2022-02-19T19:22:32.121Z',
    'size': 'XS',
    'state': 'REQUESTED'
    }
}

Valid sizes are given as a string and can be one of:

• XS (extra small).
• S (small).
• M (medium).
• L (large).
• XL (extra large).

Note: It may take some time before your engine is in the “PROVISIONED” state, where it is ready for queries. It will be in the “PROVISIONING” state before that.

To list all the engines that are currently provisioned, you can use list_engines:

list_engines(ctx, state="PROVISIONED")

If there is an error with the request, an HTTPError exception will be thrown.

Most of the API examples below assume there is a running engine (in “PROVISIONED” state) in the engine variable, and a test database in database:

# replace by your values for testing:
database = "mydatabase"
engine = "myengine"

Deleting an Engine

You can delete an engine with:

rsp = delete_engine(ctx, engine)
println(rsp)

If successful, this will return:

{'status':
    {'name': XXXX
    'state': 'DELETING',
    'message': 'engine XXXX deleted successfully'}
}

Note that since RelationalAI decouples computation from storage, deleting an engine does not delete any cloud databases. See Managing Engines for more details.

Getting Info for an Engine

You can retrieve the details for a specific compute engine with get_engine:

rsp = get_engine(ctx, engine)
println(rsp)

An HTTPError exception will be thrown if the engine specified in get_engine does not exist.

Managing Databases

Creating a Database

You can create a database with create_database, as follows:

database = "mydatabase" # adjust as needed
 
rsp = create_database(ctx, database)
println(rsp)

The result from a successful create_database call will look like this:

{
  "output": [],
  "version": 2,
  "problems": [],
  "actions": [],
  "debug_level": 0,
  "aborted": false,
  "type": "TransactionResult"
}

Cloning a Database

You can also use create_database to clone a database by specifying a source argument:

rsp = create_database(
    ctx, "mydatabase-clone",
    source=database
)

With this API call, you can clone a database from mydatabase to "mydatabase-clone", creating an identical copy. Any subsequent changes to either database will not affect the other. Cloning a database fails if the source database does not exist.

Retrieving Database Details

rsp = get_database(ctx, database)
println(rsp)

The response is a JSON object. If the database does not exist, an HTTPError exception is thrown.

Note that this call does not require a running engine.

Listing Databases

Using list_databases will list the databases available to the account:

rsp = list_databases(ctx)
# rsp = list_databases(ctx, state)
println(rsp)

The optional variable state (default: nothing) can be used to filter databases by state; for example, “CREATED”, “CREATING”, or “CREATION_FAILED”.

Deleting Databases

You can delete a database with delete_database, if the config has the right permissions.

The database is identified by its name, as used in create_database:

rsp = delete_database(ctx, database)
println(rsp)

If successful, the response will be of the form:

{'name': 'XXXXXXX', 'message': 'deleted successfully'}

Rel Models

Rel models are collections of Rel code that can be added, updated, or deleted from a dedicated database. A running engine — and a database — is required to perform operations on models.

Loading a Rel Model

The load_models function loads a Rel model source in a given database. In addition to the usual context, the database and engine arguments, it takes a Julia dictionary. This dictionary maps names to models, so that more than one named model can be loaded at one time.

For example, this is how to add a Rel model code file to a database:

model_string = """def countries = {"United States of America"; "Germany"; "Japan"; "Greece"}
def oceans = {"Arctic"; "Atlantic"; "Indian"; "Pacific"; "Southern"}"""
 
load_models(ctx, database, engine, Dict("mymodel" => model_string))

If the database already contains an installed model with the same given name, then it is replaced by the new one.

If you need to load from a file, you can read it into a string first. For example:

model_string = read("mymodel.rel", String)
 
load_models(ctx, database, engine, Dict("mymodel" => model_string))

Deleting a Rel Model

You can delete installed models from a database as follows:

delete_models(ctx, database, engine, modelname)

Note that modelname is a string vector contaning the name of the model or models to be deleted.

Listing Installed Rel Models

You can list the installed models within a database as follows:

list_models(ctx, database, engine)

This returns a JSON array of names.

To see the contents of a named model, you can use:

get_model(ctx, database, engine, modelname)

Note that the argument modelname is a string referring to the model name.

Querying a Database

The high-level API call for executing queries against the database is exec. The function call blocks until the transaction is completed or there are several timeouts indicating that the system may be inaccessible. It specifies a Rel source, which can be empty, and a set of input relations:

function exec(
    ctx::Context,
    database::AbstractString,
    engine::AbstractString,
    source;
    inputs = nothing,
    readonly = false,
    kw...
)

Here is an example of a read query using exec:

rsp = exec(
    ctx,
    database,
    engine,
    "def output = {1; 2; 3}"
)
show_result(rsp)

By default, readonly is false.

Write queries, which update base relations through the control relations insert and delete, must use readonly=false.

Here is an API call to load some CSV data and store them in the base relation my_base_relation:

data = """
name,lastname,id
John,Smith,1
Peter,Jones,2
"""
 
exec(
    ctx, database, engine,
    """
    def config:schema:name="string"
    def config:schema:lastname="string"
    def config:schema:id="int"
    def config:syntax:header_row=1
    def config:data = mydata
 
    def delete[:my_base_relation] = my_base_relation
    def insert[:my_base_relation] = load_csv[config]
    """,
    inputs = Dict("mydata" => data),
    readonly=false
)

The RelationalAI SDK for Julia also supports asynchronous transactions, through exec_async. In summary, when you issue a query to the database, the return output contains a transaction ID that can subsequently be used to retrieve the actual query results.

exec_async is defined as exec, but in this case the running processes are not blocked:

rsp_async= exec_async(
    ctx,
    database, engine,
    "def output = {1; 2; 3}"
)

If needed, you can block the running process until the transaction has reached a terminal state, i.e., "COMPLETED" or "ABORTED", through wait_until_done:

wait_until_done(ctx, rsp_async)

For instance, this can be useful for canceling an ongoing transaction:

try
    wait_until_done(ctx, rsp_async)
catch
    cancel_transaction(txn)
end

Finally, you can fetch the results:

if rsp_async.transaction["state"] == "COMPLETED"
    results = get_transaction_results(ctx, rsp_async.transaction["id"])
    println(results)
end

Similarly to get_transaction_results, you can also get metadata and problems for a given transaction ID:

metadata = get_transaction_metadata(ctx, rsp_async.transaction["id"])
problems = get_transaction_problems(ctx, rsp_async.transaction["id"])

Getting Multiple Relations Back

In order to return multiple relations, you can define subrelations of output. For example:

rsp = exec(
    ctx,
    database,
    engine,
    "def a = 1;2 def b = 3;4 def output:one = a def output:two = b"
)
 
show_result(rsp)

This gives the following output:

/:output/:two/Int64
 (3,)
 (4,)

/:output/:one/Int64
 (1,)
 (2,)

Result Structure

The response is a Julia dictionary with the following keys:

The results key is a vector with the following fields:

Each query is a complete transaction, executed in the context of the provided database.

The metadata key is a JSON string with the following fields:

Finally, the problems key is also a JSON string with the following fields:

Specifying Inputs

The exec API call takes an optional inputs dictionary that can be used to map relation names to string constants for the duration of the query. Here’s an example:

rsp = exec(
    ctx,
    database,
    engine,
    "def output = foo",
    inputs = Dict("foo" => "asdf")
)
show_result(rsp)

This will return the string "asdf" back.

Functions that transform a file and write the results to a base relation can be written in this way. The calls load_csv and load_json can actually be used in this way, via the data parameter to write results to a base relation. See, for example, the sample code using load_csv in Querying a Database.

Printing Responses

The show_result function prints API responses. See previous examples.

Loading Data: load_csv and load_json

As a convenience, the Julia API includes load_csv and load_json functions. These are not strictly necessary, since the load utilities in Rel itself can be used in a non-read-only exec query that uses the inputs option. See, for example, the sample code using load_csv in Querying a Database.

The Julia function load_csv loads data and inserts the result into the base relation named by the relation argument. Additionally, load_csv attempts to guess the schema of the data. For more control over the schema, use a non-read-only exec query using the inputs option:

function load_csv(
    ctx::Context,
    database::AbstractString,
    engine::AbstractString,
    relation::AbstractString,
    data;
    delim = nothing,    # default: ,
    header = nothing,   # a Dict from col number to name (base 1)
    header_row = nothing,   # row number of header, nothing for no header
    escapechar = nothing,   # default: \
    quotechar = nothing,    # default: "
    kw...
)

Similarly, load_json loads the data string as JSON and inserts it into the base relation named by the relation argument:

function load_json(
    ctx::Context,
    database::AbstractString,
    engine::AbstractString,
    relation::AbstractString,
    data;
    kw...
)

Example:

load_json(ctx, database, engine, "myjson",  """{"a" : "b"}""")

Note: In both cases, the relation base relation is not cleared, allowing for multipart, incremental loads. To clear it, you can do:

def delete[:relation] = relation

Listing Base Relations

list_edbs(ctx, database, engine)

This will list the base relations in the given database. The result is a JSON list of objects.

Transaction Cancellation

You can cancel an ongoing transaction by calling the following function:

rsp = cancel_transaction(ctx, id)
println(rsp)

The argument id is a string that represents the transaction ID. An example is rsp_async.transaction["id"] from a previous exec_async API call.