RelationalAI SDK for Julia

This guide presents the API for the RelationalAI SDK for Julia.

This guide presents the main features of the RelationalAI SDK for Julia, used to interact with the Relational Knowledge Graph Management System (RKGMS).

The rai-sdk-julia package is open source and is available in this Github repository:

RelationalAI/rai-sdk-julia

It includes self-contained examples of the main API functionality. Contributions and pull requests are welcome.

Note: This guide applies to rai-sdk-julia, the latest iteration of the Julia SDK. The relationalai-sdk package is deprecated.

The last section describes how the different Console notebook cell types can be implemented with this Julia SDK API.

Requirements

Check the rai-sdk-julia repository for the latest version requirements to interact with the RKGMS using the RelationalAI SDK for Julia.

Installation

The RelationalAI SDK for Julia is a standalone package. It can be installed using the Julia REPL:

using Pkg; Pkg.add(url="https://github.com/RelationalAI/rai-sdk-julia.git")

The Configuration File

To connect to the RAI Server using the Julia SDK, you need a configuration file (~/.rai/config) with RAICredentials, i.e., an ID and a secret key provided to you by an admin. To get RAICredentials, follow these steps:

The Organization/Team/Person requests an account from RAI. Once the account is set up, an admin user is assigned to the account.
This admin uses the RAI Console to create OAuth Clients. Each client is identified by a Client ID and a secret keystring.
The SDK user adds the client_id and client_secret to their ~/.rai/config file, which should look like this:

[default]
region = us-east
host = azure.relationalai.com
port = 443
client_id = ################ [from admin]
client_secret = ############## [from admin]

Note that in the place of ######### the actual user OAuth credentials should appear. This config file should be placed in the ~/.rai folder (~/.rai/config).

See the Julia SDK README for more details.

Multiple Profiles (Optional)

You can have other profiles besides [default] in separate sections of the config file. This is useful, for instance, if you have configurations for different regions.

The Julia API load_config() function takes the configuration file and the profile name as optional arguments:

using RAI: load_config
cfg = load_config(fname="~/.rai/config", profile = "default")

Permissions

Each OAuth Client has its own set of permissions, which determine what operations it can execute. Depending on the permissions, some operations (such as listing other users, or creating or deleting compute engines) may fail.

Creating a Context

Most API operations use a Context struct that contains the necessary settings for making requests against the RelationalAI REST APIs. To create a context using the default profile in your ~/.rai/config file, use:

using RAI: Context, load_config
cfg = load_config()
# to specify a non-default profile use:
# cfg = load_config(profile = "myprofile")
ctx = Context(cfg)

The remaining code examples in this document assume that you have a valid context in the ctx Julia variable and that you have brought the RAI module into the current namespace:

using RAI

You can test your configuration and context by running:

list_databases(ctx)

This should return a list with database info, assuming your keys have the corresponding permissions (see Listing Databases below).

Additionally, most of the Julia API calls throw an HTTPError exception when there is an issue. Therefore you can typically wrap the API calls discussed here in a try ... catch block similar to:

try
    list_databases(ctx)
catch e
    e isa HTTPError ? show(e) : rethrow()
end

Managing Users

A client with the right permissions can create, disable, and list the users under the account.

Creating a User

create_user(ctx, email, roles)

Here, email is a string, identifying the user, and roles is a list of roles. The roles currently supported are user and admin, with user being the default role.

Disabling a User

disable_user(ctx, user)

Listing Users

list_users(ctx)

Retrieving User Details

get_user(ctx, user)

Here, user is a string ID, e.g. "auth0|XXXXXXXXXXXXXXXXXX".

Managing OAuth Clients

OAuth clients can be managed with the following functions, provided you have the corresponding permissions:

create_oauth_client(ctx, name, permissions)

name is a string identifying the client. permissions is a list of permissions from the following supported permissions:

create:accesskey
create:compute
create:oauth_client
create:user
delete:compute
delete:database
delete:oauth_client
list:accesskey
list:compute
list:database
list:oauth_client
list:permission
list:role
list:user
read:compute
read:credits_usage
read:oauth_client
read:role
read:user
rotate:oauth_client_secret
run:transaction
update:database
update:oauth_client
update:user

Get a list of OAuth clients:

list_oauth_clients(ctx)

Get details for a specific OAuth client, identified by the string id:

get_oauth_client(ctx, id)

Delete the OAuth client identified by the string id:

delete_oauth_client(ctx, id)

Managing Engines

To query and update RAICloud databases, you will need a running engine. The following API calls create and manage them.

Creating an Engine

You can create a new engine as follows (the default size is XS):

engine = "julia_sdk_engine"
size = "XS"

rsp = create_engine(ctx, engine, size=size)
println(rsp)

API requests return a JSON value. Here is a sample result for create_engine:

{'engine': {
    'account_name': '#########',
    'created_by': '#########',
    'id': '#########',
    'name': 'julia_sdk_engine',
    'region': '#########',
    'requested_on': '2022-02-19T19:22:32.121Z',
    'size': 'XS',
    'state': 'REQUESTED'
    }
}

Valid sizes are given as a string and can be one of:

XS (extra small)
S (small)
M (medium)
L (large)
XL (extra large)

Note: It may take some time before your engine is in the “PROVISIONED” state, where it is ready for queries. It will be in the “PROVISIONING” state before that.

To list all the engines that are currently provisioned, you can use list_engines:

list_engines(ctx, state="PROVISIONED")

If there is an error with the request, an HTTPError exception will be thrown.

Most of the API examples below assume there is a running engine (in “PROVISIONED” state) in the engine variable, and a test database in database:

# replace by your values for testing:
database = "mydatabase"
engine = "myengine"

Deleting an Engine

An engine can be deleted with:

rsp = delete_engine(ctx, engine)
println(rsp)

If successful, this will return:

{'status':
    {'name': XXXX
    'state': 'DELETING',
    'message': 'engine XXXX deleted successfully'}
}

Note that since RAICloud decouples computation from storage, deleting an engine does not delete any cloud databases. See Managing Engines for more details.

Getting Info for an Engine

The details for a specific compute engine can be retrieved with get_engine:

rsp = get_engine(ctx, engine)
println(rsp)

An HTTPError exception will be thrown if the engine specified in get_engine does not exist.

Managing Databases

Creating a Database

You can create a database with create_database, as follows:

database = "mydatabase" # adjust as needed
engine = "myengine"

# Create the database, but don't overwrite it if it exists:
rsp = create_database(ctx, database, engine, overwrite=false)
println(rsp)

The result from a successful create_database call will look like this:

{
  "output": [],
  "version": 2,
  "problems": [],
  "actions": [],
  "debug_level": 0,
  "aborted": false,
  "type": "TransactionResult"
}

If you want to overwrite an existing database, you can use the overwrite = true option.

⚠

This will erase all the data in the old database.

Cloning a Database

You can also use create_database to clone a database by specifying a source argument:

# Clone the database, but not overwrite the target if it exists:
rsp = create_database(
    ctx, "mydatabase-clone", engine,
    overwrite=true, source=database
)

With this API call, you can clone a database from mydatabase to "mydatabase-clone", creating an identical copy. Any subsequent changes to either database will not affect the other. Cloning a database only succeeds if the source database already exists.

If you want to clone into an existing database, overwriting it in the process, use the overwrite=true flag.

⚠

This will erase all the data in the clone target database.

Retrieving Database Details

rsp = get_database(ctx, database)
println(rsp)

The response is a JSON object. If the database does not exist, an HTTPError exception is thrown.

Note that this call does not require a running engine.

Listing Databases

Using list_databases will list the databases available to the account:

rsp = list_databases(ctx)
# rsp = list_databases(ctx, state)
println(rsp)

The optional variable state (default: nothing) can be used to filter databases by state. For example, “CREATED”, “CREATING”, or “CREATION_FAILED”.

Deleting Databases

A database can be deleted with delete_database, if the config has the right permissions.

The database is identified by its name, as used in create_database.

rsp = delete_database(ctx, database)
println(rsp)

If successful, the response will be of the form:

{'name': 'XXXXXXX', 'message': 'deleted successfully'}

⚠

Deleting a database cannot be undone.

Rel Model Sources

Rel model sources are collections of Rel code that can be added, updated, or deleted from a particular database. Since they update a database, a running engine is required to perform operations on sources.

Installing Rel Model Sources

The load_model function installs a Rel model source in a given database. In addition to the usual context, the database and engine arguments, it takes a Julia Dictionary. This dictionary maps names to models, so that more than one named model can be installed at one time.

For example, to add a Rel model source code file to a database:

source_string = """
def countries = {"United States of America"; "Germany"; "Japan"; "Greece"}
def oceans = {"Arctic"; "Atlantic"; "Indian"; "Pacific"; "Southern"}
"""

load_model(ctx, database, engine, Dict("mysource" => source_string))

If the database already contains an installed source with the same given name, then it is replaced by the new source.

If you need to install from a file, read it into a string first. For example:

source_string = read("mymodel.rel", String)

load_model(ctx, database, engine, Dict("mysource" => source_string))

Deleting a Rel Model Source

You can delete a source from a database using the following code:

delete_model(ctx, database, engine, "mysource")

Listing Installed Rel Model Sources

You can list the sources in a database using the following code:

list_models(ctx, database, engine)

This returns a JSON array of names.

To see the contents of a named source, use:

get_model(ctx, database, engine, sourcename)

where sourcename is the name of the model.

Querying a Database

The high-level API call for running a single query/transaction against the database is exec. It specifies a Rel source (which can be empty) and a set of input relations.

function exec(
    ctx::Context,
    database::AbstractString,
    engine::AbstractString,
    source;
    inputs = nothing,
    readonly = false,
    kw...
)

For example:

rsp = exec(
    ctx,
    database,
    engine,
    "def output[x in {1;2;3}] = x * 2"
)
show_result(rsp)

Which gives the output:

// output Int64/Int64
1, 2;
2, 4;
3, 6

By convention, only the output relation is returned (as in Console notebooks), and readonly is false.

Queries meant to update EDB relations (with insert and delete) must use readonly=false.

For example, here is an API call to load some CSV data and store it in the EDB relation myedb:

data = """
name,lastname,id
John,Smith,1
Peter,Jones,2
"""

exec(
    ctx, database, engine,
    """
    def config:schema:name="string"
    def config:schema:lastname="string"
    def config:schema:id="int"
    def config:syntax:header_row=1
    def config:data = mydata

    def delete[:myedb] = myedb
    def insert[:myedb] = load_csv[config]
    """,
    inputs = Dict("mydata" => data),
    readonly=false
)

⚠

The query size is limited to 64MB. An HTTPError exception will be thrown if the request exceeds this API limit.

Getting Multiple Relations Back

As in the RAI Console, if you want to return multiple relations, you can define sub-relations of output. For example:

rsp = exec(
    ctx,
    database,
    engine,
    "def a = 1;2 def b = 3;4 def output:one = a def output:two = b"
)

show_result(rsp)

Giving the output:

// output Symbol/Int64
:two, 3;
:two, 4

// output Symbol/Int64
:one, 1;
:one, 2

Result Structure

The response is a Julia dictionary with one or more of the following keys:

field	meaning
output	A list of Julia Dict’s, each one corresponding to an output relation — in this case, only one, `output`. Future releases of the SDK may support multiple relations here.
problems	Information about any existing problems in the DB (which are not necessarily caused by the query)
actions	[can ignore]
aborted	Indicator of whether an abort resulted. For example, if an Integrity Constraint was violated.
type	[can ignore]

Each query is a complete transaction, executed in the context of the provided database.

Specifying Inputs

The exec API call takes an optional inputs dictionary that can be used to map relation names to string constants for the duration of the query, analogous to the file upload feature of Rel Console notebooks. Thus, for example,

rsp = exec(
    ctx,
    database,
    engine,
    "def output = foo",
    inputs = Dict("foo" => "asdf")
)
show_result(rsp)

will return the string "asdf" back.

Functions that transform a file and write the results to an EDB relation can be easily written in this way. The calls load_csv and load_json can actually be used in this way, via the data parameter to write results to an EDB relation. See, for example, the sample code using load_csv in Querying a Database.

Printing Responses

The show_result function prints API responses. For example:

rsp = exec(ctx, database, engine, "def output = 'a';'b';'c'")
show_result(rsp)

gives the output:

// output Char
'a';
'b';
'c'

The function show_problems prints problems.

See the examples folder for examples on how to handle HTTP exceptions.

Loading Data: `load_csv` and `load_json`

As a convenience, the Julia API includes a load_csv and load_json function. These are not strictly necessary, since the load utilities in Rel itself can be used in a non-read-only exec query that uses the inputs option. See, for example, the sample code using load_csv in Querying a Database.

The Julia function load_csv loads data and inserts the result into the EDB relation named by the relation argument. Additionally, load_csv attempts to guess the schema of the data. For more control over the schema, use a non-read-only exec query using the inputs option.

function load_csv(
    ctx::Context,
    database::AbstractString,
    engine::AbstractString,
    relation::AbstractString,
    data;
    delim = nothing,    # default: ,
    header = nothing,   # a Dict from col number to name (base 1)
    header_row = nothing,   # row number of header, nothing for no header
    escapechar = nothing,   # default: \
    quotechar = nothing,    # default: "
    kw...
)

Similarly, load_json loads the data string as JSON, and inserts it into the EDB named by the relation argument:

function load_json(
    ctx::Context,
    database::AbstractString,
    engine::AbstractString,
    relation::AbstractString,
    data;
    kw...
)

Example:

load_json(ctx, database, engine, "myjson",  """{"a" : "b"}""")

Note: In both cases, the relation EDB is not cleared, allowing for multi-part, incremental loads. To clear it, issue a non-read-only query of the form:

def delete[:relation] = relation

Listing EDB Relations

list_edbs(ctx, database, engine)

will list the EDB relations in the given database. The result is a JSON list of objects.

Notebook Cell Types

As explained in Working with RAI Notebooks there are four notebook cell types: Query, Install, Update, and Markdown. This section describes how the three cell types (Query, Install, and Update) map to Julia SDK API calls.

Query Cells

The Query cell in a Rel Console notebook corresponds to a simple exec() Julia function call, as shown below:

rsp = exec(ctx, database, engine, "def output = {(1,); (2,); (3,)}")
show_result(rsp)

Giving the output:

// output Int64
1;
2;
3

The transaction did not produce any problems:

show_problems(rsp)

Note that no rules/definitions are persisted in this case.

Install Cells

To persist IDB definitions, use the load_model API call. That is, the Install cell in a Rel Console notebook corresponds to the load_model Julia function call, as shown below:

rsp = load_model(
    ctx, database, engine,
    Dict("arity1_k_def" => "def k = {2;3;50}")
)
show_problems(rsp)

Check that the above relation was persisted using the exec API call:

rsp = exec(ctx, database, engine, "def output = k")
show_result(rsp)
// output Int64
2;
3;
5

Update Cells

The Update cell in a Rel Console notebook corresponds to an exec Julia function call with the flag readonly=false. Note that this is the default value for this flag. For example:

rsp = exec(
    ctx, database, engine,
    """def insert:employee = {(1, \"Han Solo\"); (2, \"Bart Simpson\")}""",
    readonly=false
)

You can modify the employee EDB:

rsp = exec(
    ctx, database, engine,
    """def insert:employee = {(3, \"King\"); (4, \"Queen\")}""",
    readonly=false
)

You can now query the modified employee EDB:

rsp = exec(ctx, database, engine, "def output=employee")
show_result(rsp)
// output Int64/String
// output Int64/String
1, 'Han Solo';
2, 'Bart Simpson';
3, 'King';
4, 'Queen'

To update EDB relations, always use the readonly=false option in the exec call.

RelationalAI SDK for Julia

Requirements#

Installation#

The Configuration File#

Multiple Profiles (Optional)#

Permissions#

Creating a Context#

Managing Users#

Creating a User#

Disabling a User#

Listing Users#

Retrieving User Details#

Managing OAuth Clients#

Managing Engines#

Creating an Engine#

Deleting an Engine#

Getting Info for an Engine#

Managing Databases#

Creating a Database#

Cloning a Database#

Retrieving Database Details#

Listing Databases#

Deleting Databases#

Rel Model Sources#

Installing Rel Model Sources#

Deleting a Rel Model Source#

Listing Installed Rel Model Sources#

Querying a Database#

Getting Multiple Relations Back#

Result Structure#

Specifying Inputs#

Printing Responses#

Loading Data: load_csv and load_json#

Listing EDB Relations#

Notebook Cell Types#

Query Cells#

Install Cells#

Update Cells#