Rel Primer: Advanced Syntax

This Primer introduces more advanced Rel syntax.

Introduction

This document assumes that you are familiar with basic Rel syntax, and introduces more advanced features of the Rel language.

Grounded and Ungrounded Variables

You can begin by considering what relations Rel can be expected to compute.

A variable is grounded when it can be instantiated to a specific, finite set of values. For example, in the expression x: range(1, 100, 1, x), using the built-in range relation, the variable x is instantiated to the 100 different values 1, 2, 3, ..., 100 to produce the relation. Similarly, when you write x : minimum(1, 2, x), the variable x is bound to 1.

By contrast, if you write x: minimum(1, x, 1), there are infinitely many values of x for which minimum(1, x, 1) is true; and if you write x, y: range(1, y, 1, x), there are infinitely many possible values for both x and y. These variables are said to be ungrounded.

When the RAI query engine encounters one of these cases, you will see an error stating that the definition “contains ungrounded variable(s); hence, the rule cannot be evaluated.”

In general, if you write range(x, y, z, result), you should expect to know the values of x, y, z, and say that those values grounded the value of result.

@inline Definitions

You can still declare and use definitions that are not expected to be fully computed, by using the @inline annotation. For example:

// read query
 
@inline
def mymax[x, y] = maximum[abs[x], abs[y]]
def output = mymax[-10, -20]

Here, mymax is not intended to be computed directly — you would not expect to enumerate all the tuples that satisfy it. If it were not labeled as @inline, the Rel compiler would attempt to compute mymax and report that x and y are ungrounded.

You can add the @inline annotation to any relation. If you have a large derived relation but are only interested in querying it for particular values, it can be best to inline it, so that the system does not compute all values in that relation. This is also the case if the rest of your code only needs a few values.

Relations as Arguments

Relations in Rel are first-order, meaning they do not take relations as values. But you can use @inline to define syntactically higher-order relations, which can take a relation as one of their arguments.

For example:

// read query
 
@inline
def maxmin[Relation] = (max[Relation], min[Relation])
def output = maxmin[ {1; 2; 3; 4; 5} ]
 

Here you used max and min from the Standard Library, which take relations as their argument as well.

This does not contradict the first-order nature of the language: When the @inline definition is expanded, the result is first-order again. Readers with experience with programming languages can think of inline definitions as macros that are expanded at compile-time. They are not directly evaluated themselves, keeping things first-order.

argmax is another Standard Library utility that takes a relation R as an argument, returning the rows in R that have the largest value. For example:

// read query
 
def output = argmax[{("a", 2, 3); ("b", 1, 6) ; ("c", 10, 6)}]
 

You can use argmax on the sample data from Aggregations to find the highest-paid players for a particular team. Here are the plays_for and salary relations:

plays_for

salary

// read query
 
def output = argmax[name, s : plays_for(name, "BFC") and salary(name, s)]

It is easy to generalize this query to get the highest-paid player for each team that participates in the plays_for relation:

// read query
 
def output = team: argmax[name, s : plays_for(name, team) and salary(name, s)]

Example

Here’s an example of how @inline functions can give the code a higher-order flavor. The x... syntax is explained in the Varargs section below.

// read query
 
@inline
def plusone[R][x...] = R[x...] + 1
def foo = (1, 3); (4, 5); (5, 6, 10)
def bar = plusone[foo]
def output = bar[4]

Multiple Arities

Mathematically speaking, an individual relation has a fixed arity. However, Rel lets you define relations with different arities but the same name. This is sometimes known as overloading, where you can treat a union of relations as a single relation. For example:

// read query
 
def foo = 1, 2
def foo = 1, 4, 6
def output = foo[1]

foo

output

The built-in relation arity can be used to check the arity, or arities, of a relation R. For example:

// read query
 
def output = arity[{6 ; (7, 8) }]
 

Thus, if count[arity[R]] = 1, then R is not overloaded by arity.

Varargs

Sometimes you may want to access elements in the tuples of a relation whose arity is not fixed, or simply unknown. To facilitate this, Rel features varargs. A vararg is a variable name followed by three dots: x.... It represents a sequence of variables. The sequence is of unknown length, possibly empty. When used as an argument in a relational application, a vararg matches zero or more elements in a tuple.

For example:

// read query
 
def mixed = {(1); (2, 3); (4, 5, 6)}
 
def output(x...) = mixed(x..., _)

The expression mixed(x..., _) instantiates x... to all but the last element of each tuple in the relation mixed.

You can create a relation that contains only the final elements of the tuples in mixed, or one that contains only the first elements of all tuples:

// read query
 
def mixed = {(1); (2, 3); (4, 5, 6)}
 
def output(y) = mixed(x..., y) from x...

// read query
 
def mixed = {(1); (2, 3); (4, 5, 6)}
 
def output(x) = mixed(x, y...) from y...

In general, it’s not a good idea to use varargs in a partial relational application. For example, the following returns no result:

// read query
 
def mixed = {(1); (2, 3); (4, 5, 6)}
 
def output[x...] = mixed[x...]

The system cannot know whether you mean x... to represent one, two or three variables. Each of these choices gives a different result:

// read query
 
def mixed = {(1); (2, 3); (4, 5, 6)}
 
def output[x] = mixed[x]

// read query
 
def mixed = {(1); (2, 3); (4, 5, 6)}
 
def output[x, y] = mixed[x, y]

// read query
 
def mixed = {(1); (2, 3); (4, 5, 6)}
 
def output[x, y, z] = mixed[x, y, z]

In some circumstances, a vararg in a partial application is unambiguous:

// read query
 
def mixed = {(1); (2, 3); (4, 5, 6)}
 
def output(x...) = mixed[x...]

Since there are parentheses on the left-hand side, the compiler can deduce that the relation on the right-hand side must have arity zero. Therefore x... must always match the entire tuple in mixed.

If you use a vararg in a partial application of a higher-order relation, you must enclose it in extra parentheses. For example, you may want to find the various lengths of the tuples in your relation by writing the following:

// read query
 
def mixed = {(1); (2, 3); (4, 5, 6)}
 
def output = arity[(x...)], mixed(x...) from x...

If you just write arity[y...], the compiler reports errors. Similarly, arity[(1, 2, 3)] evaluates to 3, but arity[1, 2, 3] is an incorrect partial application, because arity expects only one argument.

The example above illustrates a point. There is a simpler — and more idiomatic — way to find the arities of your relation:

// read query
 
def mixed = {(1); (2, 3); (4, 5, 6)}
 
def output = arity[mixed]

If you use more than one vararg in a relational application, there may be more than one way to match the sequence of elements in each tuple. In that case, the system will attempt all these ways, which may sometimes lead to an unexpected proliferation of computed tuples. This is illustrated by the following example:

// read query
 
def R = (0, 1, 0, 2, 0)
 
def output(x..., 999, y...) = R(x..., 0, y...)

Varargs are particularly useful for writing higher-order definitions, when you do not know in advance the arity of a relation passed as an argument.

For example, the Standard Library defines first and last as follows:

@inline
def first[R](x) = ∃(y... : R(x, y...))
 
@inline
def last[R](y) = ∃(x... : R(x..., y))

These could also be defined as:

@inline
def first[R](x) = R(x, y...) from y...
@inline
def last[R](y) = R(x..., y) from x...

You can use these library relations like this:

// read query
 
def mixed = {(1); (2, 3); (4, 5, 6)}
 
def output = first[mixed]

In Basic Syntax you saw how R[S] only works if S is a unary relation. Varargs provide you with a more general mechanism. For example, given some relation R you can extract the remainder of each tuple whose prefix matches a tuple in some relation Prefix:

// read query
 
@inline
def prefix_restrict[Prefix, R] = R[x...] from x... in Prefix
 
def foo = (1, 2, 3); (4, 5, 6); (7, 8, 9)
def r = {(1, 2); (4, 5)}
def output = prefix_restrict[r, foo]

The higher-order relation prefix_restrict is closely related to the Standard Library’s prefix_join, or <:, discussed below.

When using varargs, keep in mind that the compiler must be able to compute the arity of any requested relation. For a relation that has no fixed arity, the number of arities must be finite. For example, the following query succeeds, and foo(1) is true:

// read query
 
@inline def foo(vs..., 1) = true
def output:yes = foo(1)

In the definition of output the relation foo has arity 1, so vs... represents an empty sequence of variables.

However, the following query fails; while foo(1) constrains foo to have exactly arity 1, foo[1] only constrains it to be nonzero, so foo could have any one of an infinite number of arities:

@inline def foo(vs..., 1) = true
def output:yes = foo[1]

Note: A definition such as def output:yes = foo(1) is a concise form of def output[:yes] = foo(1). This copies the value of foo(1) into the relation output, extending its tuples with a new first element: the Symbol :yes. In this example the technique is used to make it more explicit that foo(1) evaluates to true, that is, {()}. More generally, it is very useful for distinguishing between the results of different queries in the same “run.” See, for instance, the examples in Relational Equality.

A Word About Arities

In general, using relations with small arities and working with normalized data is preferred. This keeps the arities small and helps with performance, readability, and correctness. Therefore, while varargs are very useful when writing general utilities — see the Standard Library for examples — the arity of individual relations should be as small as possible but, for best performance, the relations should be normalized in Graph Normal Form. Note how even wide CSV tables are ingested as a set of ternary relations, for example.

Multiple Types

Relations can also be overloaded by type, meaning that they can contain tuples with different types. You can think of each combination of types as a separate relation. A Rel expression can also refer to a combination of different arities and types. For example:

// read query
 
def myrel = 1; 2.0
def myrel = ("a", "b") ; (3, 4)
def output = myrel

Specialization: Symbols

Related to overloading by type, relations can be specialized, which separates them into different relations depending on the particular values they take.

This is always the case for meta values, which are values starting with a colon (:), such as :a and :b below. For example:

// read query
 
def foo[:a] = 1
def foo[:b] = 2
def output = foo

When using the RAI Console Editor in your browser, you will notice that the “physical” output view shows the specialized relations for :a and for :b separately.

Imported CSV relations are specialized by the column name, which is a symbol, so each column becomes a separate relation.

Symbols and Base Relation Updates

You can sometimes use the meta value :relname to refer to the relation relname, particularly when updating base relations, using insert[:relname]=... and delete[:relname]=.... See Working With Base Relations.

Symbols and Modules

The correspondence between meta values and specialized relations also shows up in modules, where module:foo is the same as module[:foo]. See Modules for more details.

Recursion

Rel supports recursive definitions, provided the recursion is well-founded (has a base case) and the relations computed are finite.

// read query
 
def fib[0] = 0
def fib[1] = 1
def fib[x in range[2,10,1]] = fib[x-1] + fib[x-2]
def output = fib

See Recursion for more details.

Combining Multiple Rules

As mentioned in Basic Syntax, the definitions for any given relation are combined.

This is the case even if the definitions are in separate installed models or libraries. It can be useful to extend existing relations for new types or arities. This applies even to recursive definitions, where you can add new base cases or new recursive definitions to an existing one.

// model
 
def rec[0] = 10
def rec[x] = rec[x-1] + 5, range(1, 4, 1, x)

The following definition adds a new base case:

// read query
 
def rec[3] = 200
def output = rec

The order of the rules does not matter — and they can be separated by other rules, or even be installed separately.

Another interesting feature of this example is that while the first two rules for rec were installed, the extra base case was part of a query, which enables it only for the effects of that query itself. This means that if you ask for rec again, you get only the values from the rules that were originally installed:

// read query
 
def output = rec

Point-Free Syntax

Rel’s syntax supports point-free notation, where you can omit argument variables when their omission does not lead to ambiguity. Consider the following example:

def mydomain(x) = range(1, 7, 1, x)

You can write it instead like this:

def mydomain = range[1, 7, 1]

Again, conjunctions correspond to , and disjunctions to ;. For example:

def myrel(x, y) = r1(x) and r2(y)

This can be expressed point-free as:

def myrel = r1, r2

In both cases, you get the cross product of r1 and r2.

To see how ; corresponds to or:

def myrel(x, y) = r1(x, y) or r2(x, y)

This is equivalent to:

def myrel = r1 ; r2

As another example, consider this:

// read query
 
def a = {(1, 2)}
def b = {(1, 2) ; (3, 4)}
def myrel(x, y) = a(x, y) and b(x, y)
def output = myrel

You can use intersect from the Standard Library and instead write it like this:

def myrel = intersect[a, b]

You can even use point-free recursive definitions, such as this one, for the transitive closure of the binary relation r — see below for an explanation of the composition operator .:

// read query
 
def r = {(1, 2); (2, 3); (3, 4); (2, 5)}
def tcr = r
def tcr = tcr.r // See section below for the meaning of `.`.
def output = tcr

Note that writing def myrel = a and b will give an arity error if a and b do not have arity 0, which and expects. This is also the case for or, not, and implies.

Relational Equality

In Rel, = means equality between individual, or scalar, values. That is, when = is used in a formula; the ”=” used in def myrel = ... has a special status as a reserved meta value. For example, 3 = 2 + 1 is true, and writing x = y supposes that x and y are individual values.

To test equality between relations, use equal:

// read query
 
def output = if equal({1; 2; 3} , {3; 2; 1; 2}) then "yes" else "no" end

Since Rel distributes scalar operations over relation values, using = for relations can give confusing results. For example:

// read query
 
// do not use `=` to check equality of relations!
def output:wrong = if {1} = {1; 2; 3} then "equal" else "not equal" end
// use `equals` for relations:
def output:right = if equal({1}, {1; 2; 3}) then "equal" else "not equal" end

Technical footnote: If R1 and R2 are relations, then R1 = R2 if and only if their intersection is nonempty.

Useful Relational Operators

The Rel Standard Library defines many useful relational operations.

This section describes some of the most commonly used relational operations.

Composition

Relational composition, indicated by a dot (.), is a shortcut for joining the last element of a relation with the first element of another — usually a foreign key:

// read query
 
def order_products = {(12, 3213) ; (10, 3213) ; (7, 9832)}
def product_names = {(3213, "laptop"); (9832, "iphone"); (45353, "TV")}
def output = order_products.product_names

As another example, if parent(x, y) holds when x is a parent of y, then x : x.parent.parent is the “grandparent” relation:

// read query
 
def parent = {("bill", "alice") ; ("alice" , "bob") ;
              ("alice" , "mary") ; ("jane", "john")}
def output = "bill".parent.parent

Prefix and Suffix Joins

Prefix join, written as prefix_join[R, S] or R <: S, generalizes [] to get the elements of a relation S that have a prefix in R. Using varargs notation, R <: S contains the tuples (x..., y...) in S where (x...) is in R. For example:

// read query
 
def r = {(1, 2); (2, 5)}
def s = {(1, 2, 3); (1, 5, 7); (1, 2, 8); (2, 5, 9)}
def output = r <: s

// read query
 
def json = parse_json["""{"a": {"b": 1, "c": 2}, "d": 3}"""]
def output = :a <: json

Suffix join, written as suffix_join[S, R] or S :> R is similar, but matches suffixes. R :> S contains the tuples (x..., y...) in R where (y...) is in S:

// read query
 
def r = {(1, 2, 3); (1, 5, 7); (1, 2, 8); (2, 5, 9)}
def s = {(2, 3); (5, 9)}
def output = r :> s

// read query
 
@inline def even(x) = (x%2=0)
def json = parse_json["""[ {"a": 1, "b": 2}, {"a": 3, "b": 4, "c": 6} ]"""]
def output = json :> even

Here, the prefix ([], n) indicates a JSON list, with n as the index. For details on how JSON is represented in Rel, see the JSON Import guide.

Left and Right Override

The left override operator is usually applied to relations of tuples (k..., v) with a functional dependency from the initial (key) arguments k... to the last argument v (the value). It lets you merge two such relations, giving precedence to one over the other.

left_override[R, S], also written as R <++ S, contains all the tuples (x..., v) of R, plus all the tuples (x1..., v1) of S where x1... is not in R. Often, S specifies default values for keys that are missing in R.

As a mnemonic, when you read R <++ S you can imagine S injecting new values into R, but only when those keys are missing — so R is overriding S.

// read query
 
def base  = ("a", 2) ; ("b", 4)
def defaultvalues = ("a", 10) ; ("c", 20)
def combined = (base <++ defaultvalues)
def output = combined

A common use case for override is making explicit default values, using a constant relation for the default. You can write count[R] <++ 0 if you want the count of an empty relation to be 0 rather than empty. As another example, a counter can be defined using a base relation as follows:

// write query
 
def delete[:counter] = counter
def insert[:counter] = (counter + 1) <++ 0

Note that without delete, the consecutive values will accumulate in counter, rather than replace each other. See Working With Base Relations for more details.

Here is an example of how you can add defaults for a domain, with a definition using in:

// read query
 
def base = ("a", 10)
def mydomain = ("a" ; "b" ; "c")
def filled[x in mydomain] = base[x] <++ 0
def output = filled

Rel also provides ++> (right_override, which swaps the order of the arguments.

As a special case, count[R] <++ 0 gives us a count of zero for empty sets. See Aggregating Over Empty Relations.

Types

Rel provides unary relations for testing the type of primitive values. These tests include String, Number, Int, and Float, which take one argument, and counterparts for fixed-width types, such as Floating. Example: Floating[32, float[32, 3.0]] is true.

See Rel Data Types for details on all of the primitive types.

Types are particularly useful for enforcing schemas as integrity constraints. For example,

ic { subset(myrel, (Int, Float, String) ) }

will make sure that myrel has the given type signature (Int × Float × String).

This works for both base relations and derived relations. If myrel is a base relation, then any insert that tries to add something of the wrong type will fail. If myrel is a derived relation, then any database update that would cause it to be populated by the wrong type will also fail.

Summary

This article has covered more advanced features of Rel, including higher-order definitions, overloading by arity and types, varargs, specialization, and point-free syntax.

See Also

For more on other important concepts in Rel, see:

• Integrity Constraints.

• Entities.

• Modules.

The Rel Libraries also provide many more useful relational operators and utilities.