Modules

This concept guide introduces modules in Rel.

Download this guide as a RAI notebook by clicking here.

Introduction

Modules in Rel give us a way to organize and structure our modeling, offering some distinct advantages:

  • Related definitions can be grouped together
  • Models and database logic can be parameterized and reused
  • Names can be local to each module, which prevents clashes and keeps the top-level namespace clean
  • Relations can be renamed before they are used

This concept guide introduces the basic syntax for modules in Rel, and gives some more advanced examples after that.

Simple Example

The basic module syntax lets us group relations under a common namespace. For example:

install
module store
def office = {"Boston"; "New York"; "Los Angeles"; "Chicago"}
def product = {(1, "Laptops"); (2, "Desktops"); (3, "Phones")}
end

Modules can also be thought of as nested relations. In the above example, office and product are nested inside store, and available as store[:office] and store[:product].

query
store[:office]

Relation: output

"Boston"
"Chicago"
"Los Angeles"
"New York"
query
store[:product][1]

Relation: output

"Laptops"

Instead of writing store[:office], we can write store:office. Instead of store[:product][1], we can write store:product[1]. We prefer the shorter versions.

query
if equal(store[:office], store:office) then "yes" else "no" end

Relation: output

"yes"

Importing Module Names

We can choose to import some of the definitions of a module into other namespaces and optionally rename them, using statements of the form with <module> use <relation> [as <newname>]. For example:

query
with store use office
def output = count[office]

Relation: output

4

This example shows how we can rename relations when we import:

query
with store use product as p
def output = p[1]

Relation: output

"Laptops"

Using Modules Inside Other Modules

We can use existing module definitions when we define new modules. For example:

query
module summary
with store use office
def office_count = count[office]
def product = store:product
end

def output = summary

Relation: output

:office_count4
:product1"Laptops"
:product2"Desktops"
:product3"Phones"

Note that while store:office was used to define office_count, it did not become part of summary. In contrast, we made store:product a part of summary by adding the definition def product = store:product.

Variable Scope

When variable names clash, the innermost scope takes precedence. For example:

query
def a = 10

module m
def a = 20
def b = a + 5
end

def output = m

Relation: output

:a20
:b25

Imports can be shadowed by a new definition that follows them, which will take precedence. In this case, a warning is printed:

query
module m1
def a = 1
end

with m1 use a

def a = 5

def output = a

Relation: output

5

The warning states: “‘a’ is imported but not used because there is a declaration of the same name in the same scope.”

If the def a = ... comes before the with, the import takes precedence in the code that follows it.

query
module m1
def a = 1
end
def a = 5

with m1 use a

def output = a

Relation: output

1

Nested Modules

Modules can be defined within modules (aka nested modules). This is especially useful when data is fully normalized into a Graph Normal Form.

query
module person
def ssn = 123-45-6789
module name
def first = "John"
def middle = "Q"
def last = "Public"
end
module birth
def city = "Pittsburg"
def state = "PA"
def country = "USA"
def date = parse_date["2000-01-01", "Y-m-d"]
end
end

def output = person:birth:city

Relation: output

"Pittsburg"

Module Union

Like all relations, modules can be defined in separate definitions, which are then unioned:

query
module m2
def a = 1
end

module m2
def b = 1
end

def output = m2

Relation: output

:a1
:b1

With this feature we can, for example:

  • define large or complex features of a given module separately, even in different installed sources.
  • extend an existing module for the purposes of a particular query, without persisting the change.

We can also take the union of two separate modules. For example:

query
module A
def a = 1
end

module B
def a = 1
def b = 1
end

def AB = A ; B

def output = AB

Relation: output

:a1
:b1

This lets us extend a given module with functionality defined in another.

Integrity Constraints

Currently, basic integrity constraints are supported in non-parameterized modules (future releases will extend this functionality to parameterized modules). A simple example is shown below:

query
module mymodule
def R = {1; 2}
ic {count[R] = 2}
end

def output = mymodule

Relation: output

:R1
:R2

Parameterized Modules

Modules can be made more re-usable by parameterizing them by one or more relations.

Parameterized modules must use the @inline annotation. For example, let’s build a simple module that collects a few statistics for a relation R:

install
@inline
module my_stats[R]
def my_minmax = (min[R], max[R])
def my_mean = mean[R]
def my_median = median[R]
end
query
def output = my_stats[{1; 2; 3; 5; 8; 13; 100}]

Relation: output

:my_mean18.857142857142858
:my_median5.0
:my_minmax1100

The module parameter R can be instantiated with any relation that has a numeric range. For example:

query
def pop = {("a", 10); ("b", 100); ("c", 20); ("d", 23)}
def output = my_stats[pop]

Relation: output

:my_mean38.25
:my_median21.5
:my_minmax10100

Inlining

Note that parameterized modules should be inlined. Depending on the module, instances of the module might need @inline as well, as is the case with m in the (rather artificial) example below:

query
@inline
module mymodule[x]
def multiply[y] = y * x
def divide[y] = y / x
end

@inline
def m = mymodule[10]

with mymodule[20] use divide as divides_twenty
def output:first = m:divide[240]
def output:second = divides_twenty[120]

Relation: output

:first24.0
:second6.0

By using a lowercase x when parameterizing the module mymodule, we explicitly state that this module is parameterized by a individual variable, and not a relation (in contrast with the module my_stats above).

Example: Graph Modules

Since modules are relations themselves, they can be used as arguments to other modules,

Consider a module that constructs a complete graph for a set of nodes N; that is, a graph where each node is connected to all other nodes. So we can reuse it, we first install the module definition:

install
@inline
module CompleteGraph[N]
def node = N
def edge = N,N
end

We similarly define a bipartite graph, with M nodes each connected to N nodes, and a cycle graph with N nodes (where each node is connected to one “next” node, forming a loop):

install
@inline
module BipartiteGraph[M, N]
def node = M; N
def edge = M, N
end

@inline
module CycleGraph[N]
def node = N
def edge(a in N, b in N) =
sort[N](x, a)
and sort[N](y, b)
and y = x%count[N] + 1
from x, y
end

We also define and install a separate module that computes some basic properties of a graph, taking a graph module G as its argument:

install
@inline
module GraphProperties[G]
def outdegree[v in G:node] = count[v1 : G:edge(v, v1)] <++ 0
def indegree[v in G:node] = count[v1 : G:edge(v1, v)] <++ 0
def edge_count = count[G:edge] <++ 0
end

We can now instantiate the BipartiteGraph module and pass it as a parameter to the GraphProperties module to see its properties:

query
def bg = BipartiteGraph[{"l1"; "l2"}, {"r1"; "r2"; "r3"}]

def output = GraphProperties[bg]

Relation: output

:edge_count6
:indegree"l1"0
:indegree"l2"0
:indegree"r1"2
:indegree"r2"2
:indegree"r3"2
:outdegree"l1"3
:outdegree"l2"3
:outdegree"r1"0
:outdegree"r2"0
:outdegree"r3"0

Here are more examples of how we can use these definitions:

query
def cg = CompleteGraph[range[1 ,5, 1]]
def cg_props = GraphProperties[cg]

def bg = BipartiteGraph[{1; 2}, {3; 4; 5}]
def bg_props = GraphProperties[bg]

def cycleg = CycleGraph[{"a"; "b"; "c"; "d" ; "e"}]
def cycleg_props = GraphProperties[cycleg]

module output
def complete_edge_count = cg_props:edge_count
def bipartite_edge_count = bg_props:edge_count
def cycle_edge_count = cycleg_props:edge_count
end

Relation: output

:bipartite_edge_count6
:complete_edge_count25
:cycle_edge_count5

Imported CSV files and JSON documents are modules

The result of importing CSV and JSON data can be viewed as a module.

CSV data as modules

Loading a CSV file into a relation csv_data gives us a module of the form

module csv_data
def column_one[pos] = ...
def column_two[pos] = ...
...
end

where csv_data:column_name is a relation that maps positions to values, for each column_name in the CSV file. (See the CSV Import Guide for details.) For example:

query
def config:data = """
order_id,customer,price
1,92819,3.4
3,78271,1.6
"""

def csv_data = load_csv[config]
def output = csv_data

Relation: output

:customerRelationalAITypes.FilePos(25)"92819"
:customerRelationalAITypes.FilePos(37)"78271"
:order_idRelationalAITypes.FilePos(25)"1"
:order_idRelationalAITypes.FilePos(37)"3"
:priceRelationalAITypes.FilePos(25)"3.4"
:priceRelationalAITypes.FilePos(37)"1.6"

If we instead load a CSV file row-wise, using load_csv_row_wise, the result is a module parameterized by the file position (so each instance is a row). For example:

query
def config:data = """
order_id,customer,price
1,92819,3.4
3,78271,1.6
"""

def csv_data = load_csv_row_wise[config]
def output = csv_data

Relation: output

RelationalAITypes.FilePos(25):customer"92819"
RelationalAITypes.FilePos(25):order_id"1"
RelationalAITypes.FilePos(25):price"3.4"
RelationalAITypes.FilePos(37):customer"78271"
RelationalAITypes.FilePos(37):order_id"3"
RelationalAITypes.FilePos(37):price"1.6"

This relation is equivalent to the module:

module order_csv[pos]
def order_id = ...
def customer = ...
def total_price = ...
end

which in turn is equivalent to:

def order_csv[pos, :order_id] = ...
def order_csv[pos, :customer] = ...
def order_csv[pos, :total_price] = ...

JSON data as modules

In the JSON case, the input JSON structure is naturally reflected in the resulting Rel module structure, with nested JSON structures becoming nested modules, recursively. JSON names become Rel symbols, and values become scalars (single values) or nested modules.

For example:

query
def config:data = """
{
"name": "John",
"address": {"state": "WA", "city": "Seattle"},
"age": 21
}"""

def person = load_json[config]
def output = person

Relation: output

:address:city"Seattle"
:address:state"WA"
:age21
:name"John"