Modules

This concept guide introduces modules in Rel.

Download this guide as a RAI notebook by clicking here.

Introduction

Modules in Rel provide a way to organize and structure our modeling, offering some distinct advantages:

  • Related definitions can be grouped together
  • Models and database logic can be parameterized and reused
  • Names can be local to each module, which prevents clashes and keeps the top-level namespace clean
  • Relations can be renamed before they are used

This concept guide introduces the basic syntax for modules in Rel, and gives some more advanced examples after that.

Simple Example

The basic module syntax lets us group relations under a common namespace. For example:

module store
def office = {"Boston"; "New York"; "Los Angeles"; "Chicago"}
def product = {(1, "Laptops"); (2, "Desktops"); (3, "Phones")}
end

Modules can also be thought of as nested relations. In the above example, office and product are nested inside store, and available as store[:office] and store[:product].

store[:office]

Relation: output

"Boston"
"Chicago"
"Los Angeles"
"New York"
store[:product][1]

Relation: output

"Laptops"

Instead of writing store[:office], you can write store:office. Instead of store[:product][1], you can write store:product[1]. The shorter versions are preferred.

def R = equal(store[:office], store:office)
def output = R

Relation: output

true

Importing Module Names: with

You can choose to import some of the definitions of a module into other namespaces and optionally rename them, using statements of the form with <module> use <relation> [as <newname>]. For example:

with store use office
def output = count[office]

Relation: output

4

This example shows how you can rename relations when you import:

with store use product as p
def output = p[1]

Relation: output

"Laptops"

Using Modules Inside Other Modules

You can use existing module definitions when defining new modules. For example:

module summary
with store use office
def office_count = count[office]
def product = store:product
end

def output = summary

Relation: output

:office_count4
:product1"Laptops"
:product2"Desktops"
:product3"Phones"

Note that while store:office was used to define office_count, it did not become part of summary. By contrast, store:product is part of summary, because of the definition def product = store:product.

Variable Scope

When variable names clash, the innermost scope takes precedence. For example:

def a = 10

module m
def a = 20
def b = a + 5
end

def output = m

Relation: output

:a20
:b25

Imports can be shadowed by a new definition that follows them, which will take precedence. In this case, a warning is printed:

module m1
def a = 1
end

with m1 use a

def a = 5

def output = a

Relation: output

5

The warning states: “‘a’ is imported but not used because there is a declaration of the same name in the same scope.”

If the def a = ... comes before the with, the import takes precedence in the code that follows it.

module m1
def a = 1
end
def a = 5

with m1 use a

def output = a

Relation: output

1

Nested Modules

Modules can be defined within modules (aka nested modules). This is especially useful when data is fully normalized into a Graph Normal Form.

module person
def ssn = 123-45-6789
module name
def first = "John"
def middle = "Q"
def last = "Public"
end
module birth
def city = "Pittsburg"
def state = "PA"
def country = "USA"
def date = parse_date["2000-01-01", "Y-m-d"]
end
end

def output = person:birth:city

Relation: output

"Pittsburg"

Module Union

Like all relations, modules can be defined in separate definitions, which are then unioned:

module m2
def a = 1
end

module m2
def b = 1
end

def output = m2

Relation: output

:a1
:b1

With this feature you can, for example:

  • define large or complex features of a given module separately, even in different installed sources.
  • extend an existing module for the purposes of a particular query, without persisting the change.

You can also take the union of two separate modules. For example:

module A
def a = 1
end

module B
def a = 1
def b = 1
end

def AB = A ; B

def output = AB

Relation: output

:a1
:b1

This lets us extend a given module with functionality defined in another.

Integrity Constraints

Currently, basic integrity constraints are supported in non-parameterized modules (future releases will extend this functionality to parameterized modules). A simple example is shown below:

module mymodule
def R = {1; 2}
ic {count[R] = 2}
end

def output = mymodule

Relation: output

:R1
:R2

Parameterized Modules

Modules can be made more re-usable by parameterizing them by one or more relations.

Parameterized modules must use the @inline annotation. For example, let’s build a simple module that collects a few statistics for a relation R:

@inline
module my_stats[R]
def my_minmax = (min[R], max[R])
def my_mean = mean[R]
def my_median = median[R]
end
def output = my_stats[{1; 2; 3; 5; 8; 13; 100}]

Relation: output

:my_mean18.857142857142858
:my_median5.0
:my_minmax1100

The module parameter R can be instantiated with any relation that has a numeric range. For example:

def pop = {("a", 10); ("b", 100); ("c", 20); ("d", 23)}
def output = my_stats[pop]

Relation: output

:my_mean38.25
:my_median21.5
:my_minmax10100

Inlining

Note that parameterized modules should be inlined. Depending on the module, instances of the module might need @inline as well, as is the case with m in the (rather artificial) example below:

@inline
module mymodule[x]
def multiply[y] = y * x
def divide[y] = y / x
end

@inline
def m = mymodule[10]

with mymodule[20] use divide as divides_twenty
def output:first = m:divide[240]
def output:second = divides_twenty[120]

Relation: output

:first24.0
:second6.0

Lowercasing x when parameterizing the module mymodule indicates that this module is parameterized by an individual variable, and not a relation (in contrast with the module my_stats above).

Example: Graph Modules

Since modules are relations themselves, they can be used as arguments to other modules,

Consider a module that constructs a complete graph for a set of nodes N; that is, a graph where each node is connected to all other nodes:

@inline
module CompleteGraph[N]
def node = N
def edge = N,N
end

The cell below defines a bipartite graph, with M nodes each connected to N nodes, as well as a cycle graph with N nodes (where each node is connected to one “next” node, forming a loop):

@inline
module BipartiteGraph[M, N]
def node = M; N
def edge = M, N
end

@inline
module CycleGraph[N]
def node = N
def edge(a in N, b in N) =
sort[N](x, a)
and sort[N](y, b)
and y = x%count[N] + 1
from x, y
end

The module GraphProperties defined below computes some basic properties of a graph module G:

@inline
module GraphProperties[G]
def outdegree[v in G:node] = count[v1 : G:edge(v, v1)] <++ 0
def indegree[v in G:node] = count[v1 : G:edge(v1, v)] <++ 0
def edge_count = count[G:edge] <++ 0
end

The following code instantiates the BipartiteGraph module and passes it as a parameter to the GraphProperties module to compute its properties:

def graph = BipartiteGraph[{"l1"; "l2"}, {"r1"; "r2"; "r3"}]

def output = GraphProperties[graph]

Relation: output

:edge_count6
:indegree"l1"0
:indegree"l2"0
:indegree"r1"2
:indegree"r2"2
:indegree"r3"2
:outdegree"l1"3
:outdegree"l2"3
:outdegree"r1"0
:outdegree"r2"0
:outdegree"r3"0

Here are more examples showing how you can use these definitions:

def cg = CompleteGraph[range[1 ,5, 1]]
def cg_props = GraphProperties[cg]

def bg = BipartiteGraph[{1; 2}, {3; 4; 5}]
def bg_props = GraphProperties[bg]

def cycleg = CycleGraph[{"a"; "b"; "c"; "d" ; "e"}]
def cycleg_props = GraphProperties[cycleg]

module output
def complete_edge_count = cg_props:edge_count
def bipartite_edge_count = bg_props:edge_count
def cycle_edge_count = cycleg_props:edge_count
end

Relation: output

:bipartite_edge_count6
:complete_edge_count25
:cycle_edge_count5

Imported CSV files and JSON documents are modules

The result of importing CSV and JSON data can be viewed as a module.

CSV data as modules

Loading a CSV file into a relation csv_data gives us a module of the form

module csv_data
def column_one[pos] = ...
def column_two[pos] = ...
...
end

where csv_data:column_name is a relation that maps positions to values, for each column_name in the CSV file. (See the CSV Import Guide for details.) For example:

def config:data = """
order_id,customer,price
1,92819,3.4
3,78271,1.6
"""

def csv_data = load_csv[config]
def output = csv_data

Relation: output

:customerRelationalAITypes.FilePos(1)"customer"
:customerRelationalAITypes.FilePos(25)"92819"
:customerRelationalAITypes.FilePos(37)"78271"
:order_idRelationalAITypes.FilePos(1)"order_id"
:order_idRelationalAITypes.FilePos(25)"1"
:order_idRelationalAITypes.FilePos(37)"3"
:priceRelationalAITypes.FilePos(1)"price"
:priceRelationalAITypes.FilePos(25)"3.4"
:priceRelationalAITypes.FilePos(37)"1.6"

It is also possible to load a CSV file row-wise, using load_csv_row_wise. The result is a module parameterized by the file position (so each instance is a row). For example:

def config:data = """
order_id,customer,price
1,92819,3.4
3,78271,1.6
"""

def csv_data = load_csv_row_wise[config]
def output = csv_data

Relation: output

RelationalAITypes.FilePos(1):customer"customer"
RelationalAITypes.FilePos(1):order_id"order_id"
RelationalAITypes.FilePos(1):price"price"
RelationalAITypes.FilePos(25):customer"92819"
RelationalAITypes.FilePos(25):order_id"1"
RelationalAITypes.FilePos(25):price"3.4"
RelationalAITypes.FilePos(37):customer"78271"
RelationalAITypes.FilePos(37):order_id"3"
RelationalAITypes.FilePos(37):price"1.6"

This relation is equivalent to the module:

module order_csv[pos]
def order_id = ...
def customer = ...
def total_price = ...
end

which in turn is equivalent to:

def order_csv[pos, :order_id] = ...
def order_csv[pos, :customer] = ...
def order_csv[pos, :total_price] = ...

JSON data as modules

In the JSON case, the input JSON structure is naturally reflected in the resulting Rel module structure, with nested JSON structures becoming nested modules, recursively. JSON names become Rel symbols, and values become scalars (single values) or nested modules.

For example:

def config:data = """
{
"name": "John",
"address": {"state": "WA", "city": "Seattle"},
"age": 21
}"""

def person = load_json[config]
def output = person

Relation: output

:address:city"Seattle"
:address:state"WA"
:age21
:name"John"