Skip to content
Rel
REFERENCE
Libraries
stdlib

The Standard Library (stdlib)

Broad collection of Rel relations that perform essential and commonly used tasks.

abs

abs[x]

The absolute value of x.

Examples:

abs[-2] = 2
abs[-2.0] = 2.0

Definition

@inline
def abs[x] = rel_primitive_abs[x]

acos

acos[x]

Inverse cosine (in radians).

Defined for x between -1 and 1 (inclusive).

Example:

acos[0] = 1.5707963267948966

Definition

@inline
def acos[x] = rel_primitive_acos[x]

acosh

acosh[x]

Inverse hyperbolic cosine.

Defined for x >= 1.

Definition

@inline
def acosh[x] = rel_primitive_acosh[x]

acot

acot[x]

Inverse cotangent (in radians).

Definition

@inline
def acot[x] = rel_primitive_acot[x]

add

add[x, y]
x + y

Addition.

Examples:

  • 1 + 3 is an expression with value 4
  • add[1, 3] is an expression with value 4
  • add(1, 3, 4) is a formula that is true.
1 + 3 = 4
add[1, 3] = 4
equal(add(1, 3, 4), true)

Definition

@inline
def add(x, y, z) = rel_primitive_add(x, y, z)


@inline
def (+)(x, y, z) = add(x, y, z)

Any

Any(x)

Holds for any x, where x exists. (Any functions as a wildcard.)

Example:

Integrity constraint that tests whether x is of any type:

def R = (1, 3) ; (1, "foo")

ic any_ic {subset(R, (Int, Any) )}

Definition

@inline
def Any(x) = true

argmax

argmax[R]

Find the keys for the largest value of the relation R.

Examples:

def output = argmax[{(2, 3); (1, 6)}]
1

def output = argmax[{(2, 6); (1, 6); (5, 0)}]
1
2

// find the teams with the largest aggregated salary:
argmax[d in team: sum[salary[p] for p in member[d]]]

Definition

@inline
def argmax[R] = R.(max[R])

ArgMax

ArgMax[R]

Please use argmax[R]. Deprecates in near future

Definition

@inline
def ArgMax = argmax

argmin

argmin[R]

Find the keys for the smallest value of the relation R.

Examples:

def output = argmin[{(2, 3); (1, 6)}]
2

def output = argmin[{(2, 6); (1, 6); (5, 10)}]
1
2

// find the teams with the smallest aggregated salary:
argmin[d in team: sum[salary[p] for p in member[d]]]

Definition

@inline
def argmin[R] = R.(min[R])

ArgMin

ArgMin[R]

Please use argmin[R]. Deprecates in near future

Definition

@inline
def ArgMin = argmin

arity

arity[R]

The arity of a relation. In some cases, it can be an over-approximation.

Arity is a higher-order relation that is always evaluated at compile-time.

Examples:

def output = arity[3]
1
def output = arity[{1; 2; 3}]
1
def output = arity[(1, 2)]
2
def output = arity[add]
3
def output = arity[{1; 2; (1,2)}]
1
2

Arity can be used to do meta-programming in logic. For example, the following abstraction verbalize implements specific cases using arity.

Examples:

@inline def verbalize[R] = "nullary", arity[R] = 0;
                            "unary", arity[R] = 1;
                            "binary", arity[R] = 2
def output = verbalize[true]
"nullary"
def output = verbalize[1]
"unary"

Arity can be used in higher-order abstractions to check at compile-time that they are used correctly.

Arity can be used in integrity constraints to state expectation on EDB or IDB relations. Because arity is evaluated at compile-time, it can catch mistakes in the logic before the logic executes.

Example:

def p = 1,2,3
ic { arity[p] = 3 }

Note that there is a difference between R(_, _) and arity(R) = 2. The first requires R to be non-empty, which is an runtime property of R.

Definition

@inline
def arity[R] = rel_primitive_arity[R]

asin

asin[x]

Inverse sine (in radians).

Defined for x between -1 and 1 (inclusive).

Example:

asin[1] = 1.5707963267948966

Definition

@inline
def asin[x] = rel_primitive_asin[x]

asinh

asinh[x]

Inverse hyperbolic sine.

Definition

@inline
def asinh[x] = rel_primitive_asinh[x]

atan

atan[x]

Inverse tangent (in radians).

Defined for x between -1 and 1 (inclusive).

Definition

@inline
def atan[x] = rel_primitive_atan[x]

atan2

atan2[y, x]

Inverse tangent: an angle (in radians) between the positive x-axis and the ray to the point (x, y).

Definition

@inline
def atan2[x,y] = rel_primitive_atan2[x,y]

atanh

atanh[x]

Inverse hyperbolic tangent.

Definition

@inline
def atanh[x] = rel_primitive_atanh[x]

auto_number

auto_number[R]

DEPRECATED

This function is deprecated and should be avoided. It will be removed soon.

AutoNumber a relation.

auto_number takes a relation R(xs...) and produces a relation R(xs..., i) where i is a distinct AutoNumberValue index.

Note that auto_number can give different results each time it is called.

Example:

def sample = 'a'
def output = auto_number[sample]
('a', AutoNumberValue(0x12))

def sample = {'a'; 'b'; 'c'}
def output = auto_number[sample]
('a', AutoNumberValue(0x132))
('b', AutoNumberValue(0x133))
('c', AutoNumberValue(0x134))

Definition

@inline
def auto_number = rel_primitive_auto_number

AutoNumber

AutoNumber(x)

DEPRECATED

This function is deprecated and should be avoided. It will be removed soon.

Test if the given value is an AutoNumberValue.

Definition

@inline
def AutoNumber = rel_primitive_AutoNumberValue

average

mean[R]
average[R]

The arithmetic mean of (the last argument of) the relation R, if non-empty. False if R is empty.

Definition

@inline
def average = mean

bitwise_and

bitwise_and[x, y]

Bitwise and of two integers.

Example:

bitwise_and[3,2] = 2
bitwise_and[0x10011, 0x11100] = 0x00010000

Definition

@inline
def bitwise_and[x, y] = rel_primitive_bitwise_and[x, y]

bitwise_left_shift

bitwise_left_shift[x, n]

Bitwise left shift of an integer x by n bits.

Examples:

bitwise_left_shift[8, 1] = 16
bitwise_left_shift[1, 10] = 1024
bitwise_left_shift[0xF, 1] = 0x1d

Definition

@inline
def bitwise_left_shift[x, n] = rel_primitive_bitwise_left_shift[x, n]

bitwise_not

bitwise_not[x]

Bitwise not of an integer.

Examples:

bitwise_not[8] = -9
bitwise_not[-9] = 8
bitwise_not[0x00011] = 0xffffffee

Definition

@inline
def bitwise_not[x] = rel_primitive_bitwise_not[x]

bitwise_or

bitwise_or[x, y]

Bitwise or of two integers.

Example:

bitwise_or[3, 2] = 3
bitwise_or[0x00011, 0x11100] = 0x00011111

Definition

@inline
def bitwise_or[x, y] = rel_primitive_bitwise_or[x, y]

bitwise_right_shift

bitwise_right_shift[x, n]

Bitwise right shift of an integer x by n bits while preserving the sign.

Examples:

bitwise_right_shift[1024, 1] = 512
bitwise_right_shift[-1024, 1] = -512

Definition

@inline
def bitwise_right_shift[x, n] = rel_primitive_bitwise_signed_right_shift[x, n]

bitwise_unsigned_right_shift

bitwise_unsigned_right_shift[x,n]

Bitwise right shift of an integer by n bits.

Examples:

bitwise_unsigned_right_shift[8, 1] = 4
bitwise_unsigned_right_shift[-8, 1] = 9223372036854775804

Definition

@inline
def bitwise_unsigned_right_shift[x, n] = rel_primitive_bitwise_unsigned_right_shift[x, n]

bitwise_xor

bitwise_xor[x, y]

Bitwise xor of two integers.

Example:

bitwise_xor[3, 2] = 1
bitwise_xor[0x00011, 0x11100] = 0x00011111

Definition

@inline
def bitwise_xor[x, y] = rel_primitive_bitwise_xor[x, y]

Boolean

Boolean(x)

Holds if x is a Boolean.

Example:

def json = parse_json["""{"a": true, "b": false}"""]
def output(x) = json(:a, x) and Boolean(x)

Definition

@inline
def Boolean = rel_primitive_Boolean

boolean_and

boolean_and(x, y, z)

Logical AND operator for the Boolean data type.

Example:

def output(x, y, z) = boolean_and(x, y, z) and boolean_true(z)

Definition

@inline
def boolean_and(x in Boolean, y in Boolean, z in Boolean) =
    if boolean_true(x) and boolean_true(y) then
        boolean_true(z)
    else
        boolean_false(z)

boolean_false

boolean_false(x)

Holds if x is a Boolean of value false.

Definition

@inline
def boolean_false = rel_primitive_boolean_false

boolean_not

boolean_not(x,y)

Negation(not) operator for the Boolean data type. Example:

def output(x, y) = boolean_not(x, y) and boolean_false(x)

Definition

@inline
def boolean_not(x in Boolean, y in Boolean) =
    if boolean_true(x) then
        boolean_false(y)
    else
        boolean_true(y)

boolean_or

boolean_or(x, y, z)

Logical or operator for the Boolean data type. Example:

def output(x, y, z) = boolean_or(x, y, z) and boolean_false(z)

Definition

@inline
def boolean_or(x in Boolean, y in Boolean, z in Boolean) =
    if boolean_false(x) and boolean_false(y) then
        boolean_false(z)
    else
        boolean_true(z)

boolean_true

boolean_true(x)

Holds if x is a Boolean of value true.

Definition

@inline
def boolean_true = rel_primitive_boolean_true

bottom

bottom[k, R]

Select the bottom k facts of relation R according to the sort order of R.

bottom is reverse_sort (which sorts highest to lowest) restricted to the first k facts.

Example:

def output =  bottom[2, {'a'; 'b'; 'c'; 'd'}]
(1, 'd')
(2, 'c')

Definition

@inline
def bottom[k, R] =
    reverse_sort[R][i] for i where i <= k

byte

byte[str]
byte[str, i]
byte(str, i, b)

Indexes into a string at byte position i, mapping each position i to a byte b, as a UInt8 value.

If a string contains Unicode characters, the byte at index i might be only a partial character. Be careful with your indexing logic.

Both i and b can be optionally bound externally. When only str is bound, this is the mapping from each index to its corresponding byte.

Examples: Indexing into a known byte index:

byte["abcd", 2] = 0x62
byte["中文例子", 2] = 0xb8

Abstracting over the byte index:

equal(byte["中文"],
        { 1, 0xe4;
          2, 0xb8;
          3, 0xad;
          4, 0xe6;
          5, 0x96;
          6, 0x87; })
equal((i : byte["awesome", i](0x65)), {3; 7})

Definition

@inline
def byte[s](i, b) =
    rel_primitive_byte(s, i, b) and range[1, num_bytes[s], 1](i)

capture_group_by_index

capture_group_by_index[regex, input_string, offset]

A set of capture groups, each of the form (index, substring), where index is the capture group index, and substring is the first regex match in input_string, starting at the character index specified by offset. regex can be a string or a pattern.

Offsets (character indexes) start at 1.

Example:

capture_group_by_index["(\\d+):(\\d+)",
    "Appointment from  12:45 to 1:30",
    19]

is equal to

{(1, "12"); (2, "45")}

Definition

@inline
def capture_group_by_index[re, str, offset](idx, capture) =
    rel_primitive_capture_grp[regex_compile[re], str, offset](idx, capture)

capture_group_by_name

capture_group_by_name[regex, input_string, offset]

A set of capture groups, each of the form (name, substring), where name is the capture group name, and substring is the first regex match in input_string, starting at the character index specified by offset. regex can be a string or a pattern.

Offsets (character indexes) start at 1.

Each capture group should have a unique name.

Example:

capture_group_by_name["(?<hour>\\d+):(?<minute>\\d+)",
    "Appointment from 12:45 to 1:30",
    19]

is equal to

(("hour","12"), ("minute","45"))

Definition

@inline
def capture_group_by_name[re, str, offset](name, capture) =
    exists( idx: rel_primitive_capture_grp[regex_compile[re], str, offset](idx, capture)
            and rel_primitive_capture_names(regex_compile[re], idx, name))

cart

cart[R, S]
R × S

Cartesian product.

Examples:

def output = 1 ✕ 2
(1,2)
def output = {1; 2} ✕ {3; 4}
(1,3)
(1,4)
(2,3)
(2,4)

Definition

@inline
def cart[R, S](x..., y...) = R(x...) and S(y...)

@inline
def (×) = cart

cbrt

cbrt[x]

The cube root of x.

Example:

cbrt[27] = 3.0

Definition

@inline
def cbrt[x] = rel_primitive_cbrt[x]

ceil

ceil[x]

Round up to the nearest integer.

Examples:

ceil[4.5] = 5.0
ceil[-4.5] = -4.0
ceil[4] = 4

Definition

@inline
def ceil[x] = round[:ROUND_UP, x]

char

char[str]
char[str, i]
char(str, i, c)

Indexes into a string at (Int) position i, mapping each index i to the i-th character, c.

Since this is indexed using character positions, the characters will always be whole Unicode characters.

A character is also known as a “Code Point” in the Unicode specification.

Both i and c can be optionally bound externally. When only str is bound, this is the mapping from each character index to its corresponding character.

Examples:

Indexing into a known character index:

char["abcd", 2] = 'b'
char["中文例子", 2] = '文'

Abstracting over the character index:

equal(char["中文"],
        { 1, "中"; 2, "文" })
equal((i : char["awesome", i]('e')), {3; 7})

Definition

@inline
def char[s](i, c) =
    rel_primitive_char(s, i, c) and range[1, num_chars[s], 1](i)

Char

Char(x)

Holds if x is of type Char, which has a Unicode character as its value and is specified with single quotes.

Examples:

Integrity constraint that tests whether x is of type Char:

def R = 't'

ic mychar_ic(x in R)
    Char(x)
}

Schema defined in a relation using Char:

def myrelation(x in Char, y in Int) {
    x = 'a' and y = 123
}

def output = myrelation
//output> (a, 123)

Definition

@inline
def Char = rel_primitive_Char

clamp

clamp[lo, hi, v]

clamp limits the value v to a given range(lo, hi), changing values outside the range to the closest lo, hi value if necessary. The arguments should have the same type.

Examples:

clamp[2, 5, 1] = 2
clamp[2, 5, 6] = 5
clamp[2, 5, 3] = 3

Definition

@inline
def clamp[lo, hi, v] = min[{max[{lo; v}]; hi}]

complement

complement[R]

The complement of the relation R.

Definition

@inline
def complement[R](x...) = not R(x...)

concat

concat[val1, val2]

String concatenation of two arbitrary values

Example:

concat["a", "b"] = "ab"
concat["a", 'b'] = "ab"
concat['a', "b"] = "ab"
concat['a', 'b'] = "ab"
concat["a_", 1] = "a_1"
concat[1, 3.14] = "13.14"

Definition

@inline
def concat[x, y] = rel_primitive_concat[string[x], string[y]]

cos

cos[x]

Cosine of x (given in radians).

Defined for non-infinite x.

Example:

cos[pi_float64] = -1.0

Definition

@inline
def cos[x] = rel_primitive_cos[x]

cosh

cosh[x]

Hyperbolic cosine.

Definition

@inline
def cosh[x] = rel_primitive_cosh[x]

cot

cot[x]

Cotangent of x (given in radians).

Definition

@inline
def cot[x] = rel_primitive_cot[x]

count

count[R]

Number of tuples in R, if R is not empty. If R is empty, count[R] is false.

Examples:

  • count[employee] is the number of tuples in relation employee
  • d in department: count[d.member] is the member count for every department with at least 1 member
  • count[5] is 1 because 5 is a singleton relation
  • count[{}] is false (empty). To get 0 instead, use left override, as in count[R] <++ 0.

Definition

@inline
def count[R] = sum[{R,1}]

Date

Date(x)

Holds if x is a Date.

Definition

@inline
def Date = rel_primitive_Date

date_add

date_add[d, period]
d + period

Add a Period to a Date

Example:

def d = parse_date["2021-09-21", "Y-m-d"]
def output = date_add[d, Day[20]]
Dates.Date("2021-10-11")

Definition

@inline
def date_add = rel_primitive_add_date_period

date_day

date_day[d]

Day of the month for a Date, as an integer between 1 and 31, inclusive.

Example:

def d = parse_date["2014-10-31", "Y-m-d"]
date_day[d] = 31

Definition

@inline
def date_day = rel_primitive_date_day

date_dayname

date_dayname[d]

Name of week-day (a String, e.g., Friday)

Example:

def t = parse_date["2014-01-31", "Y-m-d"]
def output = date_dayname[t]
"Friday"

Definition

@inline
def date_dayname = rel_primitive_date_dayname

date_dayofquarter

date_dayofquarter[d]

Day of quarter

Example:

def d = parse_date["2014-01-31", "Y-m-d"]
date_dayofquarter[d] = 31

Definition

@inline
def date_dayofquarter = rel_primitive_date_dayofquarter

date_dayofweek

date_dayofweek[d]

Day of the week for a date, as an integer between 1 and 7, where 1 is Monday and 7 is Sunday. (That is, 5 for Friday.)

Example:

def d = parse_date["2014-01-31", "Y-m-d"]
date_dayofweek[d] = 5

Definition

@inline
def date_dayofweek = rel_primitive_date_dayofweek

date_dayofweekofmonth

date_dayofweekofmonth[d]

Day of week of month, as an integer (e.g. 2 for second Friday of the month, 1 for the first Tuesday).

Example:

def t = parse_date["2014-01-31", "Y-m-d"]
date_dayofweekofmonth[t] = 5 // fifth Friday of that month

Definition

@inline
def date_dayofweekofmonth = rel_primitive_date_dayofweekofmonth

date_dayofyear

date_dayofyear[d]

Day of year

Example:

def t = parse_date["2014-01-31", "Y-m-d"]
def output = date_dayofyear[t]
31

Definition

@inline
def date_dayofyear = rel_primitive_date_dayofyear

date_daysinmonth

date_daysinmonth[d]

The number of days in the month for date d

Examples:

def d1 = parse_date["2014-01-30", "Y-m-d"]
date_daysinmonth[t] = 31
def d2 = parse_date["2014-02-11", "Y-m-d"]
date_daysinmonth[d2] = 28
def d = parse_date["2016-02-11", "Y-m-d"]
date_daysinmonth[d] = 29

Definition

@inline
def date_daysinmonth = rel_primitive_date_daysinmonth

date_isleapyear

date_isleapyear[d]

True iff the year for date d is a leap year.

Examples:

def notleap = parse_date["2014-01-31", "Y-m-d"]
def leap = parse_date["2016-01-31", "Y-m-d"]
def output = date_isleapyear[notleap]
false
def output = date_isleapyear[leap]
true

Definition

@inline
def date_isleapyear = rel_primitive_date_isleapyear

date_month

date_month[d]

Month of a Date, as an integer between 1 and 12.

Definition

@inline
def date_month = rel_primitive_date_month

date_monthname

date_monthname[d]

The month name for date d, as a string.

Example:

def t = parse_date["2014-01-31", "Y-m-d"]
date_monthname[t] = "January"

Definition

@inline
def date_monthname = rel_primitive_date_monthname

date_quarterofyear

date_quarterofyear[d]

Quarter to year

Example:

def d = parse_date["2014-01-31", "Y-m-d"]
date_quarterofyear[d] = 1

Definition

@inline
def date_quarterofyear = rel_primitive_date_quarterofyear

date_subtract

date_subtract[date, period]
date - period

Subtract a Period from a Date, giving another Date.

Example:

def d = parse_date["2021-09-21", "Y-m-d"]
def output = date_subtract[d, Day[1000]]
Dates.Date("2018-12-26")

Definition

@inline
def date_subtract = rel_primitive_subtract_date_period

date_week

date_week[d]

Week of the year for a Date, as an integer, where the first week is the week that contains the first Thursday of the year. Ranges between 1 and 53, inclusive.

Example:

week[parse_date["2005-01-01", "Y-m-d"]] = 53 // 53rd week of 2004.
week[parse_date["2001-01-01", "Y-m-d"]] = 1

Definition

@inline
def date_week = rel_primitive_date_week

date_year

date_year[d]

Year of a Date, as an integer.

Example:

date_year[parse_date["2020-05-22", "Y-m-d"]]
2020

Year[date_year[parse_date["2020-05-22", "Y-m-d"]]]
(2020 years,)

Definition

@inline
def date_year = rel_primitive_date_year

dates_period_days

dates_period_days[date1, date2]

The difference in days between two dates date2 and date1 as a Day data type.

Example:

def days = dates_period_days[
    2022-05-12,
    2022-05-15
]

ic { equal(days, Day[3])}

Definition

@inline
def dates_period_days = rel_primitive_dates_period

DateTime

DateTime(x)

Holds if x is a DateTime.

Definition

@inline
def DateTime = rel_primitive_DateTime

datetime_add

datetime_add[dt, period]
dt + period

Add a Period to a DateTime

Examples:

def dt = parse_datetime["2021-01-01 01:00:00", "Y-m-d H:M:S"]
def output = datetime_add[dt, Year[1]]
Dates.DateTime("2022-01-01T01:00:00")

def output = datetime_add[dt, Month[13]]
Dates.DateTime("2022-02-01T01:00:00")

def output = datetime_add[dt, Hour[1000]]
Dates.DateTime("2021-02-11T17:00:00")

Definition

@inline
def datetime_add = rel_primitive_add_datetime_period

datetime_day

datetime_day[dt, tz]

Day of the month, for a DateTime. Note that DateTime properties require a timezone string tz.

Example:

datetime_day[unix_epoch, "UTC"] = 1
datetime_day[unix_epoch, "UTC-10"] = 31

Definition

@inline
def datetime_day = rel_primitive_datetime_day

datetime_day_UTC

datetime_day_UTC[dt]

Day assuming datetime is in UTC.

Definition

@inline
def datetime_day_UTC[d] = rel_primitive_datetime_day[d, "UTC"]

datetime_dayname

datetime_dayname[t, tz]

Name of week-day (a string, e.g., Friday)

Example:

def t = parse_datetime["2014-01-31 01:00 +03:00", "YYYY-mm-dd HH:MM zzzz"]
datetime_dayname[t, "+03:00"] = "Friday"

Definition

@inline
def datetime_dayname = rel_primitive_datetime_dayname

datetime_dayofquarter

datetime_dayofquarter[dt, tz]

Day of quarter

Example:

def t = parse_datetime["2014-01-31 01:00 +03:00", "YYYY-mm-dd HH:MM zzzz"]
datetime_dayofquarter[t, "+03:00"] = 31

Definition

@inline
def datetime_dayofquarter = rel_primitive_datetime_dayofquarter

datetime_dayofweek

datetime_dayofweek[dt, tz]

Day of week (a number, e.g., 5 for Friday)

Example:

def dt = parse_datetime["2014-01-31 01:00 +03:00", "YYYY-mm-dd HH:MM zzzz"]
datetime_dayofweek[dt, "+03:00"] = 5
datetime_dayofweek[dt, "UTC"] = 4

Definition

@inline
def datetime_dayofweek = rel_primitive_datetime_dayofweek

datetime_dayofweekofmonth

datetime_dayofweekofmonth[dt, tz]

Day of week of month, as an integer (e.g. 2 for second Friday of the month, 1 for the first Tuesday).

Example:

def t = parse_datetime["2014-01-31 01:00 +03:00", "YYYY-mm-dd HH:MM zzzz"]
datetime_dayofweekofmonth[t, "+03:00"] = 5

Definition

@inline
def datetime_dayofweekofmonth = rel_primitive_datetime_dayofweekofmonth

datetime_dayofyear

datetime_dayofyear[dt, tz]

Day of year

Examples:

def dt = parse_datetime["2014-03-31 01:00 +03:00", "YYYY-mm-dd HH:MM zzzz"]
datetime_dayofyear[dt, "+03:00"] = 90
datetime_dayofyear[dt, "+00:00"] = 89

Definition

@inline
def datetime_dayofyear = rel_primitive_datetime_dayofyear

datetime_daysinmonth

datetime_daysinmonth[dt, tz]

The number of days in a datetime’s month, adjusting for timezone tz.

Examples:

def t = parse_datetime["2014-02-28 23:00 +00:00", "YYYY-mm-dd HH:MM zzzz"]
datetime_daysinmonth[t, "+03:00"] = 31
datetime_daysinmonth[t, "-03:00"] = 28

Definition

@inline
def datetime_daysinmonth = rel_primitive_datetime_daysinmonth

datetime_hour

datetime_hour[dt, tz]

Hour of a DateTime. Note that DateTime properties require a timezone string tz.

Definition

@inline
def datetime_hour = rel_primitive_datetime_hour

datetime_hour_UTC

datetime_hour_UTC[dt]

Hour assuming datetime is in UTC.

Definition

@inline
def datetime_hour_UTC[d] = rel_primitive_datetime_hour[d, "UTC"]

datetime_isleapyear

datetime_isleapyear[dt, tz]

Is it a leap year?

Examples:

def dt = parse_datetime["2014-01-31 01:00 +03:00", "YYYY-mm-dd HH:MM zzzz"]
def output = datetime_isleapyear[dt, "+03:00"]
false

def dtleap = parse_datetime["2016-01-31 01:00 +03:00", "YYYY-mm-dd HH:MM zzzz"]
def output = datetime_isleapyear[dtleap, "+00:00"]
true

Definition

@inline
def datetime_isleapyear = rel_primitive_datetime_isleapyear

datetime_minute

datetime_minute[dt, tz]

Minute of a DateTime. Note that DateTime properties require a timezone string tz.

Definition

@inline
def datetime_minute = rel_primitive_datetime_minute

datetime_minute_UTC

datetime_minute_UTC[dt]

Minute assuming datetime is in UTC.

Definition

@inline
def datetime_minute_UTC[d] = rel_primitive_datetime_minute[d, "UTC"]

datetime_month

datetime_month[dt, tz]

Month of a DateTime, as an integer between 1 and 12.

Note that DateTime properties require a timezone tz.

Examples:

def dt = parse_datetime["2021-01-01 01:00:00", "Y-m-d H:M:S"]
datetime_month[dt, "Europe/Berlin"] = 1
datetime_month[dt, "America/New_York"] = 12
datetime_month[dt, "-03:00"] = 12

Definition

@inline
def datetime_month = rel_primitive_datetime_month

datetime_month_UTC

datetime_month_UTC[dt]

Month assuming datetime is in UTC.

Definition

@inline
def datetime_month_UTC[d] = rel_primitive_datetime_month[d, "UTC"]

datetime_monthname

datetime_monthname[dt, tz]

The month name

Example:

def t = parse_datetime["2014-01-31 01:00 +03:00", "YYYY-mm-dd HH:MM zzzz"]
datetime_monthname[t, "+03:00"] = "January"

Definition

@inline
def datetime_monthname = rel_primitive_datetime_monthname

datetime_now

datetime_now

The current UTC time (the start time of the transaction) as a DateTime value.

Example:

def output = datetime_now

Definition

@inline
def datetime_now = rel_primitive_transaction_edb[:txn_start_time]

datetime_quarterofyear

datetime_quarterofyear[dt, tz]

Quarter of the year for datetime dt, as a number between 1 and 4.

Example:

def t = parse_datetime["2014-01-31 01:00 +03:00", "YYYY-mm-dd HH:MM zzzz"]
datetime_quarterofyear[t, "+03:00"] = 1

Definition

@inline
def datetime_quarterofyear = rel_primitive_datetime_quarterofyear

datetime_second

datetime_second[dt]

Second of a DateTime. Unlike other DateTime functions, datetime_second does not require a timezone argument.

Example:

datetime_second[unix_epoch] = 0

Definition

@inline
def datetime_second = rel_primitive_datetime_second

datetime_subtract

datetime_subtract[dt, period]
dt - period

Subtract a Period from a DateTime, giving another DateTime.

Example:

def dt = parse_datetime["2021-01-01 01:00:00", "Y-m-d H:M:S"]
def output = datetime_subtract[dt, Hour[1000]]
Dates.DateTime("2020-11-20T09:00:00")

Definition

@inline
def datetime_subtract = rel_primitive_subtract_datetime_period

datetime_to_nanoseconds

datetime_to_nanoseconds[dt]

Convert datetime to nanoseconds since the epoch. Assumes dt is in UTC.

Definition

@inline
def datetime_to_nanoseconds(datetime, v) =
    rel_primitive_datetimes_period_nanoseconds(unix_epoch, datetime, p) and
    rel_primitive_period_nanoseconds(p, v)
    from p

datetime_week

datetime_week[dt, tz]

Week of the year for a DateTime dt, as an integer between 0 and 53 (see date_week).

Note that DateTime properties require a timezone tz.

Definition

@inline
def datetime_week = rel_primitive_datetime_week

datetime_week_UTC

datetime_week_UTC[dt]

Week assuming datetime is in UTC.

Definition

@inline
def datetime_week_UTC[d] = rel_primitive_datetime_week[d, "UTC"]

datetime_year

datetime_year[dt, tz]

Year of a DateTime dt, as an Int. Note that DateTime properties require a timezone tz.

Examples:

def dt = parse_datetime["2021-01-01 01:00:00", "Y-m-d H:M:S"]
datetime_year[dt, "Europe/Berlin"] = 2021
datetime_year[dt, "America/New_York"] = 2020
datetime_year[dt, "-03:00"] = 2020

Definition

@inline
def datetime_year = rel_primitive_datetime_year

datetime_year_UTC

datetime_year_UTC[dt]

Year assuming datetime is in UTC.

Definition

@inline
def datetime_year_UTC[d] = rel_primitive_datetime_year[d, "UTC"]

datetimes_period_milliseconds

datetimes_period_milliseconds[dt1, dt2]

The difference between two datetimes, dt2 - dt1, as a Milliseconds data type.

In other words, equal to the milliseconds time period between dt1 and dt2 –- which when added to dt1 gives dt2.

Example:

def output = datetimes_period_milliseconds[
    unix_epoch,
    parse_datetime["2021-03-19 11:00:40", "YYYY-mm-dd HH:MM:SS"]
]

ic {is_Millisecond(output)}

Definition

@inline
def datetimes_period_milliseconds = rel_primitive_datetimes_period_milliseconds

Day

Day[n]

A period that spans n days. The period is given in units of Day.

Example:

datetime_add[unix_epoch, Day[10]]

Definition

@inline
def Day = rel_primitive_day

decimal

decimal[n, digits, v]

The n-bit decimal value with digits precision from number value v.

Definition

@inline
def decimal = rel_primitive_decimal

decimal_bit_length

decimal_bit_length[decimal]

The bit length of a fixed-point decimal number.

Example:

def g = parse_decimal[64, 2, "3.14"]
decimal_bit_length[g] = 64

Definition

@inline
def decimal_bit_length(x, y) = rel_primitive_decimal_type(x, y, _)

decimal_int_convert

decimal_int_convert[x]

Conversion from n-bit fixed-point decimal with d precision to n-bit int.

If the argument is not equivalent to an int, decimal_int_convert returns false.

Example:

decimal_int_convert[decimal[32, 2, 3.00]] = 3
decimal_int_convert[decimal[32, 2, 3.20]] = false

Definition

@inline
def decimal_int_convert = rel_primitive_decimal_int_convert

decimal_precision

decimal_precision[decimal]

The precision of a fixed-point decimal number.

Example:

def g = parse_decimal[64, 2, "3.14"]
decimal_precision[g] = 2

Definition

@inline
def decimal_precision(x, y) = rel_primitive_decimal_type(x, _, y)

decode_base16

decode_base16[encoded_str]

Decodes the base-16 encoded string to string, as per rfc4648 specifications.

Encoding/decoding specifications are found at https://datatracker.ietf.org/doc/html/rfc4648.

Example:

    def output = decode_base16["48656C6C6F21"]
    "Hello!"
    def output = decode_base16["E4BDA0E5A5BD"]
    "你好"

Definition

@inline
def decode_base16 = rel_primitive_base16_decode

decode_base32

decode_base32[encoded_str]

Decodes the base-32 encoded string to string, as per rfc4648 specifications.

Encoding/decoding specifications are found at https://datatracker.ietf.org/doc/html/rfc4648.

Example:

    def output = decode_base32["JBSWY3DPEE======"]
    "Hello!"
    def output = decode_base32["4S62BZNFXU======"]
    "你好"

Definition

@inline
def decode_base32 = rel_primitive_base32_decode

decode_base64

decode_base64[encoded_str]

Decodes the base-64 encoded string to string, as per rfc4648 specifications.

Encoding/decoding specifications are found at https://datatracker.ietf.org/doc/html/rfc4648.

Example:

    def output = decode_base64["SGVsbG8h"]
    "Hello!"
    def output = decode_base64["5L2g5aW9"]
    "你好"

Definition

@inline
def decode_base64 = rel_primitive_base64_decode

default_value

default_value[D, F, c]

Make function or relation F total for domain D by defaulting to c.

The arity of F must be the arity of D + 1.

Example:

def dom = {1;2;3;4}
def f = {(1, 321); (3, 123)}
def output = default_value[dom, f, 42]
(1, 321)
(2, 42)
(3, 123)
(4, 42)

See also: left_override (<++) and right_override (++>).

Definition

@inline
def default_value[D, F, c](k..., v) =
    F(k..., v) or (D(k...) and not F(k..., _) and v = c)

deg2rad

deg2rad[x]

Convert degrees to radians.

Definition

@inline
def deg2rad[x] = x * pi_float64 / 180.0

denominator

denominator[x]

Denominator of a rational-like number.

Examples:

denominator[rational[64, 1, 3]] = 3
denominator[rational[64, 1, 100]] = 100
denominator[rational[64, 1, 100] + rational[64, 1, 3]] = 300
denominator[parse_decimal[64, 2, "3.14"]] = 100
denominator[parse_decimal[64, 5, "3.14159"]] = 10000
denominator[5] = 1

Definition

@inline
def denominator = rel_primitive_denominator

describe

describe[R]

A summary of statistics for a dataset R. R should be a relation given in the form R(:feature_name, keys..., value). For example, data loaded using 0.

For each feature in the dataset, describe computes a set of statistics depending on the data type of that feature.

For numerical data, describe computes the count, mean, standard deviation, minimum, 25th, 50th, 75th percentiles, and maximum for that feature.

For non-numerical data, describe computes the count, unique count, mode, and mode frequency. It will also compute maximum and minimum values if the data is sortable.

If a dataset has multiple data types, it will print the numerical stats for the numerical subset, and non-numerical stats for the non-numerical subset of the data.

For example, given a dataset R in the form (produced by the CSV loader):

R = {
(:date, FilePos(29), 2020-01-01);
(:date, FilePos(60), 2020-02-02);
(:date, FilePos(91), 2020-03-03);
(:date, FilePos(127), 2020-04-04);
(:price, FilePos(29), 12.5);
(:price, FilePos(60), 14.25);
(:price, FilePos(91), 11.0);
(:price, FilePos(127), 12.25);
(:quantity, FilePos(29), 2);
(:quantity, FilePos(60), 4);
(:quantity, FilePos(91), 4);
(:quantity, FilePos(127), 3);
(:cocktail, FilePos(29), "martini");
(:cocktail, FilePos(60), "sazerac");
(:cocktail, FilePos(91), "cosmopolitan");
(:cocktail, FilePos(127), "bellini");
}

describe will compute a summary:

Example:

def d = describe[R]

d[:date]
(:mode, Dates.Date("2020-01-01"))
(:count, 4)
(:mode_freq, 1)
(:min, Dates.Date("2020-01-01"))
(:max, Dates.Date("2020-04-04"))
(:unique, 4)

d[:quantity]
(:max, 4)
(:count, 4)
(:std, 0.9574271077563381)
(:min, 2)
(:percentile75, 4.0)
(:percentile25, 2.75)
(:percentile50, 3.5)
(:mean, 3.25)

A describe_full method is also provided, which does not aggregate over columns, and instead computes the statistics for the whole dataset. However, describe_full may cause errors if any unsortable data (e.g., symbols) exists in the relation.

For example, given the relation R defined above:

Example:

describe_full[R]
(:min, 32)
(:unique, 3)
(:mode, "a")
(:std, 49.47186262580647)
(:percentile25, 89.2175)
(:min, 3.27)
(:percentile50, 53.6)
(:mean, 16.75)
(:mean, 18.8675)
(:count, 7)
(:max, 72.2)
(:max, 35)
(:mode_freq, 1)

Definition

@inline
def describe[R] = column: describe_full[R[column]]


// Stats for all types
@inline def describe_full[R, :count] = count[R]
@inline def describe_full[R, :min] = min[R]
@inline def describe_full[R, :max] = max[R]

// Stats for non-numeric types
@inline def describe_full[R, :unique] = count[last[R :> (x: not Number(x))]]
@inline def describe_full[R, :mode] = mode[R :> (x: not Number(x))]
@inline def describe_full[R, :mode_freq] = max[frequency[R :> (x: not Number(x))]]

// Stats for numeric types
@inline def describe_full[R, :mean] = mean[R :> Number]
@inline def describe_full[R, :std] = sample_stddev[R :> Number]
@inline def describe_full[R, :percentile25] = percentile[(R :> Number), 25]
@inline def describe_full[R, :percentile50] = median[R :> Number]
@inline def describe_full[R, :percentile75] = percentile[(R :> Number), 75]

diff

diff[R, S]

Set difference (complement): removes the tuples of S from R, if present.

Example:

def output = diff[{1;2;3;4} , {1;3} ]
2
4

Definition

@inline
def diff[R, S](x...) = R(x...) and not S(x...)

disjoint

disjoint(R, S)

R and S are disjoint if they have no fact in common.

Definition

@inline
def disjoint(R, S) = empty(R ∩ S)

divide

divide[x, y]
x / y

Division.

divide[x, y] is defined when either the types of x and y are the same, either operand is of type SignedInt[64] or Floating[64], one operand is a rational type and the other is a signed integer type, or one operand is a decimal type and the other is an integer type (regardless of signedness). The return type is:

  • the type of x when the types of both operands are the same float, rational, or decimal type.
  • Floating[64] when the types of both operands are integer types.
  • Floating[64] when either the type of x or the type of y is Floating[64].
  • Rational[n], where n is the greater of the number of bits in the types of x and y, when one operand is a rational type and the other is a signed integer type.
  • The type of x if x is a float type and y is of type SignedInt[64], and vice versa.
  • The type of x if x is a decimal type and y is some integer type, and vice versa.

Examples:

3.4 / 2 = 1.7
divide[6, 3] = 2.0
equal(divide(6, 3, 2.0), true)
equal(divide(6, 3, 2), false)

Definition

@inline
def divide[x, y] = rel_primitive_divide[x, y]

@inline
def (/)(x, y, z) = divide(x, y, z)

docstring

docstring[:R]

The docstring of a relation as a string.

Example:

doc"The number 0." def zero = 0
def output = docstring[:zero]
"The number 0."

Definition

@inline
def docstring = rel_primitive_docstring

domain

domain[F]

The domain of a scalar function F. The domain represents all but the last argument of F.

Example:

def output = domain[{(1, 'a', 1); (2, 'b', 4); (3, 'c', 9)}]
(1, 'a')
(2, 'b')
(3, 'c')

Definition

@inline
def domain[F](k...) = F(k..., _)

dot_join

dot_join[R, S]
R . S

Compose

Definition

@inline
def dot_join[R,S](x..., y...) = exists(t: R(x..., t) and S(t, y...))

@inline
def (.) = dot_join

double_metaphone

double_metaphone[string_value]
double_metaphone_alt[string_value]

Double Metaphone is an improvement of metaphone that considers spelling particularities in a number of other languages. double_metaphone and double_metaphone_alt returns an encoding of the phonetic representation of string_value. For many strings, double_metaphone and double_metaphone_alt returns the very same encoding. However, it may differs based on the spelling in a non-English language.

string_value could be any arbitrary long string

Examples:

@inline def dm[string] = double_metaphone[string] ; double_metaphone_alt[string]

def output = dm["Smith"]
"sm0"

def output = dm["Smythe"]
"sm0"
"xmt"

def output = dm["Christina"]
"krstn"

def output = dm["Cristine"]
"krstn"

def output = dm["I like Music"]
"ilkm"

def output = dm["i loak Museek"]
"ilkm"

def output = dm["RelationalAI"]
"rlxnl"

def output = dm["rellationalleAI"]
"rlxnl"

Definition

@inline
def double_metaphone = rel_primitive_double_metaphone

@inline
def double_metaphone_alt = rel_primitive_double_metaphone_alt

empty

empty(R)

Examples:

  • empty(1) is false
  • empty(true) is false
  • empty(false) is true

Definition

@inline
def empty(R) = not exists(x...: R(x...))

encode_base16

encode_base16[input_str]

Encodes the given string(input_str) to base16-encoded string, as per rfc4648 specifications.

Encoding/decoding specifications are found at https://datatracker.ietf.org/doc/html/rfc4648.

Examples:

    def output = encode_base16["Hello!"]
    "48656C6C6F21"
    def output = encode_base16["你好"]
    "E4BDA0E5A5BD"

Definition

@inline
def encode_base16 = rel_primitive_base16_encode

encode_base32

encode_base32[input_str]

Encodes the given string(input_str) to base32-encoded string, as per rfc4648 specifications.

Encoding/decoding specifications are found at https://datatracker.ietf.org/doc/html/rfc4648.

Example:

    def output = encode_base32["Hello!"]
    "JBSWY3DPEE======"
    def output = encode_base32["你好"]
    "4S62BZNFXU======"

Definition

@inline
def encode_base32 = rel_primitive_base32_encode

encode_base64

encode_base64[input_str]

Encodes the given string(input_str) to base64-encoded string, as per rfc4648 specifications.

Encoding/decoding specifications are found at https://datatracker.ietf.org/doc/html/rfc4648.

Example:

    def output = encode_base64["Hello!"]
    "SGVsbG8h"
    def output = encode_base64["你好"]
    "5L2g5aW9"

Definition

@inline
def encode_base64 = rel_primitive_base64_encode

Entity

Entity(x)

Holds if x is an entity.

Example:

Integrity constraint that specifies that a module always contains an entity:

def myentity:f = 1

ic my_entity_ic(x) { myentity(x) implies Entity(x) }

Definition

@inline
def Entity = rel_primitive_Entity

epoch_milliseconds

epoch_milliseconds[dt]

Milliseconds since Epoch time, as a value of type Int64.

Definition

@inline
def epoch_milliseconds =
    rel_primitive_epoch_milliseconds

eq

eq(x, y)
x = y

Equality between scalar (single) values, such as integers, strings, symbols. (For equality between relations, use equal.)

Examples:

  • 1 = 1 is true.
  • :a = :a is true
  • "a" = "a" is true
  • 2 = 2.0 is false
  • 1 = "a" is false

Definition

@inline
def eq(x, y) = rel_primitive_eq(x, y)

@inline
def (=)(x, y) = eq(x, y)

equal

equal(R, S)

Relational equality. Note that eq and = should be used only for scalar values, while equal should be used to check that two relations are the same.

Definition

@inline
def equal(R, S) = forall(x... where R(x...): S(x...)) and forall(x... where S(x...): R(x...))

erf

erf[x]

The error function of x.

External link: https://en.wikipedia.org/wiki/Error_function

Examples:

erf[2] = 0.9953222650189527
erf[-0.5] = -0.5204998778130465

Definition

@inline
def erf[x] = rel_primitive_error_function[x]

erfinv

erfinv[x]

The inverse error function of x.

External link: https://en.wikipedia.org/wiki/Error_function#Inverse_functions

Examples:

erfinv[0.1] = 0.08885599049425769
erfinv[-0.5] = -0.4769362762044699
erfinv[erf[1]] = 1.0

Definition

@inline
def erfinv[x] = rel_primitive_error_function_inverse[x]

escape_regex_metachars

escape_regex_metachars(string, escaped_string)

Escape the necessary regular expression metacharacters in string such that escaped_string can be used as a regular expression and have none of its characters interpreted as metacharacters.

Example:

def ere = escape_regex_metachars["."]
regex_match_all(ere, "abc.123")

will only give

{(4, ".")}

since . is escaped and is not treated as a metacharacter.

Definition

@inline
def escape_regex_metachars[s in String] =
    string_replace_multiple[
        string_replace[s, "\\", "\\\\"],
        {mc:
            concat["\\", mc],
            {"."; "^"; "$"; "*"; "+"; "?"; "("; ")"; "["; "|"; "{"}(mc)
        }
    ]

export_csv

export_csv[R]

This is the main entry point to specify relations that should be exported to CSV. The payload relation R is expected to be a configuration relation mapping keys to relations.

Required keywords are:

  • :path: A string specifying the location of the file that is to be created.

Optional keywords:

  • :data: A set of relations mapping a file position, which can be a key of arbitrary length or type, to the corresponding value. This is the data that will be exported. We expect the data relation to be of form: data(:COLUMN_NAME, pos val)
  • :integration: A relation containing storage integration configuration.
  • :schema: A relation containing schema information.
  • :syntax: A relation containing syntax configuration.

Example using default CSV schema and syntax and no storage integration.

def csv_data(:ORDER, pos, v) = ((1,1); (2,2); (3,3))(pos, v)
def csv_data(:LINEITEM, pos, v) = ((1,100); (2,101); (3,102))(pos, v)
def csv_data(:QUANTITY, pos, v) = ((1,2); (2,15); (3,42))(pos, v)

def export = export_csv[(:path, "/path/to/local/file.csv"); (:data, csv_data)]

Example using a custom CSV schema.

def csv_syntax[:delim] = ';'
def csv_syntax[:quotechar] = '_'

def export = export_csv[(:path, "/path/to/local/file.csv");
                        (:data, csv_data);
                        (:syntax, csv_syntax)]

Example using an Azure storage integration using a SAS token:

def integration[:provider] = "azure"
def integration[:credentials, :azure_sas_token] = "<azure_sas_token>"

def export = export_csv[(:path, "azure://<account_name>.blob.core.windows.net/container/file.csv");
                        (:data, csv_data);
                        (:integration, integration)]

Example with compound keys:

def csv_data(:ORDER, pos..., v) = (1, 2, 100; 2, 1, 40)(pos..., v)
def csv_data(:QUANTITY, pos..., v) = (1, 3, 10; 2, 3, 11; 3, 1, 12; 3, 2 ,13)(pos..., v)

def export = export_csv[(:data, csv_data; :path, "/path/to/data.csv")]

Definition

@inline
def export_csv[R](ys..., xs...) = (:envelope, (:payload, (
        (:data, ((key..., col, val): R[:data](col, key..., val))) ; (x, y...: R(x, y...), (x != :data))
    )(xs...); (:content_type, "text/csv")(xs...)))(ys...)

@inline
def export_csv_row_wise[R](ys..., xs...) = (:envelope, (:payload, R(xs...); (:content_type, "text/csv")(xs...)))(ys...)

export_json

export_json[R]

This is the main entry point to specify relations that should be exported to JSON. The payload relation R is expected to be a configuration relation mapping keys to relations.

Required keywords are:

  • :path: A string specifying the location of the file that is to be created.

Optional keywords:

  • :data: The relation(s) that should be exported.
  • :integration: A relation containing storage integration configuration.
  • :indent: Number of spaces to indent the resulting document. If not present, we’ll write a compact form.

Example for a simple JSON object

def json[:author] = "David Foster Wallace"
def json[:title, :name] = "Infinite Jest"
def json[:title, :isbn] =  "978-0-316-92004-9"

def export = export_json[(:path, "/path/to/file.json");
                         (:data, json);
                         (:indent, 2)]

This results in the following JSON:

{
  "author": "David Foster Wallace",
  "title": {
    "name": "Infinite Jest",
    "isbn": "10: 0316921173"
  }
}

Since authors usually write more than one book, we’ll make the :title path an array of objects.

Examples:

def json[:author] = "David Foster Wallace"

def titles = (1, "Infinite Jest", "978-0-316-92004-9");
             (2, "The Pale King", "978-0-316-07423-0")

def json[:titles, :[], idx] = ((:name, name);
                               (:isbn, isbn)
                               from name, isbn where titles(idx, name, isbn))

def export = export_json[(:path, "/path/to/file.json");
                         (:data, json);
                         (:indent, 2)]

The :[] marker, i.e. array marker, needs to be present when an array is to be correctly produced.

This yields the following output:

{
  "author": "David Foster Wallace",
  "titles": [
    {
      "name": "Infinite Jest",
      "isbn": "978-0-316-92004-9"
    },
    {
      "name": "The Pale King",
      "isbn": "978-0-316-07423-0"
    }
  ]
}

Definition

@inline
def export_json[R](ys..., xs...) =
    (:envelope, (
        (:payload, R(xs...));
        (:content_type, "application/json")(xs...)
    ))(ys...)

factorial

factorial[x]

Factorial of x.

Defined for non-negative x. The result is promoted to at least 64-bits.

If x is up to 64-bits, factorial is defined for values up to 20 (inclusive). If x is Int128, factorial is defined for values up to 33 (inclusive). If x is UInt128, factorial is defined for values up to 34 (inclusive).

Definition

@inline
def factorial[x] = rel_primitive_factorial[x]

filepos

filepos[v]

Creates a FilePos representing position v in a file.

Definition

@inline
def filepos = rel_primitive_filepos

FilePos

FilePos(x)

Holds if the given value is a FilePos, which are types created when importing CSV files. FilePos types are used as keys when joining columns from the same row. Currently, users cannot directly create this data type themselves.

Example:

Integrity constraint that specifies that a relation called csv contains values with RelName, FilePos, and String.

def config:data = """
   item, category
   laptop, computer
   monitor, peripheral
   """

def csv = load_csv[config]
//FilePos automatically generated

def csv_ic {
    subset(csv, (RelName, FilePos, String))
}

Definition

@inline
def FilePos = rel_primitive_FilePos

// Temporary alias for backward compatibility
@inline
def delve_period_days = rel_primitive_period_days

// Temporary alias for backward compatibility
@inline
def delve_date_daysinmonth = rel_primitive_date_daysinmonth

filepos_value

filepos_value[fp]

Retrieves the numeric position represented by a FilePos.

Definition

@inline
def filepos_value = rel_primitive_filepos_value

first

first[R]

Projection to the first argument of R.

first supports heterogeneous relations, both in arity and types.

Examples:

def output = first[(1, 2, 3); (4, 5, 6)]
1
4
def output = first[(1, 2, 3); (4, 5); 6]
1
4
6
def output = first[("a", 1); (2, 3)]
2
"a"

Definition

@inline
def first[R](x)  = (y... : R(x, y...))

FixedDecimal

FixedDecimal(nbits, ndecimals, x)

Holds if x is of type FixedDecimal with a bit size of nbits and ndecimals decimal precision.

Examples:

Integrity constraint that tests whether x is a FixedDecimal with a bit size of 64 and 4 decimal precision:

def R = decimal[64, 4, pi_float64]

ic float_type_check(x in R) {
    FixedDecimal(64, 4, x)
}

Query to check whether x is a FixedDecimal with a bit size of 64 and 4 decimal precision.

def R = {decimal[64, 4, pi_float64]; decimal[64, 10, pi_float64]}

def output(x) = R(x) and FixedDecimal(64, 4, x)
//output> 3.1416

Definition

@inline
def FixedDecimal = rel_primitive_FixedDecimal

FixedDecimal_spread

FixedDecimal_spread[mode, R, val]

This definition uses the two spread strategies implemented in int_spread_by_even and int_spread_by_ratio but specialized to input value that is a FixedDecimal. The major difference here is that in the integer case, the smallest unit to spread was 1, whereas here, the smallest unit to spread is 10^(-d) where d is the number of digits used in the FixedDecimal value val. For example, if we are trying to spread the value 3.140, the smallest unit to spread will be 0.001. mode can be :even or :ratio.

Example:

def R = {("Atlanta", "Georgia", 50);
    ("Seattle", "Washington", 80);
    ("San Francisco", "California", 10)}

def output = FixedDecimal_spread[:even, R, decimal[64, 3, 3.14]]
("Atlanta", "Georgia", 1.047)
("San Francisco", "California", 1.047)
("Seattle", "Washington", 1.046)

Definition

@inline
def FixedDecimal_spread[mode, R, val] = v..., s:
    rel_primitive_decimal_type(val, decimal_bit_length[val], decimal_precision[val]) and
    den = denominator[val] and
    digits = floor_to_int[log10[den]] and
    valinput = numerator[val] and
    (
        (mode = :even and s = int_spread_by_even[R, valinput][v...] / den) or
        (mode = :ratio and s = int_spread_by_ratio[R, valinput][v...] / den)
    )
    from valinput, digits, den

float

float[n, v]

The n-bit floating point value from the number v. Example: float[64, 3.0].

Definition

@inline
def float = rel_primitive_float

float64

float64[v]

The 64-bit floating point value from the number v, which must be a float.

Definition

@inline
def float64 = float[64]

float_int_convert

float_int_convert[x]

Conversion from float to int. If the argument is not equivalent to an int, float_int_convert returns false. (See trunc_to_int, floor_to_int for general conversion.)

Example:

float_int_convert[3.0] = 3
float_int_convert[3.2] = false

Definition

@inline
def float_int_convert = rel_primitive_float_int_convert

Floating

Floating(nbits, x)

Holds if x is an nbits' floating point number.Float(x)is a shorthand that holds ifx` is a 64-bit float.

Example:

Integrity constraint that tests if x is a 32-bit float (will throw if x is not a 32-bit Float):

def R = float[32, 1.321]

ic float_type_check(x in R) {
    Floating(32, x)
}

Definition

@inline
def Floating = rel_primitive_Floating

floor

floor[x]

Round down to the nearest integer.

Examples:

floor[4.5] = 4.0
floor[-4.5] = -5.0
floor[4] = 4

Definition

@inline
def floor[x] = round[:ROUND_DOWN, x]

floor_divide

floor_divide[x, y]

Floor Division.

Largest integer less than or equal to x/y. Computes x/y rounded towards negative infinity.

Is false (empty) when y = 0 for Integer arguments.

Examples:

floor_divide[5, 2] = 2
floor_divide[-5, 2] = -3
floor_divide[5.1, 2] = 2.0
empty(floor_divide[5.1, 0])

Definition

@inline
def floor_divide[x, y] = rel_primitive_floor_divide[x, y]

floor_to_int

floor_to_int[x]

General float-to-int conversion, floor. Argument must be a float.

Examples:

floor_to_int[3.1] = 3
floor_to_int[-3.1] = -4
floor_to_int[3] : error

Definition

@inline
def floor_to_int[x] = float_int_convert[floor[x]]

format_date

format_date[d, format]

A string where the Date d is formatted according to format.

Example:

def d = parse_date["2018-06-12", "Y-m-d"]
def output = format_date[d, "Y-mm-d"]
"2018-06-12"

For details on the format parameter, see the Julia documentation for Dates.DateFormat.

Definition

@inline
def format_date = rel_primitive_format_date

format_datetime

format_datetime[dt, format, tz]

Format a DateTime dt, with timezone tz.

Example:

def format = "yyyy-mm-dd HH:MM ZZZ"
def dt = parse_datetime["2018-03-11 01:00 America/New_York", "Y-m-d H:M Z"]
def output = format_datetime[dt, format, "America/New_York"]
"2018-03-11 01:00 EST"

For details on the format parameter, see the Julia documentation for Dates.DateFormat.

Definition

@inline
def format_datetime = rel_primitive_format_datetime

formula_card_est

formula_card_est[R]

The cardinality estimate of a given relational abstraction R as estimated by the physical query optimizer. The estimate is represented as a Float. It is typically an upper bound on the actual cardinality and can suffer from a numeric overflow, e.g. if the formula R is a big cross-product of relations. To avoid this, use the log-version log_card_est[R]. See log_card_est for more details.

Examples:

def card = formula_card_est[R]
def card = formula_card_est[a, b, c: R(a, b) and S(b, c) and T(a, c)]

Definition

@inline
def formula_card_est[R] = 2.0 ^ rel_log_card_est[R]

frequency

frequency(R, elem, c)
frequency[R, elem]
frequency[R]

Finds the frequency c of elem in (the last argument of) the relation R.

Example:

def example = {(1, "a"); (2, "b"); (3, 123); (4, 12.5); (3, "b")}
def output = frequency[example, "b"]
2

def output = frequency[example]
("a", 1)
("b", 2)
(123, 1)
(12.5, 1)

Definition

@inline
def frequency[R, elem] = count[x...: R(x..., elem)]

function

function(R)

Given a relation R, true if R is a function (from all but the last elements, to the last), and false if not.

Examples:

def output = function[{(1, 2); (2, 5)}]
true
def output = function[{(1, 2); (1, 3)}]
false
def output = function[{(1,2,3) ; (1,3,4)}]
true

Definition

@inline
def function(R) =
    forall(k..., v1, v2 where R(k..., v1) and R(k..., v2): v1 = v2)

geometric_mean

geometric_mean[R]

The geometric mean of the values of a relation.

Example:

def output = geometric_mean[{('a',1); ('b',2); ('a',3); ('b',4); ('c',5); ('d',6}]
2.99379516552

Definition

@inline
def geometric_mean[R] = power[product[R], 1 / count[R]]

gt

gt(x, y)
x > y

Definition

@inline
def gt(x, y) = rel_primitive_gt(x, y)

@inline
def (>)(x, y) = gt(x, y)

gt_eq

gt_eq(x, y)
x  y
x >= y

Definition

@inline
def gt_eq(x, y) = rel_primitive_gt_eq(x, y)

@inline
def ()(x, y) = gt_eq(x, y)

@inline
def (>=)(x, y) = gt_eq(x, y)

harmonic_mean

harmonic_mean[R]

The harmonic mean of the values of a relation.

Example:

def output = harmonic_mean[{('a',1); ('b',2); ('a',3); ('b',4); ('c',5); ('d',6}]
2.44897959184

Definition

@inline
def harmonic_mean[R] = count[R] / sum[a..., v, i: R(a..., v) and i = 1 / v]

hash

hash[R]

see hash128

Definition

@inline
def hash = hash128

Hash

Hash(x)

Holds if x is a RelationalAI HashValue generated by hash128.

Examples:

Integrity constraint that tests whether x is Hash (will throw if x is not Hash):

def R = (9, "a")

def my_hash = hash128[R]
//three value relation with hash value as third value

ic hash_type_check{
    subset(my_hash, (Any, Any, Hash))
}

Defines Hash as schema for a relation:

def R = (9, "a")
def my_hash = hash128[R]
def hashed_relation(x in Hash) = my_hash(_, _, x)

def output = hashed_relation
//output> -107770920621774551289984725953057040743

Definition

@inline
def Hash = rel_primitive_HashValue

hash128

hash128[R]

Hash128 each of the tuples in a relation.

hash128 takes a relation R(xs...) and produces a relation R(xs..., i) where i is a HashValue index. The index indicates the hash128 of each tuple of R.

Example:

def output = hash128['a']
('a', HashValue(0xbd1defa3be4617e0))

def sample = {'a'; 'b'; 'c'}
def output = hash128[sample]
('a', HashValue(0xbd1defa3be4617e0))
('b', HashValue(0x2423a34d925fc9ed))
('c', HashValue(0xd2cffd2ea6c73a7))

Definition

@inline
def hash128 = rel_primitive_hash

hash_value_uint128_convert

hash_value_uint128_convert[v]

Convert a HashValue v to UInt128.

Definition

@inline
def hash_value_uint128_convert[v] = rel_primitive_hash_value_uint128_convert[v]

haversine

haversine[r, x1, y1, x2, y2]

The great circle distance of two points, i.e (x1,y1)(x1,y1) and (x2,y2)(x2,y2), on a sphere of radius rr, using the Haversine formula. The two points are specified by their latitude and longitude in radians.

Example:

def output = haversine[10, 0, 0, 0, pi_float64/2]
15.707963267948964

Definition

@inline
def haversine[r, x1, y1, x2, y2] =
    2 * r * asin[sqrt[sin[(x2 - x1)/2] ^ 2 + cos[x1] * cos[x2] * sin[(y2 - y1) / 2] ^ 2]]

Hour

Hour[n]

A period that spans n hours. The period is given in units of Hour.

Example:

datetime_add[unix_epoch, Hour[10]]

Definition

@inline
def Hour = rel_primitive_hour

iff

F iff G
F ≡ G
F ⇔ G

If and only if (boolean equivalence), for boolean (arity 0, true or false) arguments F and G.

Definition

@inline
def iff(F, G) = (F implies G) and (G implies F)

@inline
def (≡)(F, G) = F iff G

@inline
def (⇔)(F, G) = F iff G

int

int[n, v]

The n-bit signed integer value from the integer value v.

Definition

@inline
def int = rel_primitive_int

int128

int128[v]

The 128-bit signed integer value from the integer value v.

Definition

@inline
def int128 = int[128]

int64

int64[v]

The 64-bit signed integer value from the integer value v.

Definition

@inline
def int64 = int[64]

int_float_convert

int_float_convert[x]

Conversion from int to float. Argument must be an int.

Example:

int_float_convert[3] = 3.0
int_float_convert[3.2] : error

Definition

@inline
def int_float_convert = rel_primitive_int_float_convert

int_spread_by_even

int_spread_by_even[R, value]

Spreads value to the entries in the relation R evenly as whole units. If the value to spread = 50 and R has 5 elements, each element will get 10. In case of uneven spread (e.g. spread 51 to 5 elements), the remainder gets distributed as single units in order starting from the first element after the relation has been sorted lexicographically (for details see sort).

Empty if value is negative.

Example:

def R = {("Atlanta", "Georgia");
    ("Seattle", "Washington");
    ("San Francisco", "California")}

def output = int_spread_by_even[R, 32]
("Atlanta", "Georgia", 11)
("San Francisco", "California", 11)
("Seattle", "Washington", 10)

Definition

@inline
def int_spread_by_even[R, val] = v..., s:
    Int(val) and
    R(v...) and
    val >= 0 and
    c = count[R] and
    sort[R](i, v...) and
    (
        (i <= remainder[val, c] and s = trunc_divide[val, c] + 1) or
        (i > remainder[val, c] and s = trunc_divide[val, c])
    )
    from i, c

int_spread_by_ratio

int_spread_by_ratio[R, value]

Spreads value to the entries in the relation R proportionally to their weight (which is the last element in each tuple), as whole units. For example, if the relation is {("X",90); ("Y",10)}, "X" will get 90% of value and "Y" will get 10%. Often, this split will result in a decimal number, but we want to spread whole units. To resolve this issue, we first distribute the floor value of the decimal number computed. The remainder gets distributed as single units in order starting from the element that had the highest decimal value.

Empty if value is negative.

Example:

def R = {("Atlanta", "Georgia", 50);
    ("Seattle", "Washington", 80);
    ("San Francisco", "California", 10)}

def output = int_spread_by_ratio[R, 30]
("Atlanta", "Georgia", 11)
("San Francisco", "California", 2)
("Seattle", "Washington", 17)

Definition

@inline
def int_spread_by_ratio[R, val] = v..., s:
    Int(val) and
    R(v..., w) and
    val >= 0 and
    forall(t in last[R] : t > 0) and
    s = floor_to_int[val * (w / sum[R])] + offset and
    sort[(rr, vv...) : rr = remainder[R[vv...] * val, sum[R]]](i, r, v...) and
    c = val - sum[vv... : floor_to_int[val * (R[vv...] / sum[R])]] and
    (
        (offset = 1 and i >= count[R] - c + 1) or
        (offset = 0 and i < count[R] - c + 1)
    )
    from w, i, r, c, offset

intersect

intersect[R, S]
R ∩ S

Intersect two n-ary relations R and S

Definition

@inline
def intersect[R, S](x...) = R(x...) and S(x...)

@inline
def (∩) = intersect

is_Day

is_Day(x)

Holds if x is a Day period.

Definition

@inline
def is_Day = rel_primitive_Day

is_Hour

is_Hour(x)

Holds if x is a Hour period.

Definition

@inline
def is_Hour = rel_primitive_Hour

is_Microsecond

is_Microsecond(x)

Holds if x is a Microsecond period.

Definition

@inline
def is_Microsecond = rel_primitive_Microsecond

is_Millisecond

is_Millisecond(x)

Holds if x is a Millisecond period.

Definition

@inline
def is_Millisecond = rel_primitive_Millisecond

is_Minute

is_Minute(x)

Holds if x is a Minute period.

Definition

@inline
def is_Minute = rel_primitive_Minute

is_Month

is_Month(x)

Holds if x is a Month period.

Definition

@inline
def is_Month = rel_primitive_Month

is_Nanosecond

is_Nanosecond(x)

Holds if x is a Nanosecond period.

Definition

@inline
def is_Nanosecond = rel_primitive_Nanosecond

is_Second

is_Second(x)

Holds if x is a Second period.

Definition

@inline
def is_Second = rel_primitive_Second

is_Week

is_Week(x)

Holds if x is a Week period.

Definition

@inline
def is_Week = rel_primitive_Week

is_Year

is_Year(x)

Holds if x is a Year period.

Definition

@inline
def is_Year = rel_primitive_Year

jacobian

jacobian[S, R]
jacobian(S, R, J)

Compute the Jacobian (e.g. the derivative or gradient) of relation S w.r.t. relation R.

jacobian computes the derivatives of one scalar- or vector-valued relation (R) with respect to a second scalar- or vector-valued relation (S). It returns a matrix of all of them (J) using (relational) automatic differentiation techniques.

Both relations (S and R) must be key value pairs. This is true even of scalars such as x[1], which cannot be rendered as x.

Examples

Derivative of a scalar-valued relation of one scalar variable:

def y[1] = 1.0
def z[1] = 3.0 * y[1]^2

def output = jacobian[z, y]
//output> 1,1,6.0

Derivative of a more complicated scalar-valued relation of one scalar variable:

def x[1] = 1.0
def y[1] = 3.0 * x[1]^2
def z[1] = y[1] / 2

def output = jacobian[z, x]
//output> 1,1,3.0

Gradient of a vector-valued relation with respect to a scalar input:

def y[1] = 2.0
def z[1] = 3.0 * y[1]^2
def w[1] = z[1]^2
def w[2] = y[1]

def output = jacobian[w, y]

//output> 1,1,288.0
//        2,2,1.0

Jacobian of a vector-valued relation with respect to a vector input:

def y[1] = 1.0
def y[2] = 4.0

def z[1] = 2*y[1] + 3*y[2]
def z[2] =   y[1] - y[2]

def output = jacobian[z, y]
//output> 1,1,2.0
//        1,2,3.0
//        2,1,1.0
//        2,2,-1.0

Definition

@inline
def jacobian[S, R] = rel_primitive_jacobian[S, R]

json_string

    json_string[json_relation]

The string representation of a relation that encodes JSON data.

Example:

def json_relation[:name] = "Amira"
def json_relation[:age] = 32
def json_relation[:height] = missing
def json_relation[:pets, :[], 1] = "dog"
def json_relation[:pets, :[], 2] = "rabbit"

def result = json_string[json_relation]

results in the following JSON:

{
  "name": "Amira",
  "age": 32,
  "height": null,
  "pets": [
    "dog",
    "rabbit"
  ]
}

Definition

@inline
def json_string = rel_primitive_json_string

last

last[R]

Projection to the last argument of the relation R.

last supports heterogeneous relations, both in arity and types.

Examples:

def output = last[(1, 2, 3); (4, 5, 6)]
3
6
def output = last[1; 2; 3]
1
2
3
def output = last[(1, 2); (3, "abc")]
2
"abc"
def output = last[(1, 2, 3); (4, 5)]
3
5

Definition

@inline
def last[R](y)   = (x... : R(x..., y))

left_override

left_override[R, S]
R <++ S

The (left) override operator is usually applied to relations of tuples (k..., v) with a functional dependency from the initial (key) arguments (k...) to the last (value) argument v.

R <++ S contains all the tuples of R, plus all the tuples in S whose key is not in R. Often, S specifies default values for keys that are missing in R. In the result, a value in R overrides a value in S.

There are two equivalent ways to think about R <++ S: (1) S is providing default values for keys that are missing in R. (2) The entries in R are overriding the entries in S.

ExampleValue
2 <++ 32
1.5 <++ 31.5
(1,2) <++ (1,3)(1,2)
(3,4) <++ (1,2); (3,5)(1,2); (3,4)
(3,"abc") <++ (1,2); (3,5)(1,2); (3,"abc")
(1,2); (3,5) <++ (3,4); (6,7)(1,2); (3,5); (6,7)

Override can be applied to heterogeneous relations, notably JSON-like relations. The following examples show <++ operators applied to JSON inputs.

LeftRightValue
{"a": 1, "b": 3}{"a": 2 }{"a": 1, "b": 3}
{"a": 2 }{"a": 1, "b": 3}{"a": 2, "b": 3}
{"a": 1, "b": 2}{"c": 3}{"a": 1, "b": 2, "c": 3}

See also: default_value[D, F, c].

Definition

@inline
def left_override[R, S](xs...) = R(xs...)

@inline
def left_override[R, S](xs..., v) = S(xs..., v) and not R(xs..., _)

@inline
def (<++) = left_override

levenshtein

levenshtein[string1, string2]

Calculate the Levenshtein distance between two strings.

Example:

levenshtein["kitten", "sitting"]
3

Definition

@inline
def levenshtein = rel_primitive_levenshtein

like_match

like_match(like_string, string)

Match string with a SQL “LIKE” pattern; case-sensitive.

The ’%’ character is a wildcard, and is not escapable.

Example:

like_match("%PROMO%", "this is a PROMOtion")
()

Definition

@inline
def like_match = rel_primitive_like_match

lined_csv

lined_csv[R]

Given an already loaded CSV relation R, lined_csv[R] is a copy of the relation, but with the file position field replaced with a line number. Only lines containing data, including errors, are included in the numbering. Headers and empty lines are not counted. Thus, line number 1 is the first non-empty, non-header line of the CSV relation.

Example:

def csv = load_csv[config]
def csv_with_lines = lined_csv[csv]

Definition

@inline
def lined_csv[R](col, line, rest...) = sort[second[R]](line, pos) and R(col, pos, rest...) from pos

load_binary

load_binary[URI_string]
load_binary[R]

This is the main in-language API for loading files as binary string. To load a file, either provide the string URI where the data resides, or a configuration relation R that describes the data load.

The relation R, which must be an EDB or an IDB but not a formula, can configure the data load with the following options:

Required:

  • :path: The path to the file, e.g. "s3://storage.blob.core.net/container/image.jpeg".

Optional:

  • :integration: A relation containing storage integration configuration.

Example:


def config[:path] = "azure://<account_name>.blob.core.windows.net/container/file.bin"
def config[:integration, :provider] = "azure"
def config[:integration, :credentials, :azure_sas_token] = "<azure_sas_token>"

def output = load_binary[config]

Definition

@inline
def load_binary = rel_primitive_load_binary

load_csv

load_csv[URI_string]
load_csv[R]

This is the main in-language API for loading CSV data. To load a CSV, either provide the string URI where the data resides, or a configuration relation R that describes the data load.

In the first case, all columns are imported with type String.

The relation R, which must be an EDB or an IDB but not a formula, can configure the data load with the following options:

Required: Either

  • :path: The path to the CSV file, e.g. "azure://storage.blob.core.net/container/file.csv"

Or

  • :data: A CSV formatted string value

Optional:

  • :schema: A relation containing schema information.
  • :syntax: A relation containing syntax configuration.
  • :integration: A relation containing storage integration configuration.

The resulting relation is of the form (column_name, file_position, value). The file_position uniquely identifies each row in the CSV.

Examples:

Loading from literal string value:

def config[:data] = """
    A,B,C
    1,2,3
    """

def my_data = load_csv[config]

Loading a CSV file using a constant path:

def y = load_csv["s3://mybucket/path/test.csv"]
def x = y[:A, _]

In the example above, all values for column :A (“A” in the original file) are stored in x.

This can also be used to only load specific columns: load_csv[path](:A, xs...).

Loading a CSV file using a configuration relation config:

def config[:path] = "/path/to/file.csv"
def config[:syntax, :delim] = '_'
def config[:syntax, :header] = (1, :A); (2, :B); (3, :C)
def config[:schema, :A] = "int"
def config[:schema, :B] = "string"
def config[:schema, :C] = "decimal(64,2)"

def csv = load_csv[config]

Lastly, load_csv also supports error handling. If the CSV can’t be read (e.g. if it is badly formatted), parsing errors are stored in the :load_errors “column”, which can be accessed the same way as any other column. It has the format (:load_errors, file_position, error_column, raw_line). Note : Since the arities of regular columns and the error relations differ, use the splat operator (”…”) to make your program as resilient as possible (cf. examples above).

Definition

@inline
def load_csv = rel_primitive_load_csv

load_csv_row_wise

load_csv_row_wise[R]

Load a CSV file into a “row-wise” relation, with the position as the first argument, and the column name second. Included for backward compatibility, we now recommend using load_csv instead, which puts the column name first.

Definition

@inline
def load_csv_row_wise[R][pos, col] = rel_primitive_load_csv[R][col, pos]

load_json

load_json[filename_or_url]
load_json[R]

The relation R, which currently needs to either be an EDB or an IDB but no formula, can be used to configure the data load with the following options:

Required: Either

  • :path: The path to the JSON file, e.g. "azure://storage.blob.core.net/container/file.json"

Or

  • :data: A JSON formatted string value

Optional:

  • :integration: A relation containing storage integration configuration.

Examples:

Loading literal JSON values:

def data = "{\"name\": \"Anton\", \"age\": 56}"
def config[:data] = data
def json = load_json[config]

Loading a JSON file using a constant path:

def y = load_json["test.json"]

Loading a JSON file using a configuration relation:

def config[:path] = "/path/to/file.json"
def json = load_json[config]

Loading a JSON file from behind a private Azure container using a SAS token:

def config[:path] = "azure://<account_name>.blob.core.windows.net/container/file.json"
def config[:integration, :provider] = "azure"
def config[:integration, :credentials, :azure_sas_token] = "<azure_sas_token>"
def json = load_json[config]

Definition

@inline
def load_json = rel_primitive_load_json

load_jsonlines

load_jsonlines[filename_or_url]
load_jsonlines[R]

Parse and load a JSONLines file. JSONLines is a file format that considers each line of the text file as a separate JSON object.

Example:

def data = "{\"name\":\"Anton\",\"age\":56}\n{\"name\": \"Alex\", \"age\":44}"
def config[:data] = data
def output = load_jsonlines[config]

ic {equal(output[1,:name], "Anton")}
ic {equal(output[1,:age], 56)}
ic {equal(output[2,:name], "Alex")}
ic {equal(output[2,:age], 44)}

Definition

@inline
def load_jsonlines = rel_primitive_load_jsonlines

log

log[x, y]

Logarithm of y with given base x.

Defined for non-negative x and y.

Example:

log[2, 8] = 3.0

Definition

@inline
def log[x, y] = rel_primitive_log[x, y]

log10

log10[x]

Base 10 logarithm.

Defined for non-negative x.

log10[1000] = 3.0

Definition

@inline
def log10[x] = rel_primitive_log10[x]

log_card_est

log_card_est[R]

The log base 2 of the cardinality estimate of a given relational abstraction R.

Definition

@inline
def log_card_est[R] = rel_log_card_est[R]

lowercase

lowercase[string_or_char]

A string where all the characters are converted to lowercase. If a character is already lowercase or has no lowercase version, it remains unchanged.

Example:

def output = lowercase["aB1c"]
"ab1c"
def output = lowercase['Â']
'â'

lowercase does not take a locale option and does not handle local-specific case mapping rules.

Definition

@inline
def lowercase = rel_primitive_lowercase

lt

lt(x, y)
x < y

Definition

@inline
def lt(x, y) = rel_primitive_lt(x, y)

@inline
def (<)(x, y) = lt(x, y)

lt_eq

lt_eq(x, y)
x  y
x <= y

Definition

@inline
def lt_eq(x, y) = rel_primitive_lt_eq(x, y)

@inline
def ()(x, y) = lt_eq(x, y)

@inline
def (<=)(x, y) = lt_eq(x, y)

mae

mae[YHAT, Y]

Mean absolute error (MAE, L1)

Definition

@inline
def mae[YHAT,Y] = mean[x... : abs[Y[x...] - YHAT[x...]]]

max

max[R]

Maximum of (the last argument of) the relation R, if non-empty. If R is empty, max[R] is false (empty). Use <++ (left override) if you need a specific value for the case where R is empty.

Example:

max[{(2, 3); (1, 6)}] = 6

Definition

@inline
def max[R] = rel_primitive_reduce_noinit[maximum, R]

Max

Max[R]

Please use max[R]. Deprecates in near future

Definition

@inline
def Max = max

max_k

max_k[K, R]

The maximum K distinct values of (the last column of) the relation R, if non-empty. If R contains fewer than K distinct values (in its last column), then max_k will have fewer than K values too. If R is empty, max_k[K, R] is false. Currently, the maximum supported value of K is 10000.

Example:

max_k[2, {(2, 3); (1, 6); (3, 5); (1, 1); (4, 1); (2, 6)}]

is {5; 6}.

Definition

@inline
def max_k[K, R] = rel_primitive_mink[R, K, 1]

maximum

maximum[x, y]

Maximum of two arguments. The arguments should have the same type.

Examples:

maximum[3, 4] = 4
maximum[3.0, 4.0] = 4.0

Definition

@inline
def maximum[x, y] = rel_primitive_max[x, y]

mean

mean[R]
average[R]

The arithmetic mean of (the last argument of) the relation R, if non-empty. Empty if R is empty.

Definition

@inline
def mean[R] = sum[R] / count[R]

mean_normalization

mean_normalization[R]

Mean normalization: xn=(xnmmean[R])/mmax[R]mmin[R]x_n = (x_n - { m mean}[R]) / { m max}[R] - { m min}[R]

Definition

@inline
def mean_normalization[R][x...] =
    (R[x...] - mean[R]) / (max[R] - min[R]), (max[R] > min[R])

median

median[R]

The median of R according to the sort order of the last element. If the median index is between two values, then median takes the mean of the middle two values.

Example:

def output = median[{(1, 12); (2, 16); (3, 24)}]
16

Definition

@inline
def median[R] = percentile[R, 50]

metaphone

metaphone[string_value, length]

metaphone returns an encoding of the phonetic representation of string_value. metaphone lets you compare words and sentences based on how they sound in English. The metaphone is considered as an improvement of the soundex algorithm, and does a better job at matching words and names which sound similar.

string_value could be any arbitrary long string. length is the length of the encoding that is used for comparison purpose. A length of 4 is commonly used.

Examples:

def output = metaphone["Smith", 4]
"sm0"

def output = metaphone["Smythe", 4]
"sm0"

def output = metaphone["Christina", 4]
"xrst"

def output = metaphone["Cristine", 4]
"krst"

def output = metaphone["I like Music", 4]
"ilkm"

def output = metaphone["i loak Museek", 4]
"ilkm"

def output = metaphone["RelationalAI", 4]
"rlxn"

def output = metaphone["rellationalleAI", 4]
"rlxn"

Here a simple illustration. Consider the following:

def my_favorite_bands = { "Metallica"; "Iron Maiden"; "Wolfheart"; "Insomium" }
def do_I_like = "Mee ta li ka"
def output = {y in my_favorite_bands : metaphone(do_I_like, 4, z), metaphone[y, 4] = z from z}

the script above returns "Metallica" since since "Mee ta li ka" and "Metallica" have the same metaphone code.

Definition

@inline
def metaphone = rel_primitive_metaphone

Microsecond

Microsecond[n]

A period that spans n microseconds. The period is given in units of Microsecond. (One second is 10610^6 microseconds.)

Example:

datetime_add[unix_epoch, Microsecond[5 * 10^6]]

gives

Dates.DateTime("1970-01-01T00:00:05")

Definition

@inline
def Microsecond = rel_primitive_microsecond

Millisecond

Millisecond[n]

A period that spans n milliseconds. The period is given in units of Millisecond. (One second is 10310^3 milliseconds.)

Example:

datetime_add[unix_epoch, Millisecond[20 * 10^3]]

gives

Dates.DateTime("1970-01-01T00:00:20")

Definition

@inline
def Millisecond = rel_primitive_millisecond

min

min[R]

Minimum of (the last argument of) the relation R, if non-empty. If R is empty, min[R] is false. Use <++ (left override) if you need a different value in this case.

Example:

min[{(2, 3); (1, 6)}] = 3

Definition

@inline
def min[R] = rel_primitive_reduce_noinit[minimum, R]

Min

Min[R]

Please use min[R]. Deprecates in near future

Definition

@inline
def Min = min

min_k

min_k[K, R]

The minimum K distinct values of (the last column of) the relation R, if non-empty. If R contains fewer than K distinct values (in its last column), then min_k will have fewer than K values too. If R is empty, min_k[K, R] is false. Currently, the maximum supported value of K is 10000.

Example:

min_k[2, {(2, 3); (1, 6); (3, 5); (1, 1); (4, 1); (2, 6)}]

is {1; 3}.

Definition

@inline
def min_k[K, R] = rel_primitive_mink[R, K, 0]

min_max_normalization

min_max_normalization[R]

Min-Max normalization: xn=(xnmmin[R])/mmax[R]mmin[R]x_n = (x_n - { m min}[R]) / { m max}[R] - { m min}[R].

Also known as min-max scaling or rescaling.

Definition

@inline
def min_max_normalization[R][x...] =
    (R[x...] - min[R]) / (max[R] - min[R]), (max[R] > min[R])

minimum

minimum[x, y]

Minimum of two arguments. The arguments should have the same type.

Examples:

minimum[3, 4] = 3
minimum[3.0, 4.0] = 3.0

Definition

@inline
def minimum[x, y] = rel_primitive_min[x, y]

Minute

Minute[n]

A period that spans n minutes. The period is given in units of Minute.

Example:

datetime_add[unix_epoch, Minute[100]]

Definition

@inline
def Minute = rel_primitive_minute

missing

missing

Singleton missing value.

Examples:

missing[] = missing
equal( (x : missing(x)) , {missing} )

Definition

@inline
def missing = rel_primitive_missing

Missing

Missing(x)

Tests whether x has the data type Missing.

The only data value that is of type Missing is missing. Used for imported data that contains explicit nulls (such as JSON data).

Note: Missing (with a capital) refers to the data type, while missing is the data itself.

Examples:

Query that returns all relations without missing as the second argument:

def R = { (1, 2); (3, missing) }

def output(x, y) = R(x, y) and not Missing(y)
//output> (1, 2)

Definition

@inline
def Missing = rel_primitive_Missing

mode

mode[R]

Finds the most common value in (the last argument of) the relation R. If there are multiple most common values, then mode chooses the first one according to the sort order.

Examples:

def example = {(1, "a"); (2, "b"); (3, 123); (4, 12.5); (3, "b")}
mode[example] = "b"

def example = {(1, 1); (2, 1); (3, 1); (4, "b"); (5, "b"); (6, "c"); (7, "c")}
mode[example] = 1

Definition

@inline
def mode[R] = last[top[1, argmax[frequency[R]]]]

model_is_function

ic model_is_function

An integrity constraint ensuring that rel:catalog:model has a functional dependency from the name to the actual value.

Definition

ic model_is_function(k) { function(rel:catalog:model[k]) }

modulo

modulo[x, y]

Modulus after division, in the range [0,y), if y is positive, or (y,0] if y is negative.

Satisfies x = floor_divide[x,y] * y + modulo[x, y].

Is false (empty) when y = 0 for Integer arguments.

Examples:

modulo[8, 3] = 2
modulo[8, -3] = -1
modulo[-8, 3] = 1
empty(modulo[3,0])

modulo[x, y] is defined when either the types of x and y are the same, or either operand is of type SignedInt[64] or Floating[64]. The return type is:

  • the type of x when the types of both operands are the same.
  • SignedInt[n] or UnsignedInt[n] where n is the greater of the number of bits in the types of x and y and the signedness is chosen to match that of the type of y, and both x and y are of integer types.
  • Floating[64] when either the type of x or the type of y is Floating[64].
  • The type of x when x is a non-integer type and y is of type SignedInt[64].
  • The type of y when y is a non-integer type and x is of type SignedInt[64].
  • Rational[n] where n is the greater of the number of bits in the types of x and y, and one operand is a rational type and the other is a SignedInt[64].

Definition

@inline
def modulo[x, y] = rel_primitive_modulo[x, y]

Month

Month[n]

A period that spans n months. The period is given in units of Month.

Example:

datetime_add[unix_epoch, Month[10]]

Definition

@inline
def Month = rel_primitive_month

mse

mse[YHAT, Y]

Mean squared error (MSE, L2)

Definition

@inline
def mse[YHAT, Y] = Σ[x... : (Y[x...] - YHAT[x...]) ^ 2] / count[Y]

multiply

multiply[x, y]
x * y

Multiplication.

Examples:

3 * 2 = 6
equal(multiply(3.0, 2, 6.0), true)

Definition

@inline
def multiply[x, y] = rel_primitive_multiply[x, y]

@inline
def (*)(x, y, z) = multiply(x, y, z)

murmurhash3f

murmurhash3f[v]

Hash v according to the MurmurHash3f algorithm with the default seed 0, as a UInt128.

Equivalent to murmurhash3f_with_seed[0, v].

Definition

@inline
def murmurhash3f[v] = murmurhash3f_with_seed[int64[0], v]

murmurhash3f_with_seed

murmurhash3f_with_seed[seed in (Int64UInt128), key]

Hash key with seed via the MurmurHash3F algorithm, and yield the result as a UInt128.

The key may be any singleton value supported in Rel, (e.g. [U]Int{8,16,32,64,128}, Float{16,32,64}, Char, String, RelName, FilePos, Missing, …).

The seed must be a value in (Int64 ∪ UInt128).

Note that the standard specification for MurmurHash3F does not include 128-bit seeds, so the result of hashing with a 128-bit seed will be different than a hash with the same value stored in a smaller integer type (i.e. hashing with the seeds uint64[1] and uint128[1] will result in different hashes).

This function is an implementation (in Julia) of the reference implementation of MurmurHash3F, i.e. MurmurHash3x64128. The original can be retrieved via:

https://github.com/aappleby/smhasher/blob/ 61a0530f28277f2e850bfc39600ce61d02b518de/src/MurmurHash3.cpp

The reference implementation was written by Austin Appleby and made public domain.

Note that this implementation departs from the standard, in order to provide different hashes for byte-identical values of different types. While this implementation’s internals should produce byte-identical results to the reference implementation, the primary entrypoint here, (murmurhash3f[seed, key]) typically will not: To distinguish byte-identical values of different types, this function mixes type information into seeds prior to hashing.

Also note that the reference implementation supports only UInt32 seeds; for such seeds, this implementation’s internals should provide byte-identical results. But this implementation accepts seeds up to and including 128-bit integer types. (For UInt32 seeds, the reference implementation initializes h1/h2, the two UInt64s in which it accumulates/mixes the digest, with copies of seed. This implementation does the same for UInt32 seeds, and also for all other seed types that are less than or equal to 64 bits. For 128-bit seeds, this implementation initializes h1 with the seed’s low bits and h2 with the seed’s high bits.)

Examples

// Without a seed
murmurhash3f["cat"] = 0x37322aa78b4b4ef4816da65505e8efaa
// With a seed
murmurhash3f_with_seed[8675309, "cat"] = 0x9fd1d18093ba94b6f454b48e58011643
murmurhash3f_with_seed[8675309, :bunny] = 0x56f800aed5ff3b7300ace61c691e7568

// byte-identical values of different types hash differently
murmurhash3f[0x01] = 0x6ed3439777f613f7df39df0849d45e09
murmurhash3f[1] = 0x9116cd3f0a651c49f1674100935b29bf

Definition

@inline
def murmurhash3f_with_seed[seed, v] = rel_primitive_murmurhash3f_with_seed[seed, v]

Nanosecond

Nanosecond[n]

A period that spans n nanoseconds. The period is given in units of Nanosecond. (One second is 10910^9 nanoseconds.)

Example:

datetime_add[unix_epoch, Nanosecond[5*10^9]]

gives

Dates.DateTime("1970-01-01T00:00:05")

Definition

@inline
def Nanosecond = rel_primitive_nanosecond

nanoseconds_to_datetime

nanoseconds_to_datetime[ns]

Convert nanoseconds (from the unix epoch) to datetime.

Analogous to datetime.datetime.utcfromtimestamp(seconds) in python.

Example:

nanoseconds_to_datetime[24 * 60 * 60 * 10^9]

is 1970-01-02T00:00:00 (one day after unix_epoch).

Definition

@inline
def nanoseconds_to_datetime[nanoseconds] = datetime_add[unix_epoch, Nanosecond[nanoseconds]]

natural_exp

natural_exp[x]

Exponentiation with the base of the natural log, e.

Definition

@inline
def natural_exp[x] = rel_primitive_natural_exp[x]

natural_log

natural_log[x]

Natural logarithm (ln) (base e).

Defined for non-negative x.

Definition

@inline
def natural_log[x] = rel_primitive_natural_log[x]

neq

neq(x, y)
x  y
x != y

Examples:

  • 1 != 2 is true
  • 1 != 1.0 is true
  • 1 != "a" is true

Definition

@inline
def neq(x, y) = rel_primitive_neq(x, y)

@inline
def ()(x, y) = neq(x, y)

@inline
def (!=)(x, y) = neq(x, y)

num_bytes

num_bytes[str]

The number of bytes in a string value.

If a string contains Unicode characters, it’s possible that some characters take up more than one byte, so num_bytes[str] may be larger than num_chars[str].

Example:

num_bytes["abcd"] = 4
num_bytes["中文例子"] = 12

Definition

@inline
def num_bytes = rel_primitive_num_bytes

num_chars

num_chars[str]

The number of (Unicode) characters in a string, i.e. the length of the string.

A character is also known as a “Code Point” in the Unicode specification.

Example:

num_chars["abcd"] = 4
num_chars["中文例子"] = 4

Definition

@inline
def num_chars = rel_primitive_num_chars

Number

Number(x)

Holds if x is a number (for example, Int ,Float, or their fixed-width versions).

See also: Int and Float in intrinsics.rel.

Example:

Integrity constraint that tests whether x is of type Number (throws if x is not of type Number):

def R  = {1; 1.5; uint[8,3]; 4; float[32, 1.321]}

ic number_ic { subset(R, Number) }

Definition

@inline
def Number = rel_primitive_Number

numerator

numerator[x]

Numerator of a rational-like number.

Examples:

numerator[rational[64, 1, 3]] = 1
numerator[rational[64, 1, 100]] = 1
numerator[rational[64, 1, 100] + rational[64, 1, 3]]  =  103
numerator[parse_decimal[64, 2, "3.14"]]  =  314
numerator[parse_decimal[64, 5, "3.14159"]]  =  314159
numerator[5]  =  5

Definition

@inline
def numerator = rel_primitive_numerator

pack

pack(x₁, ..., xₙ, v)

Creates a packed structure v storing the values (x₁, ..., xₙ) in sequence.

pack cannot be used with point-free syntax.

Direct usage of pack should be avoided, as it serves primarily as a low-level interface for value types.

Definition

@inline
def pack = rel_primitive_pack_logical

parse_date

parse_date[d, format]

Parse Date.

This function is only defined for a valid combination of date string d and format string format, and invalid arguments will evaluate to false (the empty relation, {}).

Examples:

def output = parse_date["2018-06-12", "Y-m-d"]
2018-06-12

For details on the format parameter, see the Julia documentation for Dates.DateFormat.

Definition

@inline
def parse_date = rel_primitive_parse_date

parse_datetime

parsedatetime[datestring, format] parsedatetime(datestring, format, datetime)

Maps a (date_string, format) pair, where date_string contains the datetime information in a String and format specifies the datetime format, to a Datetime value type holding the corresponding datetime information.

This function is only defined for a valid combination of date string date_string and format string format. Invalid arguments will evaluate to false (the empty relation, {}).

Examples:

def output = parse_datetime["2018-06-12 13:00 +00:00", "YYYY-mm-dd HH:MM zzzz"]
//output> Dates.DateTime("2018-06-12T13:00:00")

def output = parse_datetime["2018-03-11 01:00 America/New_York", "Y-m-d H:M Z"]
//output> Dates.DateTime("2018-03-11T06:00:00")

For details on the supported date format specified in format, see the Julia documentation for Dates.DateFormat.

Definition

@inline
def parse_datetime = rel_primitive_parse_datetime

parse_decimal

parse_decimal[bits, digits, string]

Parse a string representation of a fixed-point decimal number to a decimal[a,b]

The implementation does not consider a locale. The decimal separator is always . and the thousands separator is always ,. The thousands separator is optional.

If the string has more digits after the decimal point than supported by the b parameter, then the number is rounded to the nearest number.

This function is only defined for valid arguments. For invalid arguments it will be false (the empty relation, {}).

Examples:

parse_decimal[64, 2, "3.14"]
FixedDecimal{Int64,2}(3.14)

parse_decimal[64, 1, "4.27"]
FixedDecimal{Int64,1}(4.3)

parse_decimal[64, 2, ""]
FixedDecimal{Int64,2}(0.00)

parse_decimal[64, 2, "1,234.567"]
FixedDecimal{Int64,2}(1234.57)

parse_decimal[64, 2, "invalid"]
{}

Definition

@inline
def parse_decimal = rel_primitive_parse_decimal

parse_float

parse_float[string]

Parse a string representation of a float to a Floating[64]

Note that this currently does not consider a locale (decimal separator is always .)

Is false (empty) if the string fails to parse as an integer.

Examples:

parse_float["3.14"] = 3.14
parse_float["3"] = 3.0
empty(parse_int["hello"])

Definition

@inline
def parse_float = rel_primitive_parse_float

parse_int

parse_int[string]

Parse a string representation of an integer to a SignedInt[64]

Is false (empty) if the string fails to parse as an integer.

Example:

parse_int["123"] = 123
empty(parse_int["hello"])

Definition

@inline
def parse_int = rel_primitive_parse_int

parse_json

parse_json[json_string_value]

Parses a JSON value directly from a string.

Example:

def json = parse_json["""{"name": "Anton", "age": 56}"""]

Definition

@inline
def parse_json = rel_primitive_parse_json

parse_uuid

parse_uuid[str]

Parse a UUID string (in the standard 8-4-4-4-12 format) to a UInt128 value.

Example:

parse_uuid["22b4a8a1-e548-4eeb-9270-60426d66a48e"] = 0x22b4a8a1e5484eeb927060426d66a48e

Definition

@inline
def parse_uuid = rel_primitive_parse_uuid

pattern_match

pattern_match(pattern, string)

Match string with a pattern (pre-compiled regular expression)

Example:

def p = regex_compile["^.*@.*$"]
def output = pattern_match(p, "foo@example.com")
true

Definition

@inline
def pattern_match = rel_primitive_pattern_match

percentile

percentile[R, p]

Select the element of R corresponding to the p-th percentile, computed with linear interpolation.

If the p-th percentile does not correspond to a single element, then percentile_continuous computes a weighted average of the two values surrounding the p-th percentile.

There are a number of different approaches for computing linearly interpolated percentiles. Here, the rank of the percentile is computed by x = p(N - 1) + 1, which corresponds to the C = 1 variant in https://en.wikipedia.org/wiki/Percentile. This is also the approach used in Julia’s libraries and NumPy.

Note that percentile is not defined for p <= 0 or p > 100.

Example:

def output = percentile[{(1, 12); (2, 24); (3, 16)}, 60]
17.6

Definition

@inline
def percentile[R, p] = value :
    // index is a float value, we multiply by count[R] - 1 first rather than divide by
    // 100.0 first to avoid imprecision
    index = (p * (count[R] - 1.0)) / 100.0 + 1.0 and

    // compute the lower and upper bounds surrounding the value for the p-th percentile
    value = sum[i, s:
        // the value at index i contributes (1 - abs[i - index]) proportion of the final
        // percentile
        s = v * (1 - abs[i - index]) and

        // sort[y, x...] instead of sort[last[R]] to handle duplicate values
        sort[y, x...: R(x..., y)](i, v, rest...) and

        // only for the values surrounding index
        index - 1.0 <= 1.0 * i <= index + 1.0
        from v, rest...
    ]
    from index

percentile_nearest

percentile_nearest[R, p]

Select the element of R corresponding to the p-th percentile.

The p-th percentile is the smallest value in the list such that no more than p percent of the data is strictly less than the value and at least p percent of the data is less than or equal to that value (nearest-rank method in https://en.wikipedia.org/wiki/Percentile). This function computes the p-th percentile based on ranking the last argument of R.

Note that percentile_nearest is not defined for p <= 0 or p > 100.

Example, here 33% of the values are strictly less than 16, and 66% of the values are less than or equal to 16, so the 60-th percentile is 16:

def output = percentile_nearest[{(1, 12); (2, 24); (3, 16)}, 60]
16

Definition

@inline
def percentile_nearest[R, p] =
    // we need to sort[y, x...] instead of sort[last[R]] to handle duplicates properly
    first[(sort[y, x...: R(x..., y)][i] from i
        where i = float_int_convert[ceil[(p / 100.0) * count[R]]])]

period_add

period_add[period1, period2]

Add two periods. For now, they need to be of the same type.

def output = period_add[Minute[3], Minute[4]]
Minute[7]

Definition

@inline
def period_add = rel_primitive_add

period_day_to_int

period_day_to_int[day]

Conversion from period Day to int.

In other words, returns the value of Day data type as an integer.

Example:

def num_of_days = period_day_to_int[Day[3]]
ic { equal(num_of_days, 3) }

Definition

@inline
def period_day_to_int = rel_primitive_period_days

period_max

period_max[period1, period2]

Maximum of two periods. The result is expressed as a nanosecond period.

Example:

def output = period_max[Minute[3],Minute[4]] = Minute[4]
true

Definition

@inline
def period_max[x,y] = rel_primitive_max_period[x,y]

period_min

period_min[period1, period2]

Minimum of two periods. The result is expressed as a nanosecond period.

Example:

def output = period_min[Minute[3],Minute[4]] = Minute[3]
true

Definition

@inline
def period_min[x,y] = rel_primitive_min_period[x,y]

pi_float64

pi_float64

The constant pi.

Example:

cos[pi_float64] = -1.0

Definition

@inline
def pi_float64 = 3.14159265358979323846

pivot

pivot[R]

Associates each element of an input tuple with its index in the tuple. The result is always a binary relation.

pivot[(t1, ..., tn)] generates pairs (1, t1);... (i, ti); ... (n, tn) This is useful to match up long tuples with names (see zip), for example.

Example:

def output = pivot[{(5, 60, 7, 2)}]
(1, 5)
(2, 60)
(3, 7)
(4, 2)

pivot[true] and pivot[false] are both false (the empty relation) For unary relations, pivot[{(x)}] is {(1,x)}

Definition

@inline
def pivot[R] = rel_primitive_pivot[R]

pop_standard_deviation

pop_standard_deviation[R]
pop_stddev[R]

Population standard deviation

Definition

@inline
def pop_standard_deviation[F] = sqrt[pop_variance[F]]

@inline
def pop_stddev = pop_standard_deviation

pop_variance

pop_variance[R]

Population variance

Definition

@inline
def pop_variance[F] = mean[squared_deviation[F]], count[F] > 1

pop_zscore_normalization

pop_zscore_normalization[R]

Z-score normalization (population) of (the last argument of) the relation R.

Definition

@inline
def pop_zscore_normalization[R][x...] =
    (R[x...] - mean[R]) / pop_stddev[R]

power

power[x, y]
x ^ y

Exponentiation: xx to the power of yy.

Examples:

power[2, 3] = 8
power[3.0, 2] = 9.0
power[9, 0.5] = 3.0

Definition

@inline
def power[x, y] = rel_primitive_power[x, y]

@inline
def (^)(x, y, z) = power(x, y, z)

prefix_join

prefix_join[R, S]
R <: S

The prefix join (or restriction) of S to R consists of the tuples (x..., y...) in S where the prefix (x...) is in R.

That is, R <: S contains the tuples in S that have a prefix in R.

ExampleNormalized
n <: edgex, y: edge(x, y) and x = n
female <: parentx, y: female(x) and parent(x, y)
t <: playersx, p: players(x, p) and t = x
t.players <: agep, v: players(t, p) and age(p, v)
intern <: salaryp, v: intern(p) and salary(p, v)

The restriction operator can also be used to select subsets of JSON-like relations:

  • :a <: json where json is {"a": {"b": 1, "c": 2}, "d": 3} has value {"a": {"b": 1, "c": 2}}
  • (:[], _, :a) <: json where json is [ {"a": 1, "b": 2}, {"a": 3, "b": 4} ] has value [ {"a": 1}, {"a": 3} ].

Definition

@inline
def prefix_join[R, S](x..., y...) = R(x...) and S(x..., y...)

@inline
def (<:) = prefix_join

product

product[R]

Product of (the last argument of) the relation R, if non-empty. If R is empty, product[R] is false.

Example:

product[{(1, 4); (2, 5)}] = 20

Definition

@inline
def product[R] = rel_primitive_reduce_noinit[*, R]

proper_subset

proper_subset(R, S)
R ⊂ S

R is a proper subset of S if subset(R, S) and R and S are not equal. This means there has to be at least one fact in S that is not in R.

Definition

@inline
def proper_subset(R, S) = subset(R, S) and not equal(R, S)

@inline
def (⊂) = proper_subset

proper_superset

proper_superset(R, S)
R ⊃ S

Inverse of propersubset. See propersubset.

Definition

@inline
def proper_superset(R, S) = superset(R, S) and not equal(R, S)

@inline
def (⊃) = proper_superset

rad2deg

rad2deg[x]

Convert radians to degrees.

Definition

@inline
def rad2deg[x] = x / pi_float64 * 180.0

random_mersenne_twister

random_mersenne_twister[seed, R]

Generates a pseudorandom number for every tuple in R using the Mersenne Twister PRNG.

random_mersenne_twister takes a relation R(xs...) and produces a relation with the tuples (xs..., r) where r is a pseudorandom number generated by using the Mersenne Twister algorithm with the given seed. The value r is a Float64 in the range of [0, 1).

The seed is required to be a number convertible to an unsigned 128-bit integer (UInt128) or a DateTime. For a seed dt of type DateTime the value of epoch_milliseconds[dt] is used as seed.

Please note that if seed has more than one value, then the smallest value in seed will be used as the seed (according to the native sort order that is used by sort). If seed is empty, then the result is empty.

Example:

def output = random_mersenne_twister[0, true]
(0.8236475079774124,)

def output = random_mersenne_twister[random_uint128, 'a']
('a', 0.590845)

def sample = {'a'; 'b'; 'c'}
def output = random_mersenne_twister[random_uint128, sample]
('a', 0.590845)
('b', 0.766797)
('c', 0.566237)

def output = random_mersenne_twister[datetime_now, true]
(0.09946255273771532,)

Definition

@inline
def random_mersenne_twister[SEED, R] =
    rel_primitive_mersenne_twister[
        // We specifically need a single value in the SEED, which is why we take the top-1.
        // We need to take the conjunction with `R` because otherwise we may not get the
        // same number of groups, which is not supported by the foreign function
        // implementation.
        (_internal_convert_seed[top[1, SEED, 1]], exists(R)),
        // The restriction to the SEED is needed to make sure that the SEED is not empty for
        // the current group. This is a limitation in the implementation of the foreign
        // function.
        (R, exists(_internal_convert_seed[top[1, SEED, 1]]))]

@inline
def _internal_convert_seed[x in Number] = uint128[x]

@inline
def _internal_convert_seed[x in DateTime] = uint128[epoch_milliseconds[x]]

random_threefry_float64

random_threefry_float64[key1, key2]

Generates a pseudorandom number of type Float64 between 0.0 and 1.0 using the Threefry algorithm.

The Threefry algorithm depends on two keys: key1 and key2 of type UInt64 or Int64. One key can be used as the seed. The other key can be used as the stream position. Which key is used for what is up to the user.

Examples:

Getting a single random Float64 number:

def output = random_threefry_float64[1234, 1]
0.14708645730224323

Generating a sequence of random Float64 numbers using a user-specific seed:

def output = random_threefry_float64[1234, i] for i in range[1, 3, 1]
(1, 0.14708645730224323)
(2, 0.18124311325337028)
(3, 0.14463837673485513)

Using a randomly generated (but deterministic) seed via random_threefry_uint64:

def seed = random_threefry_uint64[1234, 0]

def output = random_threefry_float64[seed, i] for i in range[1, 3, 1]
(1, 0.2959011588531637)
(2, 0.5971368640131505)
(3, 0.4259318598026802)

Generating a random Float64 number from the indeterministic random seed random_uint64:

def output = random_threefry_float64[random_uint64, 0]

Definition

@inline
def random_threefry_float64[key1, key2] = rel_primitive_threefry[key1, key2] . rel_primitive_threefry_uniform

random_threefry_uint64

random_threefry_uint64[key1, key2]

Generates a pseudorandom number of type UInt64 using the Threefry algorithm.

The Threefry algorithm depends on two keys: key1 and key2 of type UInt64 or Int64. One key can be used as the seed. The other key can be used as the stream position. Which key is used for what is up to the user.

Note that the output of random_threefry_uint64 can be used as a key again in a subsequent call. This capability allows the user to start a new random sequence from a previous sequence without the need to track which keys have already been used.

Examples:

Getting a single random UInt64 number:

def output = random_threefry_uint64[1234, 1]
0xbf025a77543cc31d

Generating a sequence of random UInt64 numbers:

def output = random_threefry_uint64[1234, i] for i in range[1, 3, 1]
(1, 0xbf025a77543cc31d)
(2, 0xe532e65f2dc0c673)
(3, 0x1bb25070549d29e7)

Generating two sequences of random UInt64 numbers by creating a new seed from the previous seed:

def seed1 = random_threefry_uint64[1234, 0]
def seed2 = random_threefry_uint64[seed1, 0]

def output:one = random_threefry_uint64[seed1, i] for i in range[1, 3, 1]
def output:two = random_threefry_uint64[seed2, i] for i in range[1, 3, 1]
(:one, 1, 0x2ca4bc02da81f726)
(:one, 2, 0x12898ddf6262c27b)
(:one, 3, 0xd606d09ded02d4dd)
(:two, 1, 0x48879dd755a393be)
(:two, 2, 0xc5683a27c1e876a7)
(:two, 3, 0xfdb025894557c350)

Definition

@inline
def random_threefry_uint64 = rel_primitive_threefry

random_uint128

random_uint128

Random number generated using the random device. It is an unsigned 128-bit integer.

A single random number is generated using the random device of the operating system for every transaction. Within the transaction, the random number does not change. The random number can be used as the seed of a pseudo-random number generator.

Examples:

def output = random_uint128
(0x648fa1de9056c1c7de7fc61bc138f5a8,)

Please note that random numbers need to be used with caution in views that need to be materialized or even incrementally maintained. If a view depends on a random number, then it needs to be recomputed every time the view is needed because the random number changes with every transaction and the view is always be consistent with its inputs.

If a random seed should not change with every transaction, then the state of a seed should be managed explicitly as an EDB. For example, a seed uses for a PRNG can be initialized in one transaction and then the same value is used from that point. In this way, materialized views involving random numbers can be reused.

Definition

@inline
def random_uint128 = rel_primitive_transaction_edb[:txn_random_uint128]

random_uint64

random_uint64

Random number generated using the random device. It is an unsigned 64-bit integer.

This is a truncation of random_uint128 to the lower 64 bits. It does not provide an additional source of random numbers.

Please check out the documentation for random_uint128 to get more information about how to use random numbers in various situations.

Example:

def output = random_uint64
(0xde7fc61bc138f5a8,)

Definition

@inline
def random_uint64 = uint128_uint64_truncate[random_uint128]

range

range(low, hi, stride, x)

Generate a relation with integer values between low and high, inclusive, advancing by stride each time.

  • If low <= hi and stride > 0, then range[low, hi, stride] has all the values x = low + i * stride <= hi for non-negative integers i.
  • Otherwise (i.e. if low > hi or stride <= 0), then range[low, hi, stride] is empty.

Example:

def output = range[1, 10, 4]
1
5
9

Definition

@inline
def range = rel_primitive_range

rational

rational[n, num, denom]

The n-bit rational value from integer numerator num and denominator denom.

Definition

@inline
def rational = rel_primitive_rational

Rational

Rational(nbits, x)

Holds if x is an nbits rational constructed with rational[nbits, numerator, denominator].

Example:

Query that checks that x is of type Rational with a bit size of nbits, a numerator of -5, and a denominator of -7:

def my_rational = rational[16, -5, -7]

def output(x) = my_rational(x) and Rational(16, x)
//output> 5/7

Definition

@inline
def Rational = rel_primitive_Rational

rational_convert

rational_convert[n, x]

Convert a number x to a rational with n-bit integer representation.

Any type that supports numerator and denominator is supported.

Examples:

rational_convert[64, 3]  =  (3//1,)
rational_convert[64, decimal[64, 2, 0.75]]  =  (3//4,)

Definition

@inline
def rational_convert[n, a] =
    rational[n, numerator[a], denominator[a]]

regex_compile

regex_compile(regex, pattern)

Compile a regular expression to a pattern. If regex is a pattern, rather than a string, then pattern = regex.

Example:

def p = regex_compile["^.*@.*$"]
pattern_match(p, "foo@example.com")
()

Definition

@inline
def regex_compile = rel_primitive_regex_compile

regex_match

regex_match(regex, string)

Match string with a regular expression (string or pattern).

Note that if the entire string needs to be matched, then a leading ^ and trailing $ are required in the regular expression.

Example:

regex_match("^.*@.*$", "a@b")

Definition

@inline
def regex_match = rel_primitive_regex_match

regex_match_all

regex_match_all[regex, input_string]

All the occurrences of a regular expression regex (string or pattern) in string, including all the matches and offsets.

Each match is a pair (i, s) with the character index i where that matching string starts and the matching string s.

Example:

regex_match_all["(?<hour>\\d+):(?<minute>\\d+)",
                "Appointment from  12:45 to 1:30"]

is equal to

{(19, "12:45"); (28, "1:30")}

Definition

@inline
def regex_match_all[re, str] = rel_primitive_regex_match_mv[regex_compile[re], str]

rel:integration:load_partitioned_csv

rel:integration:load_partitioned_csv[URI_string]
rel:integration:load_partitioned_csv[R]

This is the main in-language API for loading partitioned CSV data into a single logical relation.

For detailed information on CSV loading, see load_csv.

To load a partitioned CSV, specify the following:

  • :path: The path to the first CSV partition,

e.g. "azure://storage.blob.core.net/container/file_1_of_24.csv". For more than one partition to get picked up, please note the naming schema: Name partitions using [...]_m_of_n[...], where m is the current partition and n the total number of partitions (e.g. mycsv_1_of_16.csv, mycsv_2_of_16.csv, etc.).

  • Only the first partition should include header information.
  • Make sure all partitions are stored under the same :path prefix.

The resulting relation is of the form (column_name, partition, line, value). partition, line uniquely identifies each row in the CSV.

If errors occur, the resulting relation is of the form (:load_errors, partition, line, error_column, raw_line).

partition and line are UInt32s. To address them, please use uint[32, <number>].

If only a single partition is provided or data is directly passed into the load, the resulting relation looks just like a load_csv call with an added partition = uint[32, 1].

Definition

@inline
def rel:integration:load_partitioned_csv = rel_primitive_load_part_csv

RelName

RelName(x)

Holds if x is a relation name.

Examples:

Integrity constraint that specifies that a module always contains a RelName and an integer:

def mymodule:f = 1

ic { subset(mymodule, (RelName, Int)) }

def output = mymodule
//output> (:f, 1)

Definition

@inline
def RelName = rel_primitive_RelName

relname_string

relnamestring[relationname] relnamestring(relationname, string)

Maps a RelName (relation_name) to its corresponding String value (string). Contains relation_name and string when relation_name matches string.

Relation names (aka RelNames), when treated as data, start with the symbol :.

Can be used to

  1. Verify that a RelName and string match. relname_string(relation_name, string) evaluates to true if relation_name matches string. If relation_name does not match string, relname_string(relation_name, string) evaluates to false (the empty relation, {}).
  2. Convert a RelName to a string (relnamestring[relationname] provides shorthand for this use).

Note: converting a string to a RelName is not supported.

Examples:

Verify that a RelName matches a string.

relname_string(:rel, "rel")
//evaluates to `true` because the RelName `:rel` matches the string `"rel"`.

Convert RelName :employees to string:

def output = x: relname_string(:employees, x)
//output> "employees"

Convert RelName :employees to string using equivalent shorthand:

def output = relname_string[:employees]

Definition

@inline
def relname_string = rel_primitive_relname_string

remainder

remainder[x, y]
x % y

Remainder from Euclidean division: a value with the same sign as x, and smaller in magnitude than y. This value is always exact.

Satisfies x = (x / y) * y + (x % y).

Is false (empty) when y = 0 for Integer arguments.

Examples:

8 % 3 = 2
remainder[8, 3] = 2
remainder[8, -3] = 2
remainder[-8, 3] = -2
empty(remainder[6, 0])

remainder[x, y] is defined when either the types of x and y are the same, or either operand is of type SignedInt[64] or Floating[64]. The return type is:

  • the type of x when the types of both operands are the same.
  • SignedInt[n] or UnsignedInt[n] where n is the greater of the number of bits in the types of x and y and the signedness is chosen to match that of the type of x, and both x and y are of integer types.
  • Floating[64] when either the type of x or the type of y is Floating[64].
  • The type of x when x is a non-integer type and y is of type SignedInt[64].
  • The type of y when y is a non-integer type and x is of type SignedInt[64].
  • Rational[n] where n is the greater of the number of bits in the types of x and y, and one operand is a rational type and the other is a SignedInt[64].

Definition

@inline
def remainder[x, y] = rel_primitive_remainder[x, y]

@inline
def (%)(x, y, z) = remainder(x, y, z)

reverse_sort

reverse_sort[R]

Reverse Sort

Like sort, except the ordering is reversed.

Example:

def sample = {'a'; 'b'; 'c'}
def output = reverse_sort[sample]
(1, 'c')
(2, 'b')
(3, 'a')

def sample = {(:g, 'a'); (:g, 'b'); (:g, 'c'); (:h, 'd'); (:h, 'e'); (:h, 'f')}
def output = x: reverse_sort[sample[x]]
(:g, 1, 'c')
(:g, 2, 'b')
(:g, 3, 'a')
(:h, 1, 'f')
(:h, 2, 'e')
(:h, 3, 'd')

Definition

@inline
def reverse_sort[R](ri, args...) =
    sort[R](i, args...) and
    ri = count[R] - i + 1
    from i

right_override

right_override[R, S]
R ++> S

Right override is identical to left override, except the arguments are swapped.

R ++> S contains all of S, plus all the tuples in R whose key is not in S.

There are two equivalent ways to think about R ++> S: (1) R is providing default values for keys that are missing in S. (2) The entries in S are overriding the entries in R.

A few examples to illustrate the difference with left-override:

ExampleValue
2 ++> 33
(1,2) ++> (1,3)(1,3)
(3,4) ++> (1,2); (3,5)(1,2); (3,5)
(3,"abc") ++> (1,2); (3,5)(1,2); (3,5)
(1,2); (3,5) ++> (3,4); (6,7)(1,2); (3,4); (6,7)

Definition

@inline
def right_override[R, S](xs...) = S(xs...)

@inline
def right_override[R, S](xs..., v) = R(xs..., v) and not S(xs..., _)

@inline
def (++>) = right_override

rmse

rmse[YHAT, Y]

Root mean square error (RMSE)

Definition

@inline
def rmse[YHAT,Y] = sqrt[mse[YHAT,Y]]

round

round[mode, x]

Round to the nearest integer according to the given rounding mode.

Definition

@inline
def round(mode, x, rounded) = {
    (mode = :ROUND_UP,                rounded = rel_primitive_round_up[x]);
    (mode = :ROUND_DOWN,              rounded = rel_primitive_round_down[x]);
    (mode = :ROUND_NEAREST,           rounded = rel_primitive_round_nearest[x]);
    (mode = :ROUND_NEAREST_TIES_AWAY, rounded = rel_primitive_round_nearest_ties_away[x]);
    (mode = :ROUND_NEAREST_TIES_UP,   rounded = rel_primitive_round_nearest_ties_up[x]);
    (mode = :ROUND_TO_ZERO,           rounded = rel_primitive_round_to_zero[x]) }

sample_standard_deviation

sample_standard_deviation[R]
sample_stddev[R]

Sample standard deviation of (the last argument of) the relation R.

Definition

@inline
def sample_standard_deviation[F] = sqrt[sample_variance[F]]

sample_stddev

see: sample_standard_deviation

Definition

@inline
def sample_stddev = sample_standard_deviation

sample_variance

sample_variance[R]

Sample variance

Definition

@inline
def sample_variance[F] = sum[squared_deviation[F]] / (count[F] - 1), count[F] > 1

sample_zscore_normalization

sample_zscore_normalization[R]

Z-score normalization (sample) of (the last argument of) the relation R.

Often simply referred to as ‘standardization’.

Definition

@inline
def sample_zscore_normalization[R][x...] = (R[x...] - mean[R]) / sample_stddev[R]

Second

Second[n]

A period that spans n seconds. The period is given in units of Second.

Example:

datetime_add[unix_epoch, Second[100]]

Definition

@inline
def Second = rel_primitive_second

second

second[R]

Projection to the second argument of R.

second supports heterogeneous relations, both in arity and types.

Examples:

def output = second[(1, 2, 3); (4, 5)]
2
5
def output = second[(1, 2); (3, "abc")]
2
"abc"

Definition

@inline
def second[R](x) = (y... : R(_, x, y...))

sign

sign[x]

The sign of x: either 0, -1, or 1.

Definition

@inline
def sign[x] = rel_primitive_sign[x]

SignedInt

SignedInt(nbits, x)

Holds if x is an nbits signed integer.

Examples:

Integrity constraint that tests whether x is an 8-bit signed integer:

def R = int[8, 5]

ic unsigned_type_check(x in R) {
    SignedInt(8, x)
}

Definition

@inline
def SignedInt = rel_primitive_SignedInt

sin

sin[x]

Sine of x (given in radians).

Defined for non-infinite x.

Example:

sin[pi_float64/2] = 1.0

Definition

@inline
def sin[x] = rel_primitive_sin[x]

sinh

sinh[x]

Hyperbolic sine.

Definition

@inline
def sinh[x] = rel_primitive_sinh[x]

sort

sort[R]

Sort a relation.

sort takes a relation R(xs...) and produces a relation R(i, xs...) where i is an Int64 index starting at 1. The index indicates the ordering of the tuples of R.

The relation should have a fixed arity. Otherwise, the result is empty.

The ordering is the native sort order of values in the system and cannot be customized with a comparator. It is possible though to transform the input or output of sort to achieve a different sort order (see reverse_sort for an example).

Example:

def sample = 'a'
def output = sort[sample]
(1, 'a')

def sample = {'a'; 'b'; 'c'}
def output = sort[sample]
(1, 'a')
(2, 'b')
(3, 'c')

Similar to aggregations, it is possible to separately sort per group. In the following example, the sort is grouped by free variable g.

def sample = {(:g, 'a'); (:g, 'b'); (:g, 'c'); (:h, 'd'); (:h, 'e'); (:h, 'f')}
def output = x: sort[sample[x]]
(:g, 1, 'a')
(:g, 2, 'b')
(:g, 3, 'c')
(:h, 1, 'd')
(:h, 2, 'e')
(:h, 3, 'f')

Definition

@inline
def sort = rel_primitive_sort

soundex

soundex[string_value]

soundex returns a 4-letter encoding the phonetic representation of string_value. soundex lets you compare words and sentences based on how they sound in English.

string_value could be any arbitrary long string, but only the first few syllables are considered for the result

Examples:

def output = soundex["Smith"]
"s530"

def output = soundex["Smythe"]
"s530"

def output = soundex["Christina"]
"c623"

def output = soundex["Cristine"]
"c623"

Here a simple illustration. Consider the following:

def my_favorite_bands = { "Metallica"; "Iron Maiden"; "Wolfheart"; "Insomium" }
def do_I_like = "Mee ta li ka"
def output = exists({ y : soundex[my_favorite_bands][y] and equal(y, soundex[do_I_like]) })

the script above evaluates to true since "Mee ta li ka" and "Metallica" have the same soundex code.

Definition

@inline
def soundex = rel_primitive_soundex

spread

spread[mode, R, val]

This definition unifies the spread definitions on integers and FixedDecimal values using the two spreading strategies. Namely, this definition wraps FixedDecimal_spread, int_spread_by_even, and int_spread_by_ratio.

mode can be :even for even spread, or :ratio for ratio spread. R is the relation to execute the spread on. val is the value to spread.

Example:

def R = {("Atlanta", "Georgia", 50);
    ("Seattle", "Washington", 80);
    ("San Francisco", "California", 10)}

def out1 = spread[:even, R, decimal[64, 3, 31.0]]
("Atlanta", "Georgia", 50, 10.334)
("San Francisco", "California", 10, 10.333)
("Seattle", "Washington", 80, 10.333)

def out2 = spread[:even, R, 31]
("Atlanta", "Georgia", 50, 11)
("San Francisco", "California", 10, 10)
("Seattle", "Washington", 80, 10)

Definition

@inline
def spread[mode, R, val](v...) =
(
    (Int(val) and mode = :even and int_spread_by_even[R, val](v...)) or
    (Int(val) and mode = :ratio and int_spread_by_ratio[R, val](v...)) or
    (
        rel_primitive_decimal_type(val, decimal_bit_length[val], decimal_precision[val]) and
        FixedDecimal_spread[mode, R, val](v...)
    )
)

sqrt

sqrt[x]

The non-negative square root of x.

Defined for non-negative x.

Definition

@inline
def sqrt[x] = rel_primitive_sqrt[x]

squared

squared[R]

Square of a relation: the value of each last element is squared.

Examples:

def example = {(1, 2); (3, 4)}
equal(square[example], {(1, 4); (3, 16)}

Definition

@inline
def squared[F][x...] = F[x...] ^ 2

squared_deviation

squared_deviation[R]

Squared deviation: squared deviation from the mean of (the last argument of) the relation R.

Definition

@inline
def squared_deviation[F][x...] = (F[x...] - mean[F]) ^ 2

string

string[x]

Convert ints, floats, dates, periods, to strings:

Examples:

string[1] = "1"
string[3.4] = "3.4"
string[unix_epoch] = "1970-01-01T00:00:00"
string[Hour[1]] = "1 hour"
string["a"] = "a"
string[:a] = "a"

Definition

@inline
def string = rel_primitive_string

String

String(x)

Holds if x is a String. Strings are specified by double quotes, for example, "abc".

Examples:

Integrity constraint that tests whether x is of type String:

def R = "foo"

ic string_type_check(x in R) {
    String(x)
}

Schema defined in a relation using String:

def myrelation(x in String, y in Int) {
    x = "abc" and y = 123
}

def output = myrelation
//output> (abc, 123)

Definition

@inline
def String = rel_primitive_String

string_join

string_join[separator, R]

Concatenate together all the String values in the second argument of the binary relation R, if non-empty, separated by the separator string, following the ordering provided by the Ints in the first argument of R. If R is empty, string_join[sep, R] is false.

R is required to be a binary relation, containing only (Int, String) tuples, where the Int serves as the sort key for the concatenation. Otherwise, string_join[sep, R] is false. (Note: this restriction may be lifted in the future.)

Example:

string_join[", ", {(1, "a"); (2, "b"); (3, "c")}] = "a, b, c"
string_join[" ", {(1, "hello"); (2, "world")}] = "hello world"
string_join["->", sort[{"x"; "y"; "z"}]] = "x->y->z"

Definition

@inline
def string_join[sep, R] =
    rel_primitive_reduce_noinit[_str_join2[sep], R],
    // R is required to be a Binary relation with numeric keys to provide an ordering
    R ⊆ (Int, String)

// This helper function is reduced over the values in R, with the provided separator.
@inline
def _str_join2[sep, a,b] = concat[concat[a, sep], b]

string_length

string_length[string]

DEPRECATED

This function is deprecated. Please prefer num_chars[string] or num_bytes[string], depending on your need.

Example:

string_length["foo"] = 3

Definition

@inline
def string_length = num_chars

string_replace

string_replace[string, old_str, new_str]

Replace a string or character in string with the specific string or character.

Examples:

string_replace["One Two Three", "One", "1"] = "1 Two Three"
string_replace["Rel", 'R', 'r'] = "rel"

Definition

@inline
def string_replace = rel_primitive_replace

string_replace_multiple

string_replace_multiple[input, R]

Replaces all occurrences of old string in input with the corresponding new string, for each tuple (old, new) in relation R.

Examples:

string_replace_multiple["One Two Three", {("One", "1"); ("Two", "2")}]
// "1 2 Three"
string_replace_multiple["One Two Three", {('O', 'o'); ('T', 't')}]
// "one two three"

The behavior is undefined if new generates more occurrences of old.

Definition

@inline
def string_replace_multiple[input, R] = rel_primitive_reduce[(x,y,z: z=string_replace[x,y,R[y]]), input, (x: R(x,_))]

string_split

string_split[delimiter, text]

Split the string text into the substrings delimited by delimiter (and the start/end of text). The expression string_split[delimiter, string] evaluates to a relation of the format (index, sub_string) where index is the substring’s position of type Int and sub_string is the matched substring of type String. delimiter can be a String or a Char.

Example:

string_split['@', "user@email.com"] = {(1, "user"); (2, "email.com")}
string_split[" ", "a b  c"] = {(1, "a"); (2, "b"); (3, ""); (4, "c")}

Definition

@inline
def string_split(dlm, text, i, sub_string) =
    dlm_string = string[dlm] and
    _delims(dlm_string, text, i, pos) and
    _delims(dlm_string, text, i + 1, next_pos) and
    substring[text, pos + num_chars[dlm_string], next_pos - 1](sub_string) and
    {String; Char}(dlm)
    from dlm_string, pos, next_pos

// Helper gives us the positions of all delimeters in string `s`.
@inline
def _delims[dlm in String, s in String] =
    sort[
        -num_chars[dlm] + 1;
        first[regex_match_all[escape_regex_metachars[dlm], s]];
        num_chars[s] + 1
    ]

string_trim

string_trim[s]

To remove leading and tailing whitespaces in given string s

Example:

string_trim["  Good Day!!!  "] = "Good Day!!!"

Definition

@inline
def string_trim[s] = string_replace[s, regex_compile["^\\s+|\\s+$"], ""]

subset

subset(R, S)
R ⊆ S

R is a subset of S if all facts in R are also in S.

Subset is the relational variant of implication, but note that there is a subtle difference for arity-overloaded relations due to usage of varargs.

  • forall(x, y: R(x, y) implies S(x, y)) is true if the implication is true for all binary facts of R. If R also contains facts of a different arity, then the implication remains true, even when R ⊈ S.
  • subset(R, S), defined as forall(xs... where R(xs...) : S(xs...)), is true only if there exists no fact in R, of any arity, that is not also in S.

Definition

@inline
def subset(R, S) = forall(x... where R(x...): S(x...))

@inline
def (⊆) = subset

substring

substring[string, index1, index2]

Substring of a string, selecting characters in range [index1:index2] (both inclusive).

The index of the first character is 1.

When index1 and index2 are out of bounds on the same end, or index2 < index1, the empty string is returned. Otherwise, both indexes are clamped in range.

Example:

substring["abcd", 2, 3] = "bc"
substring["abcd", 3, 2] = ""
substring["中文例子", 2, 3] = "文例"
substring["word", 0, 10] = "word"

Definition

@inline
def substring = rel_primitive_substring

substring_bytes

substring_bytes[string, index1, index2]

Extracts substring of a string between the byte range [index1:index2] (both inclusive). The byte index starts with 1.

When index1 and index2 are out of bounds on the same end, or index2 < index1, the empty string is returned. Otherwise, both indexes are clamped in range.

Use substring to extract substrings by character indexes.

Examples:

substring_bytes["abcd", 2, 3] = "bc"
substring_bytes["abcd", 3, 2] = ""
substring_bytes["中文例子", 2, 3] = """��"""
substring_bytes["word", 0, 10] = "word"

Definition

@inline
def substring_bytes = rel_primitive_substring_bytes

subtract

subtract[x, y]
x - y

Subtraction.

Examples:

8 - 5 = 3
subtract[2, 3] = -1
equal(subtract(8, 5, 3), true)

Definition

@inline
def subtract(x, y, z) = rel_primitive_subtract(x, y, z)

@inline
def (-)(x, y, z) = subtract(x, y, z)

suffix_join

suffix_join[R, S]
R :> S

The suffix join (or restriction) of R to S consists of the tuples (x..., y...) in R where the suffix (y...) is in S.

That is, R :> S contains the tuples in R that have a suffix in S.

ExampleNormalized
edge :> nx, y: edge(x, y) and y = n
parent :> femalex, y: parent(x, y) and female(y)
account_balance :> positivex, v: account_balance(x, v) and positive(v)
brand_name :> "Tesla"b, s: brand_name(b, s) and s = "Tesla"

The restriction operator can also be used to select subsets of JSON-like relations:

  • json :> even where json is [ {"a": 1, "b": 2}, {"a": 3, "b": 4, "c": 6} ] has value [ {"b": 2}, {"b": 4, "c": 6} ]
  • json :> (:b, even) where json is [ {"a": 1, "b": 2}, {"a": 3, "b": 4, "c": 6} ] has value [ {"b": 2}, {"b": 4} ]

Definition

@inline
def suffix_join[R, S](x..., y...) = R(x..., y...) and S(y...)

@inline
def (:>) = suffix_join

sum

sum[R]
Σ[R]

Sum of (the last argument of) the relation R, if non-empty. If R is empty, sum[R] is false (empty). To get 0 instead, use left override, as in sum[R] <++ 0.

Example:

def salary = {("John", 10) ; ("Mary", 20); ("Paul", 17) ; ("Peter", 15) }
def output = sum[salary]
62

def member = {("A", "John"); ("B", "Mary") ; ("A", "Paul") ; ("C" , "Peter") }
def department = {"A"; "B"; "C" }
def output = d in department: sum[salary[p] for p in member[d]]
("A", 27)
("B", 20)
("C", 15)

Definition

@inline
def sum[R] = rel_primitive_reduce_noinit[+, R]

sum_int

sum_int[R]

Integer sum of (the last argument of) the relation R, which should contain integers. sum_int[R] = 0 for empty R.

Definition

@inline
def sum_int[R] = rel_primitive_reduce[+, 0, R]

superset

superset(R, S)
R ⊇ S

Inverse of subset. See subset.

Definition

@inline
def superset(R, S) = forall(x... where S(x...): R(x...))

@inline
def (⊇) = superset

tan

tan[x]

Tangent of x (given in radians).

Defined for non-infinite x.

Example:

tan[pi_float64/4] = 0.9999999999999999

Definition

@inline
def tan[x] = rel_primitive_tan[x]

tanh

tanh[x]

Hyperbolic tangent.

Definition

@inline
def tanh[x] = rel_primitive_tanh[x]

top

top[k, R]

Select the top k facts of relation R according to the sort order of R.

top is sort restricted to first k tuples. top takes a relation R(xs...) and produces a relation R(i, xs...) where i is an Int64 index starting at 1. The index indicates the ordering of the tuples of R.

If k is bigger than number of tuples in the relation R, then top[k,R] is the sorted relation itself. This means that the cardinality is the minimum of k and the cardinality of R.

Example:

def output = top[2, {'a'; 'b'; 'c'; 'd'}]
(1, 'a')
(2, 'b')

Note: Currently top[k, R] only supports groupbys in the R-expression but not in the k-expression. For example, the following is supported:

def output[g] = top[K, R[g]]  // `g` here is a groupby for `R`

However the following is NOT supported:

def output[g] = top[K[g], R]  // `g` here is a groupby for `K`

Definition

@inline
def top[K, R] = rel_primitive_top[R, K]

total

total(D, F)

Given a domain D and a relation F, check that there is a value in F for every element of the domain D. The arity of F must be the arity of D + 1.

Definition

@inline
def total(D, F) =
    forall(k... where D(k...): F(k..., _))

transpose

transpose[R]

Transpose a binary relation (swap the two arguments).

Example:

def output = transpose[{(1, 2); (3, 4)}]
(2, 1)
(4, 3)

Definition

@inline
def transpose[R](y, x) = R(x, y)

trunc

trunc[x]

Round toward zero to the nearest integer. Result has the same type as the argument.

Examples:

trunc[3.9] = 3.0
trunc[-4.9] = -4.0
trunc[4] = 4

Definition

@inline
def trunc[x] = round[:ROUND_TO_ZERO, x]

trunc_divide

trunc_divide[x, y]

Integer Division.

Quotient from Euclidean division. Computes x/y rounded towards zero.

Is false (empty) when y = 0 for Integer arguments.

Examples:

trunc_divide[5, 2] = 2
trunc_divide[-5, 2] = -2
trunc_divide[5.1, 2] = 2.0
empty(trunc_divide[5, 0])

Definition

@inline
def trunc_divide[x, y] = rel_primitive_trunc_divide[x, y]

@inline
def (÷)(x, y, z) = trunc_divide(x, y, z)

trunc_to_int

trunc_to_int[x]

General float-to-int conversion, truncating. Argument must be a float.

Examples:

trunc_to_int[3.1] = 3
trunc_to_int[-3.1] = -3
trunc_to_int[3] : error

Definition

@inline
def trunc_to_int[x] = float_int_convert[trunc[x]]

uint

uint[n, v]

The n-bit unsigned integer value from the integer value v.

Definition

@inline
def uint = rel_primitive_uint

uint128

uint128[v]

The 128-bit unsigned integer value from the integer value v.

Definition

@inline
def uint128 = uint[128]

uint128_hash_value_convert

uint128_hash_value_convert[v]

Convert a provided UInt128 value v to the HashValue type. This is an internal-facing function, used for the implementation of hashing.

Definition

@inline
def uint128_hash_value_convert[v] = rel_primitive_uint128_hash_value_convert[v]

uint128_uint64_truncate

uint128_uint64_truncate[v]

Truncates v of type UInt128 to a UInt64 number.

Truncates the high order bits in v and converts the remaining bits to UInt64. Truncate always succeeds. For values larger than 2^64, consists of only the lower bits.

Examples:

uint128_uint64_truncate[uint128[1]] = 1
uint128_uint64_truncate[uint128[2] ^ 64 + uint128[1]] = 1

Definition

@inline
def uint128_uint64_truncate = rel_primitive_uint128_uint64_truncate

uint64

uint64[v]

The 64-bit unsigned integer value from the integer value v.

Definition

@inline
def uint64 = uint[64]

union

union[R, S]
R ∪ S

Union two n-ary relations R and S

Definition

@inline
def union[R, S](x...) = R(x...) or S(x...)

@inline
def (∪) = union

unit_normalization

unit_normalization[R]

Unit vector normalization: xn=xn/(Σ(x2))0.5x_n = x_n / ( Σ (x^2) ) ^ {0.5}

Definition

@inline
def unit_normalization[R][x...] = R[x...] / sqrt[sum[squared[R]]]

unix_epoch

unix_epoch

The beginning of the UNIX Epoch, midnight on 1 January 1970 (UTC).

Definition

@inline
def unix_epoch = parse_datetime["1970-01-01 00 UTC", "Y-m-d H Z"]

unpack

unpack(v, x₁, ..., xₙ)

Unpacks a packed structure v into its arguments x₁, ..., xₙ.

unpack cannot be used with point-free syntax.

Direct usage of unpack should be avoided, as it serves primarily as a low-level interface for value types.

Definition

@inline
def unpack(v, xs...) = rel_primitive_pack_logical(xs..., v)

UnsignedInt

UnsignedInt(nbits, x)

Holds if x is an unsigned integer of the bit length nbits.

Example:

Integrity constraint that tests whether x is a 32-bit unsigned integer:

def my_unsigned_int = uint[32, 1234]

ic my_unsigned_ic(x) {
    my_unsigned_int(x) implies UnsignedInt(32, x)
}

Definition

@inline
def UnsignedInt = rel_primitive_UnsignedInt

uppercase

uppercase[string_or_char]

A string where all the characters are converted to uppercase. If a character is already uppercase or has no uppercase version, it remains unchanged.

Example:

def output = uppercase["aB1c"]
"AB1C"
def output = uppercase['â']
'Â'

uppercase does not take a locale option and does not handle local-specific case mapping rules.

Definition

@inline
def uppercase = rel_primitive_uppercase

uuid_string

uuid_string[v]

Convert a UInt128 value to the standard UUID format

Example:

uuid_string[0x22b4a8a1e5484eeb927060426d66a48e] = "22b4a8a1-e548-4eeb-9270-60426d66a48e"

Definition

@inline
def uuid_string[v in UInt128] = rel_primitive_uuid_string[v]

@inline
def uuid_string[v in Hash] = rel_primitive_uuid_string[hash_value_uint128_convert[v]]

Week

Week[n]

A period that spans n weeks. The period is given in units of Week.

Example:

datetime_add[unix_epoch, Week[10]]

Definition

@inline
def Week = rel_primitive_week

weighted_mean

weighted_mean[R, W]

The weighted mean of values in relation R with weights defined by W. The last attribute of the relation representing the weights is used to compute the weighted mean. When the arity of the relation that represents the weights is different from the arity of the input relation the user is expected to align the arities before passing them to weighted_mean.(See examples 2 and 3)

Example 1 (The arity of both R and W are the same):

def inputs = {(1,'a',10); (2,'b',10); (3,'c',30); (4,'d',40); (5,'e',50)}

def weights = {(1,'a',11); (2,'b',12); (3,'c',13); (4,'d',14); (5,'e',15)}

def output = weighted_mean[inputs, weights]

29.692307692307693

Example 2 (The arity of R and W are different):

def inputs = {(1,'a',10); (2,'b',10); (3,'c',30); (4,'d',40); (5,'e',50)}

def weights = {(1,11); (2,12); (3,13); (4,14); (5,15)}

def output = weighted_mean[inputs, (n: _, weights[n])]

29.692307692307693

Example 3 (The arity of R and W are different):

def inputs = {(1,'a',100,10); (2,'a',200,10); (3,'b',300,30); (4,'c',400,40); (5,'b',500,50)}

def weights = {(1,'a',11); (2,'a',12); (3,'b',13); (4,'c',14); (5,'b',15)}

def output = weighted_mean[input, (n, c: _, weights[n,c])]

29.692307692307693

Definition

@inline
def weighted_mean[R,W] = sum[k...: R[k...] * W[k...]] / sum[domain[R] <: W]

xor

F xor G
F ≢ G
F ⇎ G
F ⊻ G

Exclusive OR, for boolean (arity 0, true or false) arguments F and G.

Definition

@inline
def xor(F, G) = not(F ≡ G)

@inline
def (⊻)(F, G) = F xor G

@inline
def (≢)(F, G) = F xor G

@inline
def (⇎)(F, G) = F xor G

Year

Year[n]

A period that spans n years. The period is given in units of Year.

Example:

datetime_add[unix_epoch, Year[10]]

Definition

@inline
def Year = rel_primitive_year

zip

zip[R, S]

Pair-wise combine two n-ary tuples into a relation of n pairs.

Example:

def foods = {(:hamburger, :chicken, :hot_dog, :fries, :macaroni, :pizza, :salad, :milk, :ice_cream)}
def cost = { (2.49, 2.89, 1.50, 1.89, 2.09, 1.99, 2.49, 0.89, 1.59) }
def output = zip[foods, cost]
(:chicken, 2.89)
(:fries, 1.89)
(:hamburger, 2.49)
(:hot_dog, 1.5)
(:ice_cream, 1.59)
(:macaroni, 2.09)
(:milk, 0.89)
(:pizza, 1.99)
(:salad, 2.49)

Definition

@inline def zip[R,S](x, y) = rel_primitive_pivot[R](i, x) and rel_primitive_pivot[S](i, y) from i

¬

not F
¬F

Logical negation, for boolean (arity 0, true or false) argument F.

Definition

@inline
def (¬)(F) = not F

Σ

sum[R]
Σ[R]

Sum of (the last argument of) the relation R, if non-empty. false if R is empty.

Example:

d in department: Σ[salary[p] for p in member[d]]

Definition

@inline
def Σ = sum

F implies G
F ⇒ G
G ⇐ F

Logical implication, for boolean (arity 0, true or false) arguments F and G.

Definition

@inline
def (⇒)(F, G) = F implies G

@inline
def (⇐)(F, G) = G implies F

F and G
F ∧ G

Logical and (conjunction).

Definition

@inline
def (∧)(F, G) = F and G

F or G
F ∨ G

Logical or (disjunction), for boolean (arity 0, true or false) arguments F and G.

Definition

@inline
def (∨)(F, G) = F or G