The Standard Library (stdlib)
Broad collection of Rel relations that perform essential and commonly used tasks.
abs
abs[x]
The absolute value of x.
Examples:
abs[-2] = 2
abs[-2.0] = 2.0
Definition
@inline
def abs[x] = rel_primitive_abs[x]
acos
acos[x]
Inverse cosine (in radians).
Defined for x
between -1 and 1 (inclusive).
Example:
acos[0] = 1.5707963267948966
Definition
@inline
def acos[x] = rel_primitive_acos[x]
acosh
acosh[x]
Inverse hyperbolic cosine.
Defined for x
>= 1.
Definition
@inline
def acosh[x] = rel_primitive_acosh[x]
acot
acot[x]
Inverse cotangent (in radians).
Definition
@inline
def acot[x] = rel_primitive_acot[x]
add
add[x, y]
x + y
Addition.
Examples:
1 + 3
is an expression with value 4add[1, 3]
is an expression with value 4add(1, 3, 4)
is a formula that istrue
.
1 + 3 = 4
add[1, 3] = 4
equal(add(1, 3, 4), true)
Definition
@inline
def add(x, y, z) = rel_primitive_add(x, y, z)
@inline
def (+)(x, y, z) = add(x, y, z)
Any
Any(x)
Holds for any x
, where x
exists. (Any
functions as a wildcard.)
Example:
Integrity constraint that tests whether x
is of any type:
def R = (1, 3) ; (1, "foo")
ic any_ic {subset(R, (Int, Any) )}
Definition
@inline
def Any(x) = true
argmax
argmax[R]
Find the keys for the largest value of the relation R.
Examples:
def output = argmax[{(2, 3); (1, 6)}]
1
def output = argmax[{(2, 6); (1, 6); (5, 0)}]
1
2
// find the teams with the largest aggregated salary:
argmax[d in team: sum[salary[p] for p in member[d]]]
Definition
@inline
def argmax[R] = R.(max[R])
ArgMax
ArgMax[R]
Please use argmax[R]. Deprecates in near future
Definition
@inline
def ArgMax = argmax
argmin
argmin[R]
Find the keys for the smallest value of the relation R.
Examples:
def output = argmin[{(2, 3); (1, 6)}]
2
def output = argmin[{(2, 6); (1, 6); (5, 10)}]
1
2
// find the teams with the smallest aggregated salary:
argmin[d in team: sum[salary[p] for p in member[d]]]
Definition
@inline
def argmin[R] = R.(min[R])
ArgMin
ArgMin[R]
Please use argmin[R]. Deprecates in near future
Definition
@inline
def ArgMin = argmin
arity
arity[R]
The arity of a relation. In some cases, it can be an over-approximation.
Arity is a higher-order relation that is always evaluated at compile-time.
Examples:
def output = arity[3]
1
def output = arity[{1; 2; 3}]
1
def output = arity[(1, 2)]
2
def output = arity[add]
3
def output = arity[{1; 2; (1,2)}]
1
2
Arity can be used to do meta-programming in logic. For example, the following abstraction verbalize
implements specific cases using arity
.
Examples:
@inline def verbalize[R] = "nullary", arity[R] = 0;
"unary", arity[R] = 1;
"binary", arity[R] = 2
def output = verbalize[true]
"nullary"
def output = verbalize[1]
"unary"
Arity can be used in higher-order abstractions to check at compile-time that they are used correctly.
Arity can be used in integrity constraints to state expectation on EDB or IDB relations. Because arity
is evaluated at compile-time, it can catch mistakes in the logic before the logic executes.
Example:
def p = 1,2,3
ic { arity[p] = 3 }
Note that there is a difference between R(_, _)
and arity(R) = 2
. The first requires R
to be non-empty, which is an runtime property of R
.
Definition
@inline
def arity[R] = rel_primitive_arity[R]
asin
asin[x]
Inverse sine (in radians).
Defined for x
between -1 and 1 (inclusive).
Example:
asin[1] = 1.5707963267948966
Definition
@inline
def asin[x] = rel_primitive_asin[x]
asinh
asinh[x]
Inverse hyperbolic sine.
Definition
@inline
def asinh[x] = rel_primitive_asinh[x]
atan
atan[x]
Inverse tangent (in radians).
Defined for x
between -1 and 1 (inclusive).
Definition
@inline
def atan[x] = rel_primitive_atan[x]
atan2
atan2[y, x]
Inverse tangent: an angle (in radians) between the positive x-axis and the ray to the point (x, y).
Definition
@inline
def atan2[x,y] = rel_primitive_atan2[x,y]
atanh
atanh[x]
Inverse hyperbolic tangent.
Definition
@inline
def atanh[x] = rel_primitive_atanh[x]
auto_number
auto_number[R]
DEPRECATED
This function is deprecated and should be avoided. It will be removed soon.
AutoNumber a relation.
auto_number
takes a relation R(xs...)
and produces a relation R(xs..., i)
where i
is a distinct AutoNumberValue
index.
Note that auto_number
can give different results each time it is called.
Example:
def sample = 'a'
def output = auto_number[sample]
('a', AutoNumberValue(0x12))
def sample = {'a'; 'b'; 'c'}
def output = auto_number[sample]
('a', AutoNumberValue(0x132))
('b', AutoNumberValue(0x133))
('c', AutoNumberValue(0x134))
Definition
@inline
def auto_number = rel_primitive_auto_number
AutoNumber
AutoNumber(x)
DEPRECATED
This function is deprecated and should be avoided. It will be removed soon.
Test if the given value is an AutoNumberValue.
Definition
@inline
def AutoNumber = rel_primitive_AutoNumberValue
average
mean[R]
average[R]
The arithmetic mean of (the last argument of) the relation R, if non-empty. False if R is empty.
Definition
@inline
def average = mean
bitwise_and
bitwise_and[x, y]
Bitwise and
of two integers.
Example:
bitwise_and[3,2] = 2
bitwise_and[0x10011, 0x11100] = 0x00010000
Definition
@inline
def bitwise_and[x, y] = rel_primitive_bitwise_and[x, y]
bitwise_left_shift
bitwise_left_shift[x, n]
Bitwise left shift of an integer x
by n
bits.
Examples:
bitwise_left_shift[8, 1] = 16
bitwise_left_shift[1, 10] = 1024
bitwise_left_shift[0xF, 1] = 0x1d
Definition
@inline
def bitwise_left_shift[x, n] = rel_primitive_bitwise_left_shift[x, n]
bitwise_not
bitwise_not[x]
Bitwise not
of an integer.
Examples:
bitwise_not[8] = -9
bitwise_not[-9] = 8
bitwise_not[0x00011] = 0xffffffee
Definition
@inline
def bitwise_not[x] = rel_primitive_bitwise_not[x]
bitwise_or
bitwise_or[x, y]
Bitwise or
of two integers.
Example:
bitwise_or[3, 2] = 3
bitwise_or[0x00011, 0x11100] = 0x00011111
Definition
@inline
def bitwise_or[x, y] = rel_primitive_bitwise_or[x, y]
bitwise_right_shift
bitwise_right_shift[x, n]
Bitwise right shift of an integer x
by n
bits while preserving the sign.
Examples:
bitwise_right_shift[1024, 1] = 512
bitwise_right_shift[-1024, 1] = -512
Definition
@inline
def bitwise_right_shift[x, n] = rel_primitive_bitwise_signed_right_shift[x, n]
bitwise_unsigned_right_shift
bitwise_unsigned_right_shift[x,n]
Bitwise right shift of an integer by n bits.
Examples:
bitwise_unsigned_right_shift[8, 1] = 4
bitwise_unsigned_right_shift[-8, 1] = 9223372036854775804
Definition
@inline
def bitwise_unsigned_right_shift[x, n] = rel_primitive_bitwise_unsigned_right_shift[x, n]
bitwise_xor
bitwise_xor[x, y]
Bitwise xor
of two integers.
Example:
bitwise_xor[3, 2] = 1
bitwise_xor[0x00011, 0x11100] = 0x00011111
Definition
@inline
def bitwise_xor[x, y] = rel_primitive_bitwise_xor[x, y]
Boolean
Boolean(x)
Holds if x
is a Boolean
.
Example:
def json = parse_json["""{"a": true, "b": false}"""]
def output(x) = json(:a, x) and Boolean(x)
Definition
@inline
def Boolean = rel_primitive_Boolean
boolean_and
boolean_and(x, y, z)
Logical AND operator for the Boolean
data type.
Example:
def output(x, y, z) = boolean_and(x, y, z) and boolean_true(z)
Definition
@inline
def boolean_and(x in Boolean, y in Boolean, z in Boolean) =
if boolean_true(x) and boolean_true(y) then
boolean_true(z)
else
boolean_false(z)
boolean_false
boolean_false(x)
Holds if x
is a Boolean
of value false
.
Definition
@inline
def boolean_false = rel_primitive_boolean_false
boolean_not
boolean_not(x,y)
Negation(not
) operator for the Boolean
data type. Example:
def output(x, y) = boolean_not(x, y) and boolean_false(x)
Definition
@inline
def boolean_not(x in Boolean, y in Boolean) =
if boolean_true(x) then
boolean_false(y)
else
boolean_true(y)
boolean_or
boolean_or(x, y, z)
Logical or
operator for the Boolean
data type. Example:
def output(x, y, z) = boolean_or(x, y, z) and boolean_false(z)
Definition
@inline
def boolean_or(x in Boolean, y in Boolean, z in Boolean) =
if boolean_false(x) and boolean_false(y) then
boolean_false(z)
else
boolean_true(z)
boolean_true
boolean_true(x)
Holds if x
is a Boolean
of value true
.
Definition
@inline
def boolean_true = rel_primitive_boolean_true
bottom
bottom[k, R]
Select the bottom k
facts of relation R
according to the sort order of R.
bottom
is reverse_sort
(which sorts highest to lowest) restricted to the first k
facts.
Example:
def output = bottom[2, {'a'; 'b'; 'c'; 'd'}]
(1, 'd')
(2, 'c')
Definition
@inline
def bottom[k, R] =
reverse_sort[R][i] for i where i <= k
byte
byte[str]
byte[str, i]
byte(str, i, b)
Indexes into a string at byte position i
, mapping each position i
to a byte b
, as a UInt8 value.
If a string contains Unicode characters, the byte at index i
might be only a partial character. Be careful with your indexing logic.
Both i
and b
can be optionally bound externally. When only str
is bound, this is the mapping from each index to its corresponding byte.
Examples: Indexing into a known byte index:
byte["abcd", 2] = 0x62
byte["中文例子", 2] = 0xb8
Abstracting over the byte index:
equal(byte["中文"],
{ 1, 0xe4;
2, 0xb8;
3, 0xad;
4, 0xe6;
5, 0x96;
6, 0x87; })
equal((i : byte["awesome", i](0x65)), {3; 7})
Definition
@inline
def byte[s](i, b) =
rel_primitive_byte(s, i, b) and range[1, num_bytes[s], 1](i)
capture_group_by_index
capture_group_by_index[regex, input_string, offset]
A set of capture groups, each of the form (index, substring)
, where index
is the capture group index, and substring
is the first regex match in input_string
, starting at the character index specified by offset
. regex
can be a string or a pattern.
Offsets (character indexes) start at 1.
Example:
capture_group_by_index["(\\d+):(\\d+)",
"Appointment from 12:45 to 1:30",
19]
is equal to
{(1, "12"); (2, "45")}
Definition
@inline
def capture_group_by_index[re, str, offset](idx, capture) =
rel_primitive_capture_grp[regex_compile[re], str, offset](idx, capture)
capture_group_by_name
capture_group_by_name[regex, input_string, offset]
A set of capture groups, each of the form (name, substring)
, where name
is the capture group name, and substring
is the first regex match in input_string
, starting at the character index specified by offset
. regex
can be a string or a pattern.
Offsets (character indexes) start at 1.
Each capture group should have a unique name.
Example:
capture_group_by_name["(?<hour>\\d+):(?<minute>\\d+)",
"Appointment from 12:45 to 1:30",
19]
is equal to
(("hour","12"), ("minute","45"))
Definition
@inline
def capture_group_by_name[re, str, offset](name, capture) =
exists( idx: rel_primitive_capture_grp[regex_compile[re], str, offset](idx, capture)
and rel_primitive_capture_names(regex_compile[re], idx, name))
cart
cart[R, S]
R × S
Cartesian product.
Examples:
def output = 1 ✕ 2
(1,2)
def output = {1; 2} ✕ {3; 4}
(1,3)
(1,4)
(2,3)
(2,4)
Definition
@inline
def cart[R, S](x..., y...) = R(x...) and S(y...)
@inline
def (×) = cart
cbrt
cbrt[x]
The cube root of x.
Example:
cbrt[27] = 3.0
Definition
@inline
def cbrt[x] = rel_primitive_cbrt[x]
ceil
ceil[x]
Round up to the nearest integer.
Examples:
ceil[4.5] = 5.0
ceil[-4.5] = -4.0
ceil[4] = 4
Definition
@inline
def ceil[x] = round[:ROUND_UP, x]
char
char[str]
char[str, i]
char(str, i, c)
Indexes into a string at (Int) position i
, mapping each index i
to the i
-th character, c
.
Since this is indexed using character positions, the characters will always be whole Unicode characters.
A character is also known as a “Code Point” in the Unicode specification.
Both i
and c
can be optionally bound externally. When only str
is bound, this is the mapping from each character index to its corresponding character.
Examples:
Indexing into a known character index:
char["abcd", 2] = 'b'
char["中文例子", 2] = '文'
Abstracting over the character index:
equal(char["中文"],
{ 1, "中"; 2, "文" })
equal((i : char["awesome", i]('e')), {3; 7})
Definition
@inline
def char[s](i, c) =
rel_primitive_char(s, i, c) and range[1, num_chars[s], 1](i)
Char
Char(x)
Holds if x
is of type Char
, which has a Unicode character as its value and is specified with single quotes.
Examples:
Integrity constraint that tests whether x
is of type Char
:
def R = 't'
ic mychar_ic(x in R)
Char(x)
}
Schema defined in a relation using Char
:
def myrelation(x in Char, y in Int) {
x = 'a' and y = 123
}
def output = myrelation
//output> (a, 123)
Definition
@inline
def Char = rel_primitive_Char
clamp
clamp[lo, hi, v]
clamp limits the value v
to a given range(lo, hi), changing values outside the range to the closest lo
, hi
value if necessary. The arguments should have the same type.
Examples:
clamp[2, 5, 1] = 2
clamp[2, 5, 6] = 5
clamp[2, 5, 3] = 3
Definition
@inline
def clamp[lo, hi, v] = min[{max[{lo; v}]; hi}]
complement
complement[R]
The complement of the relation R.
Definition
@inline
def complement[R](x...) = not R(x...)
concat
concat[val1, val2]
String concatenation of two arbitrary values
Example:
concat["a", "b"] = "ab"
concat["a", 'b'] = "ab"
concat['a', "b"] = "ab"
concat['a', 'b'] = "ab"
concat["a_", 1] = "a_1"
concat[1, 3.14] = "13.14"
Definition
@inline
def concat[x, y] = rel_primitive_concat[string[x], string[y]]
cos
cos[x]
Cosine of x (given in radians).
Defined for non-infinite x
.
Example:
cos[pi_float64] = -1.0
Definition
@inline
def cos[x] = rel_primitive_cos[x]
cosh
cosh[x]
Hyperbolic cosine.
Definition
@inline
def cosh[x] = rel_primitive_cosh[x]
cot
cot[x]
Cotangent of x (given in radians).
Definition
@inline
def cot[x] = rel_primitive_cot[x]
count
count[R]
Number of tuples in R, if R is not empty. If R is empty, count[R]
is false
.
Examples:
count[employee]
is the number of tuples in relationemployee
d in department: count[d.member]
is the member count for every department with at least 1 membercount[5]
is 1 because5
is a singleton relationcount[{}]
isfalse
(empty). To get 0 instead, use left override, as incount[R] <++ 0
.
Definition
@inline
def count[R] = sum[{R,1}]
Date
Date(x)
Holds if x
is a Date
.
Definition
@inline
def Date = rel_primitive_Date
date_add
date_add[d, period]
d + period
Add a Period to a Date
Example:
def d = parse_date["2021-09-21", "Y-m-d"]
def output = date_add[d, Day[20]]
Dates.Date("2021-10-11")
Definition
@inline
def date_add = rel_primitive_add_date_period
date_day
date_day[d]
Day of the month for a Date, as an integer between 1 and 31, inclusive.
Example:
def d = parse_date["2014-10-31", "Y-m-d"]
date_day[d] = 31
Definition
@inline
def date_day = rel_primitive_date_day
date_dayname
date_dayname[d]
Name of week-day (a String, e.g., Friday
)
Example:
def t = parse_date["2014-01-31", "Y-m-d"]
def output = date_dayname[t]
"Friday"
Definition
@inline
def date_dayname = rel_primitive_date_dayname
date_dayofquarter
date_dayofquarter[d]
Day of quarter
Example:
def d = parse_date["2014-01-31", "Y-m-d"]
date_dayofquarter[d] = 31
Definition
@inline
def date_dayofquarter = rel_primitive_date_dayofquarter
date_dayofweek
date_dayofweek[d]
Day of the week for a date, as an integer between 1 and 7, where 1 is Monday and 7 is Sunday. (That is, 5
for Friday
.)
Example:
def d = parse_date["2014-01-31", "Y-m-d"]
date_dayofweek[d] = 5
Definition
@inline
def date_dayofweek = rel_primitive_date_dayofweek
date_dayofweekofmonth
date_dayofweekofmonth[d]
Day of week of month, as an integer (e.g. 2 for second Friday of the month, 1 for the first Tuesday).
Example:
def t = parse_date["2014-01-31", "Y-m-d"]
date_dayofweekofmonth[t] = 5 // fifth Friday of that month
Definition
@inline
def date_dayofweekofmonth = rel_primitive_date_dayofweekofmonth
date_dayofyear
date_dayofyear[d]
Day of year
Example:
def t = parse_date["2014-01-31", "Y-m-d"]
def output = date_dayofyear[t]
31
Definition
@inline
def date_dayofyear = rel_primitive_date_dayofyear
date_daysinmonth
date_daysinmonth[d]
The number of days in the month for date d
Examples:
def d1 = parse_date["2014-01-30", "Y-m-d"]
date_daysinmonth[t] = 31
def d2 = parse_date["2014-02-11", "Y-m-d"]
date_daysinmonth[d2] = 28
def d = parse_date["2016-02-11", "Y-m-d"]
date_daysinmonth[d] = 29
Definition
@inline
def date_daysinmonth = rel_primitive_date_daysinmonth
date_isleapyear
date_isleapyear[d]
True iff the year for date d
is a leap year.
Examples:
def notleap = parse_date["2014-01-31", "Y-m-d"]
def leap = parse_date["2016-01-31", "Y-m-d"]
def output = date_isleapyear[notleap]
false
def output = date_isleapyear[leap]
true
Definition
@inline
def date_isleapyear = rel_primitive_date_isleapyear
date_month
date_month[d]
Month of a Date, as an integer between 1 and 12.
Definition
@inline
def date_month = rel_primitive_date_month
date_monthname
date_monthname[d]
The month name for date d
, as a string.
Example:
def t = parse_date["2014-01-31", "Y-m-d"]
date_monthname[t] = "January"
Definition
@inline
def date_monthname = rel_primitive_date_monthname
date_quarterofyear
date_quarterofyear[d]
Quarter to year
Example:
def d = parse_date["2014-01-31", "Y-m-d"]
date_quarterofyear[d] = 1
Definition
@inline
def date_quarterofyear = rel_primitive_date_quarterofyear
date_subtract
date_subtract[date, period]
date - period
Subtract a Period from a Date, giving another Date.
Example:
def d = parse_date["2021-09-21", "Y-m-d"]
def output = date_subtract[d, Day[1000]]
Dates.Date("2018-12-26")
Definition
@inline
def date_subtract = rel_primitive_subtract_date_period
date_week
date_week[d]
Week of the year for a Date, as an integer, where the first week is the week that contains the first Thursday of the year. Ranges between 1 and 53, inclusive.
Example:
week[parse_date["2005-01-01", "Y-m-d"]] = 53 // 53rd week of 2004.
week[parse_date["2001-01-01", "Y-m-d"]] = 1
Definition
@inline
def date_week = rel_primitive_date_week
date_year
date_year[d]
Year of a Date, as an integer.
Example:
date_year[parse_date["2020-05-22", "Y-m-d"]]
2020
Year[date_year[parse_date["2020-05-22", "Y-m-d"]]]
(2020 years,)
Definition
@inline
def date_year = rel_primitive_date_year
dates_period_days
dates_period_days[date1, date2]
The difference in days between two dates date2
and date1
as a Day
data type.
Example:
def days = dates_period_days[
2022-05-12,
2022-05-15
]
ic { equal(days, Day[3])}
Definition
@inline
def dates_period_days = rel_primitive_dates_period
DateTime
DateTime(x)
Holds if x
is a DateTime
.
Definition
@inline
def DateTime = rel_primitive_DateTime
datetime_add
datetime_add[dt, period]
dt + period
Add a Period to a DateTime
Examples:
def dt = parse_datetime["2021-01-01 01:00:00", "Y-m-d H:M:S"]
def output = datetime_add[dt, Year[1]]
Dates.DateTime("2022-01-01T01:00:00")
def output = datetime_add[dt, Month[13]]
Dates.DateTime("2022-02-01T01:00:00")
def output = datetime_add[dt, Hour[1000]]
Dates.DateTime("2021-02-11T17:00:00")
Definition
@inline
def datetime_add = rel_primitive_add_datetime_period
datetime_day
datetime_day[dt, tz]
Day of the month, for a DateTime. Note that DateTime properties require a timezone string tz
.
Example:
datetime_day[unix_epoch, "UTC"] = 1
datetime_day[unix_epoch, "UTC-10"] = 31
Definition
@inline
def datetime_day = rel_primitive_datetime_day
datetime_day_UTC
datetime_day_UTC[dt]
Day assuming datetime is in UTC.
Definition
@inline
def datetime_day_UTC[d] = rel_primitive_datetime_day[d, "UTC"]
datetime_dayname
datetime_dayname[t, tz]
Name of week-day (a string, e.g., Friday
)
Example:
def t = parse_datetime["2014-01-31 01:00 +03:00", "YYYY-mm-dd HH:MM zzzz"]
datetime_dayname[t, "+03:00"] = "Friday"
Definition
@inline
def datetime_dayname = rel_primitive_datetime_dayname
datetime_dayofquarter
datetime_dayofquarter[dt, tz]
Day of quarter
Example:
def t = parse_datetime["2014-01-31 01:00 +03:00", "YYYY-mm-dd HH:MM zzzz"]
datetime_dayofquarter[t, "+03:00"] = 31
Definition
@inline
def datetime_dayofquarter = rel_primitive_datetime_dayofquarter
datetime_dayofweek
datetime_dayofweek[dt, tz]
Day of week (a number, e.g., 5
for Friday
)
Example:
def dt = parse_datetime["2014-01-31 01:00 +03:00", "YYYY-mm-dd HH:MM zzzz"]
datetime_dayofweek[dt, "+03:00"] = 5
datetime_dayofweek[dt, "UTC"] = 4
Definition
@inline
def datetime_dayofweek = rel_primitive_datetime_dayofweek
datetime_dayofweekofmonth
datetime_dayofweekofmonth[dt, tz]
Day of week of month, as an integer (e.g. 2 for second Friday of the month, 1 for the first Tuesday).
Example:
def t = parse_datetime["2014-01-31 01:00 +03:00", "YYYY-mm-dd HH:MM zzzz"]
datetime_dayofweekofmonth[t, "+03:00"] = 5
Definition
@inline
def datetime_dayofweekofmonth = rel_primitive_datetime_dayofweekofmonth
datetime_dayofyear
datetime_dayofyear[dt, tz]
Day of year
Examples:
def dt = parse_datetime["2014-03-31 01:00 +03:00", "YYYY-mm-dd HH:MM zzzz"]
datetime_dayofyear[dt, "+03:00"] = 90
datetime_dayofyear[dt, "+00:00"] = 89
Definition
@inline
def datetime_dayofyear = rel_primitive_datetime_dayofyear
datetime_daysinmonth
datetime_daysinmonth[dt, tz]
The number of days in a datetime’s month, adjusting for timezone tz
.
Examples:
def t = parse_datetime["2014-02-28 23:00 +00:00", "YYYY-mm-dd HH:MM zzzz"]
datetime_daysinmonth[t, "+03:00"] = 31
datetime_daysinmonth[t, "-03:00"] = 28
Definition
@inline
def datetime_daysinmonth = rel_primitive_datetime_daysinmonth
datetime_hour
datetime_hour[dt, tz]
Hour of a DateTime. Note that DateTime properties require a timezone string tz
.
Definition
@inline
def datetime_hour = rel_primitive_datetime_hour
datetime_hour_UTC
datetime_hour_UTC[dt]
Hour assuming datetime is in UTC.
Definition
@inline
def datetime_hour_UTC[d] = rel_primitive_datetime_hour[d, "UTC"]
datetime_isleapyear
datetime_isleapyear[dt, tz]
Is it a leap year?
Examples:
def dt = parse_datetime["2014-01-31 01:00 +03:00", "YYYY-mm-dd HH:MM zzzz"]
def output = datetime_isleapyear[dt, "+03:00"]
false
def dtleap = parse_datetime["2016-01-31 01:00 +03:00", "YYYY-mm-dd HH:MM zzzz"]
def output = datetime_isleapyear[dtleap, "+00:00"]
true
Definition
@inline
def datetime_isleapyear = rel_primitive_datetime_isleapyear
datetime_minute
datetime_minute[dt, tz]
Minute of a DateTime. Note that DateTime properties require a timezone string tz
.
Definition
@inline
def datetime_minute = rel_primitive_datetime_minute
datetime_minute_UTC
datetime_minute_UTC[dt]
Minute assuming datetime is in UTC.
Definition
@inline
def datetime_minute_UTC[d] = rel_primitive_datetime_minute[d, "UTC"]
datetime_month
datetime_month[dt, tz]
Month of a DateTime, as an integer between 1 and 12.
Note that DateTime properties require a timezone tz
.
Examples:
def dt = parse_datetime["2021-01-01 01:00:00", "Y-m-d H:M:S"]
datetime_month[dt, "Europe/Berlin"] = 1
datetime_month[dt, "America/New_York"] = 12
datetime_month[dt, "-03:00"] = 12
Definition
@inline
def datetime_month = rel_primitive_datetime_month
datetime_month_UTC
datetime_month_UTC[dt]
Month assuming datetime is in UTC.
Definition
@inline
def datetime_month_UTC[d] = rel_primitive_datetime_month[d, "UTC"]
datetime_monthname
datetime_monthname[dt, tz]
The month name
Example:
def t = parse_datetime["2014-01-31 01:00 +03:00", "YYYY-mm-dd HH:MM zzzz"]
datetime_monthname[t, "+03:00"] = "January"
Definition
@inline
def datetime_monthname = rel_primitive_datetime_monthname
datetime_now
datetime_now
The current UTC time (the start time of the transaction) as a DateTime
value.
Example:
def output = datetime_now
Definition
@inline
def datetime_now = rel_primitive_transaction_edb[:txn_start_time]
datetime_quarterofyear
datetime_quarterofyear[dt, tz]
Quarter of the year for datetime dt
, as a number between 1 and 4.
Example:
def t = parse_datetime["2014-01-31 01:00 +03:00", "YYYY-mm-dd HH:MM zzzz"]
datetime_quarterofyear[t, "+03:00"] = 1
Definition
@inline
def datetime_quarterofyear = rel_primitive_datetime_quarterofyear
datetime_second
datetime_second[dt]
Second of a DateTime. Unlike other DateTime functions, datetime_second
does not require a timezone argument.
Example:
datetime_second[unix_epoch] = 0
Definition
@inline
def datetime_second = rel_primitive_datetime_second
datetime_subtract
datetime_subtract[dt, period]
dt - period
Subtract a Period from a DateTime, giving another DateTime.
Example:
def dt = parse_datetime["2021-01-01 01:00:00", "Y-m-d H:M:S"]
def output = datetime_subtract[dt, Hour[1000]]
Dates.DateTime("2020-11-20T09:00:00")
Definition
@inline
def datetime_subtract = rel_primitive_subtract_datetime_period
datetime_to_nanoseconds
datetime_to_nanoseconds[dt]
Convert datetime to nanoseconds since the epoch. Assumes dt
is in UTC.
Definition
@inline
def datetime_to_nanoseconds(datetime, v) =
rel_primitive_datetimes_period_nanoseconds(unix_epoch, datetime, p) and
rel_primitive_period_nanoseconds(p, v)
from p
datetime_week
datetime_week[dt, tz]
Week of the year for a DateTime dt
, as an integer between 0 and 53 (see date_week
).
Note that DateTime properties require a timezone tz
.
Definition
@inline
def datetime_week = rel_primitive_datetime_week
datetime_week_UTC
datetime_week_UTC[dt]
Week assuming datetime is in UTC.
Definition
@inline
def datetime_week_UTC[d] = rel_primitive_datetime_week[d, "UTC"]
datetime_year
datetime_year[dt, tz]
Year of a DateTime dt
, as an Int
. Note that DateTime properties require a timezone tz
.
Examples:
def dt = parse_datetime["2021-01-01 01:00:00", "Y-m-d H:M:S"]
datetime_year[dt, "Europe/Berlin"] = 2021
datetime_year[dt, "America/New_York"] = 2020
datetime_year[dt, "-03:00"] = 2020
Definition
@inline
def datetime_year = rel_primitive_datetime_year
datetime_year_UTC
datetime_year_UTC[dt]
Year assuming datetime is in UTC.
Definition
@inline
def datetime_year_UTC[d] = rel_primitive_datetime_year[d, "UTC"]
datetimes_period_milliseconds
datetimes_period_milliseconds[dt1, dt2]
The difference between two datetimes, dt2 - dt1
, as a Milliseconds
data type.
In other words, equal to the milliseconds time period between dt1
and dt2
–- which when added to dt1
gives dt2
.
Example:
def output = datetimes_period_milliseconds[
unix_epoch,
parse_datetime["2021-03-19 11:00:40", "YYYY-mm-dd HH:MM:SS"]
]
ic {is_Millisecond(output)}
Definition
@inline
def datetimes_period_milliseconds = rel_primitive_datetimes_period_milliseconds
Day
Day[n]
A period that spans n
days. The period is given in units of Day
.
Example:
datetime_add[unix_epoch, Day[10]]
Definition
@inline
def Day = rel_primitive_day
decimal
decimal[n, digits, v]
The n
-bit decimal value with digits
precision from number value v
.
Definition
@inline
def decimal = rel_primitive_decimal
decimal_bit_length
decimal_bit_length[decimal]
The bit length of a fixed-point decimal number.
Example:
def g = parse_decimal[64, 2, "3.14"]
decimal_bit_length[g] = 64
Definition
@inline
def decimal_bit_length(x, y) = rel_primitive_decimal_type(x, y, _)
decimal_int_convert
decimal_int_convert[x]
Conversion from n
-bit fixed-point decimal with d
precision to n
-bit int.
If the argument is not equivalent to an int, decimal_int_convert
returns false
.
Example:
decimal_int_convert[decimal[32, 2, 3.00]] = 3
decimal_int_convert[decimal[32, 2, 3.20]] = false
Definition
@inline
def decimal_int_convert = rel_primitive_decimal_int_convert
decimal_precision
decimal_precision[decimal]
The precision of a fixed-point decimal number.
Example:
def g = parse_decimal[64, 2, "3.14"]
decimal_precision[g] = 2
Definition
@inline
def decimal_precision(x, y) = rel_primitive_decimal_type(x, _, y)
decode_base16
decode_base16[encoded_str]
Decodes the base-16 encoded string to string, as per rfc4648 specifications.
Encoding/decoding specifications are found at https://datatracker.ietf.org/doc/html/rfc4648.
Example:
def output = decode_base16["48656C6C6F21"]
"Hello!"
def output = decode_base16["E4BDA0E5A5BD"]
"你好"
Definition
@inline
def decode_base16 = rel_primitive_base16_decode
decode_base32
decode_base32[encoded_str]
Decodes the base-32 encoded string to string, as per rfc4648 specifications.
Encoding/decoding specifications are found at https://datatracker.ietf.org/doc/html/rfc4648.
Example:
def output = decode_base32["JBSWY3DPEE======"]
"Hello!"
def output = decode_base32["4S62BZNFXU======"]
"你好"
Definition
@inline
def decode_base32 = rel_primitive_base32_decode
decode_base64
decode_base64[encoded_str]
Decodes the base-64 encoded string to string, as per rfc4648 specifications.
Encoding/decoding specifications are found at https://datatracker.ietf.org/doc/html/rfc4648.
Example:
def output = decode_base64["SGVsbG8h"]
"Hello!"
def output = decode_base64["5L2g5aW9"]
"你好"
Definition
@inline
def decode_base64 = rel_primitive_base64_decode
default_value
default_value[D, F, c]
Make function or relation F
total for domain D
by defaulting to c
.
The arity of F must be the arity of D + 1.
Example:
def dom = {1;2;3;4}
def f = {(1, 321); (3, 123)}
def output = default_value[dom, f, 42]
(1, 321)
(2, 42)
(3, 123)
(4, 42)
See also: left_override
(<++
) and right_override
(++>
).
Definition
@inline
def default_value[D, F, c](k..., v) =
F(k..., v) or (D(k...) and not F(k..., _) and v = c)
deg2rad
deg2rad[x]
Convert degrees to radians.
Definition
@inline
def deg2rad[x] = x * pi_float64 / 180.0
denominator
denominator[x]
Denominator of a rational-like number.
Examples:
denominator[rational[64, 1, 3]] = 3
denominator[rational[64, 1, 100]] = 100
denominator[rational[64, 1, 100] + rational[64, 1, 3]] = 300
denominator[parse_decimal[64, 2, "3.14"]] = 100
denominator[parse_decimal[64, 5, "3.14159"]] = 10000
denominator[5] = 1
Definition
@inline
def denominator = rel_primitive_denominator
describe
describe[R]
A summary of statistics for a dataset R
. R
should be a relation given in the form R(:feature_name, keys..., value)
. For example, data loaded using 0
.
For each feature in the dataset, describe
computes a set of statistics depending on the data type of that feature.
For numerical data, describe
computes the count, mean, standard deviation, minimum, 25th, 50th, 75th percentiles, and maximum for that feature.
For non-numerical data, describe
computes the count, unique count, mode, and mode frequency. It will also compute maximum and minimum values if the data is sortable.
If a dataset has multiple data types, it will print the numerical stats for the numerical subset, and non-numerical stats for the non-numerical subset of the data.
For example, given a dataset R
in the form (produced by the CSV loader):
R = {
(:date, FilePos(29), 2020-01-01);
(:date, FilePos(60), 2020-02-02);
(:date, FilePos(91), 2020-03-03);
(:date, FilePos(127), 2020-04-04);
(:price, FilePos(29), 12.5);
(:price, FilePos(60), 14.25);
(:price, FilePos(91), 11.0);
(:price, FilePos(127), 12.25);
(:quantity, FilePos(29), 2);
(:quantity, FilePos(60), 4);
(:quantity, FilePos(91), 4);
(:quantity, FilePos(127), 3);
(:cocktail, FilePos(29), "martini");
(:cocktail, FilePos(60), "sazerac");
(:cocktail, FilePos(91), "cosmopolitan");
(:cocktail, FilePos(127), "bellini");
}
describe
will compute a summary:
Example:
def d = describe[R]
d[:date]
(:mode, Dates.Date("2020-01-01"))
(:count, 4)
(:mode_freq, 1)
(:min, Dates.Date("2020-01-01"))
(:max, Dates.Date("2020-04-04"))
(:unique, 4)
d[:quantity]
(:max, 4)
(:count, 4)
(:std, 0.9574271077563381)
(:min, 2)
(:percentile75, 4.0)
(:percentile25, 2.75)
(:percentile50, 3.5)
(:mean, 3.25)
A describe_full
method is also provided, which does not aggregate over columns, and instead computes the statistics for the whole dataset. However, describe_full
may cause errors if any unsortable data (e.g., symbols) exists in the relation.
For example, given the relation R
defined above:
Example:
describe_full[R]
(:min, 32)
(:unique, 3)
(:mode, "a")
(:std, 49.47186262580647)
(:percentile25, 89.2175)
(:min, 3.27)
(:percentile50, 53.6)
(:mean, 16.75)
(:mean, 18.8675)
(:count, 7)
(:max, 72.2)
(:max, 35)
(:mode_freq, 1)
Definition
@inline
def describe[R] = column: describe_full[R[column]]
// Stats for all types
@inline def describe_full[R, :count] = count[R]
@inline def describe_full[R, :min] = min[R]
@inline def describe_full[R, :max] = max[R]
// Stats for non-numeric types
@inline def describe_full[R, :unique] = count[last[R :> (x: not Number(x))]]
@inline def describe_full[R, :mode] = mode[R :> (x: not Number(x))]
@inline def describe_full[R, :mode_freq] = max[frequency[R :> (x: not Number(x))]]
// Stats for numeric types
@inline def describe_full[R, :mean] = mean[R :> Number]
@inline def describe_full[R, :std] = sample_stddev[R :> Number]
@inline def describe_full[R, :percentile25] = percentile[(R :> Number), 25]
@inline def describe_full[R, :percentile50] = median[R :> Number]
@inline def describe_full[R, :percentile75] = percentile[(R :> Number), 75]
diff
diff[R, S]
Set difference (complement): removes the tuples of S
from R
, if present.
Example:
def output = diff[{1;2;3;4} , {1;3} ]
2
4
Definition
@inline
def diff[R, S](x...) = R(x...) and not S(x...)
disjoint
disjoint(R, S)
R
and S
are disjoint if they have no fact in common.
Definition
@inline
def disjoint(R, S) = empty(R ∩ S)
divide
divide[x, y]
x / y
Division.
divide[x, y]
is defined when either the types of x
and y
are the same, either operand is of type SignedInt[64]
or Floating[64]
, one operand is a rational type and the other is a signed integer type, or one operand is a decimal type and the other is an integer type (regardless of signedness). The return type is:
- the type of
x
when the types of both operands are the same float, rational, or decimal type. Floating[64]
when the types of both operands are integer types.Floating[64]
when either the type ofx
or the type ofy
isFloating[64]
.Rational[n]
, wheren
is the greater of the number of bits in the types ofx
andy
, when one operand is a rational type and the other is a signed integer type.- The type of
x
ifx
is a float type andy
is of typeSignedInt[64]
, and vice versa. - The type of
x
ifx
is a decimal type andy
is some integer type, and vice versa.
Examples:
3.4 / 2 = 1.7
divide[6, 3] = 2.0
equal(divide(6, 3, 2.0), true)
equal(divide(6, 3, 2), false)
Definition
@inline
def divide[x, y] = rel_primitive_divide[x, y]
@inline
def (/)(x, y, z) = divide(x, y, z)
docstring
docstring[:R]
The docstring of a relation as a string.
Example:
doc"The number 0." def zero = 0
def output = docstring[:zero]
"The number 0."
Definition
@inline
def docstring = rel_primitive_docstring
domain
domain[F]
The domain of a scalar function F
. The domain represents all but the last argument of F
.
Example:
def output = domain[{(1, 'a', 1); (2, 'b', 4); (3, 'c', 9)}]
(1, 'a')
(2, 'b')
(3, 'c')
Definition
@inline
def domain[F](k...) = F(k..., _)
dot_join
dot_join[R, S]
R . S
Compose
Definition
@inline
def dot_join[R,S](x..., y...) = exists(t: R(x..., t) and S(t, y...))
@inline
def (.) = dot_join
double_metaphone
double_metaphone[string_value]
double_metaphone_alt[string_value]
Double Metaphone is an improvement of metaphone
that considers spelling particularities in a number of other languages. double_metaphone
and double_metaphone_alt
returns an encoding of the phonetic representation of string_value
. For many strings, double_metaphone
and double_metaphone_alt
returns the very same encoding. However, it may differs based on the spelling in a non-English language.
string_value
could be any arbitrary long string
Examples:
@inline def dm[string] = double_metaphone[string] ; double_metaphone_alt[string]
def output = dm["Smith"]
"sm0"
def output = dm["Smythe"]
"sm0"
"xmt"
def output = dm["Christina"]
"krstn"
def output = dm["Cristine"]
"krstn"
def output = dm["I like Music"]
"ilkm"
def output = dm["i loak Museek"]
"ilkm"
def output = dm["RelationalAI"]
"rlxnl"
def output = dm["rellationalleAI"]
"rlxnl"
Definition
@inline
def double_metaphone = rel_primitive_double_metaphone
@inline
def double_metaphone_alt = rel_primitive_double_metaphone_alt
empty
empty(R)
Examples:
empty(1)
isfalse
empty(true)
isfalse
empty(false)
istrue
Definition
@inline
def empty(R) = not exists(x...: R(x...))
encode_base16
encode_base16[input_str]
Encodes the given string(input_str
) to base16-encoded string, as per rfc4648 specifications.
Encoding/decoding specifications are found at https://datatracker.ietf.org/doc/html/rfc4648.
Examples:
def output = encode_base16["Hello!"]
"48656C6C6F21"
def output = encode_base16["你好"]
"E4BDA0E5A5BD"
Definition
@inline
def encode_base16 = rel_primitive_base16_encode
encode_base32
encode_base32[input_str]
Encodes the given string(input_str
) to base32-encoded string, as per rfc4648 specifications.
Encoding/decoding specifications are found at https://datatracker.ietf.org/doc/html/rfc4648.
Example:
def output = encode_base32["Hello!"]
"JBSWY3DPEE======"
def output = encode_base32["你好"]
"4S62BZNFXU======"
Definition
@inline
def encode_base32 = rel_primitive_base32_encode
encode_base64
encode_base64[input_str]
Encodes the given string(input_str
) to base64-encoded string, as per rfc4648 specifications.
Encoding/decoding specifications are found at https://datatracker.ietf.org/doc/html/rfc4648.
Example:
def output = encode_base64["Hello!"]
"SGVsbG8h"
def output = encode_base64["你好"]
"5L2g5aW9"
Definition
@inline
def encode_base64 = rel_primitive_base64_encode
Entity
Entity(x)
Holds if x
is an entity.
Example:
Integrity constraint that specifies that a module always contains an entity:
def myentity:f = 1
ic my_entity_ic(x) { myentity(x) implies Entity(x) }
Definition
@inline
def Entity = rel_primitive_Entity
epoch_milliseconds
epoch_milliseconds[dt]
Milliseconds since Epoch time, as a value of type Int64
.
Definition
@inline
def epoch_milliseconds =
rel_primitive_epoch_milliseconds
eq
eq(x, y)
x = y
Equality between scalar (single) values, such as integers, strings, symbols. (For equality between relations, use equal
.)
Examples:
1 = 1
istrue
.:a = :a
istrue
"a" = "a"
istrue
2 = 2.0
isfalse
1 = "a"
isfalse
Definition
@inline
def eq(x, y) = rel_primitive_eq(x, y)
@inline
def (=)(x, y) = eq(x, y)
equal
equal(R, S)
Relational equality. Note that eq
and =
should be used only for scalar values, while equal
should be used to check that two relations are the same.
Definition
@inline
def equal(R, S) = forall(x... where R(x...): S(x...)) and forall(x... where S(x...): R(x...))
erf
erf[x]
The error function of x
.
External link: https://en.wikipedia.org/wiki/Error_function
Examples:
erf[2] = 0.9953222650189527
erf[-0.5] = -0.5204998778130465
Definition
@inline
def erf[x] = rel_primitive_error_function[x]
erfinv
erfinv[x]
The inverse error function of x
.
External link: https://en.wikipedia.org/wiki/Error_function#Inverse_functions
Examples:
erfinv[0.1] = 0.08885599049425769
erfinv[-0.5] = -0.4769362762044699
erfinv[erf[1]] = 1.0
Definition
@inline
def erfinv[x] = rel_primitive_error_function_inverse[x]
escape_regex_metachars
escape_regex_metachars(string, escaped_string)
Escape the necessary regular expression metacharacters in string
such that escaped_string
can be used as a regular expression and have none of its characters interpreted as metacharacters.
Example:
def ere = escape_regex_metachars["."]
regex_match_all(ere, "abc.123")
will only give
{(4, ".")}
since .
is escaped and is not treated as a metacharacter.
Definition
@inline
def escape_regex_metachars[s in String] =
string_replace_multiple[
string_replace[s, "\\", "\\\\"],
{mc:
concat["\\", mc],
{"."; "^"; "$"; "*"; "+"; "?"; "("; ")"; "["; "|"; "{"}(mc)
}
]
export_csv
export_csv[R]
This is the main entry point to specify relations that should be exported to CSV. The payload relation R
is expected to be a configuration relation mapping keys to relations.
Required keywords are:
:path
: A string specifying the location of the file that is to be created.
Optional keywords:
:data
: A set of relations mapping a file position, which can be a key of arbitrary length or type, to the corresponding value. This is the data that will be exported. We expect the data relation to be of form: data(:COLUMN_NAME, pos val):integration
: A relation containing storage integration configuration.:schema
: A relation containing schema information.:syntax
: A relation containing syntax configuration.
Example using default CSV schema and syntax and no storage integration.
def csv_data(:ORDER, pos, v) = ((1,1); (2,2); (3,3))(pos, v)
def csv_data(:LINEITEM, pos, v) = ((1,100); (2,101); (3,102))(pos, v)
def csv_data(:QUANTITY, pos, v) = ((1,2); (2,15); (3,42))(pos, v)
def export = export_csv[(:path, "/path/to/local/file.csv"); (:data, csv_data)]
Example using a custom CSV schema.
def csv_syntax[:delim] = ';'
def csv_syntax[:quotechar] = '_'
def export = export_csv[(:path, "/path/to/local/file.csv");
(:data, csv_data);
(:syntax, csv_syntax)]
Example using an Azure storage integration using a SAS token:
def integration[:provider] = "azure"
def integration[:credentials, :azure_sas_token] = "<azure_sas_token>"
def export = export_csv[(:path, "azure://<account_name>.blob.core.windows.net/container/file.csv");
(:data, csv_data);
(:integration, integration)]
Example with compound keys:
def csv_data(:ORDER, pos..., v) = (1, 2, 100; 2, 1, 40)(pos..., v)
def csv_data(:QUANTITY, pos..., v) = (1, 3, 10; 2, 3, 11; 3, 1, 12; 3, 2 ,13)(pos..., v)
def export = export_csv[(:data, csv_data; :path, "/path/to/data.csv")]
Definition
@inline
def export_csv[R](ys..., xs...) = (:envelope, (:payload, (
(:data, ((key..., col, val): R[:data](col, key..., val))) ; (x, y...: R(x, y...), (x != :data))
)(xs...); (:content_type, "text/csv")(xs...)))(ys...)
@inline
def export_csv_row_wise[R](ys..., xs...) = (:envelope, (:payload, R(xs...); (:content_type, "text/csv")(xs...)))(ys...)
export_json
export_json[R]
This is the main entry point to specify relations that should be exported to JSON. The payload relation R
is expected to be a configuration relation mapping keys to relations.
Required keywords are:
:path
: A string specifying the location of the file that is to be created.
Optional keywords:
:data
: The relation(s) that should be exported.:integration
: A relation containing storage integration configuration.:indent
: Number of spaces to indent the resulting document. If not present, we’ll write a compact form.
Example for a simple JSON object
def json[:author] = "David Foster Wallace"
def json[:title, :name] = "Infinite Jest"
def json[:title, :isbn] = "978-0-316-92004-9"
def export = export_json[(:path, "/path/to/file.json");
(:data, json);
(:indent, 2)]
This results in the following JSON:
{
"author": "David Foster Wallace",
"title": {
"name": "Infinite Jest",
"isbn": "10: 0316921173"
}
}
Since authors usually write more than one book, we’ll make the :title
path an array of objects.
Examples:
def json[:author] = "David Foster Wallace"
def titles = (1, "Infinite Jest", "978-0-316-92004-9");
(2, "The Pale King", "978-0-316-07423-0")
def json[:titles, :[], idx] = ((:name, name);
(:isbn, isbn)
from name, isbn where titles(idx, name, isbn))
def export = export_json[(:path, "/path/to/file.json");
(:data, json);
(:indent, 2)]
The :[]
marker, i.e. array marker, needs to be present when an array is to be correctly produced.
This yields the following output:
{
"author": "David Foster Wallace",
"titles": [
{
"name": "Infinite Jest",
"isbn": "978-0-316-92004-9"
},
{
"name": "The Pale King",
"isbn": "978-0-316-07423-0"
}
]
}
Definition
@inline
def export_json[R](ys..., xs...) =
(:envelope, (
(:payload, R(xs...));
(:content_type, "application/json")(xs...)
))(ys...)
factorial
factorial[x]
Factorial of x.
Defined for non-negative x
. The result is promoted to at least 64-bits.
If x
is up to 64-bits, factorial
is defined for values up to 20 (inclusive). If x
is Int128
, factorial
is defined for values up to 33 (inclusive). If x
is UInt128
, factorial
is defined for values up to 34 (inclusive).
Definition
@inline
def factorial[x] = rel_primitive_factorial[x]
filepos
filepos[v]
Creates a FilePos representing position v
in a file.
Definition
@inline
def filepos = rel_primitive_filepos
FilePos
FilePos(x)
Holds if the given value is a FilePos, which are types created when importing CSV files. FilePos types are used as keys when joining columns from the same row. Currently, users cannot directly create this data type themselves.
Example:
Integrity constraint that specifies that a relation called csv
contains values with RelName
, FilePos
, and String
.
def config:data = """
item, category
laptop, computer
monitor, peripheral
"""
def csv = load_csv[config]
//FilePos automatically generated
def csv_ic {
subset(csv, (RelName, FilePos, String))
}
Definition
@inline
def FilePos = rel_primitive_FilePos
// Temporary alias for backward compatibility
@inline
def delve_period_days = rel_primitive_period_days
// Temporary alias for backward compatibility
@inline
def delve_date_daysinmonth = rel_primitive_date_daysinmonth
filepos_value
filepos_value[fp]
Retrieves the numeric position represented by a FilePos.
Definition
@inline
def filepos_value = rel_primitive_filepos_value
first
first[R]
Projection to the first argument of R.
first
supports heterogeneous relations, both in arity and types.
Examples:
def output = first[(1, 2, 3); (4, 5, 6)]
1
4
def output = first[(1, 2, 3); (4, 5); 6]
1
4
6
def output = first[("a", 1); (2, 3)]
2
"a"
Definition
@inline
def first[R](x) = ∃(y... : R(x, y...))
FixedDecimal
FixedDecimal(nbits, ndecimals, x)
Holds if x
is of type FixedDecimal
with a bit size of nbits
and ndecimals
decimal precision.
Examples:
Integrity constraint that tests whether x
is a FixedDecimal
with a bit size of 64
and 4
decimal precision:
def R = decimal[64, 4, pi_float64]
ic float_type_check(x in R) {
FixedDecimal(64, 4, x)
}
Query to check whether x
is a FixedDecimal
with a bit size of 64
and 4
decimal precision.
def R = {decimal[64, 4, pi_float64]; decimal[64, 10, pi_float64]}
def output(x) = R(x) and FixedDecimal(64, 4, x)
//output> 3.1416
Definition
@inline
def FixedDecimal = rel_primitive_FixedDecimal
FixedDecimal_spread
FixedDecimal_spread[mode, R, val]
This definition uses the two spread strategies implemented in int_spread_by_even
and int_spread_by_ratio
but specialized to input value that is a FixedDecimal. The major difference here is that in the integer case, the smallest unit to spread was 1
, whereas here, the smallest unit to spread is 10^(-d)
where d
is the number of digits used in the FixedDecimal
value val
. For example, if we are trying to spread the value 3.140
, the smallest unit to spread will be 0.001
. mode
can be :even
or :ratio
.
Example:
def R = {("Atlanta", "Georgia", 50);
("Seattle", "Washington", 80);
("San Francisco", "California", 10)}
def output = FixedDecimal_spread[:even, R, decimal[64, 3, 3.14]]
("Atlanta", "Georgia", 1.047)
("San Francisco", "California", 1.047)
("Seattle", "Washington", 1.046)
Definition
@inline
def FixedDecimal_spread[mode, R, val] = v..., s:
rel_primitive_decimal_type(val, decimal_bit_length[val], decimal_precision[val]) and
den = denominator[val] and
digits = floor_to_int[log10[den]] and
valinput = numerator[val] and
(
(mode = :even and s = int_spread_by_even[R, valinput][v...] / den) or
(mode = :ratio and s = int_spread_by_ratio[R, valinput][v...] / den)
)
from valinput, digits, den
float
float[n, v]
The n
-bit floating point value from the number v
. Example: float[64, 3.0]
.
Definition
@inline
def float = rel_primitive_float
float64
float64[v]
The 64
-bit floating point value from the number v
, which must be a float.
Definition
@inline
def float64 = float[64]
float_int_convert
float_int_convert[x]
Conversion from float to int. If the argument is not equivalent to an int, float_int_convert
returns false
. (See trunc_to_int
, floor_to_int
for general conversion.)
Example:
float_int_convert[3.0] = 3
float_int_convert[3.2] = false
Definition
@inline
def float_int_convert = rel_primitive_float_int_convert
Floating
Floating(nbits, x)
Holds if x
is an nbits' floating point number.
Float(x)is a shorthand that holds if
x` is a 64-bit float.
Example:
Integrity constraint that tests if x
is a 32-bit float (will throw if x
is not a 32-bit Float):
def R = float[32, 1.321]
ic float_type_check(x in R) {
Floating(32, x)
}
Definition
@inline
def Floating = rel_primitive_Floating
floor
floor[x]
Round down to the nearest integer.
Examples:
floor[4.5] = 4.0
floor[-4.5] = -5.0
floor[4] = 4
Definition
@inline
def floor[x] = round[:ROUND_DOWN, x]
floor_divide
floor_divide[x, y]
Floor Division.
Largest integer less than or equal to x/y. Computes x/y rounded towards negative infinity.
Is false
(empty) when y = 0
for Integer arguments.
Examples:
floor_divide[5, 2] = 2
floor_divide[-5, 2] = -3
floor_divide[5.1, 2] = 2.0
empty(floor_divide[5.1, 0])
Definition
@inline
def floor_divide[x, y] = rel_primitive_floor_divide[x, y]
floor_to_int
floor_to_int[x]
General float-to-int conversion, floor. Argument must be a float.
Examples:
floor_to_int[3.1] = 3
floor_to_int[-3.1] = -4
floor_to_int[3] : error
Definition
@inline
def floor_to_int[x] = float_int_convert[floor[x]]
format_date
format_date[d, format]
A string where the Date d
is formatted according to format
.
Example:
def d = parse_date["2018-06-12", "Y-m-d"]
def output = format_date[d, "Y-mm-d"]
"2018-06-12"
For details on the format parameter, see the Julia documentation for Dates.DateFormat
.
Definition
@inline
def format_date = rel_primitive_format_date
format_datetime
format_datetime[dt, format, tz]
Format a DateTime dt
, with timezone tz
.
Example:
def format = "yyyy-mm-dd HH:MM ZZZ"
def dt = parse_datetime["2018-03-11 01:00 America/New_York", "Y-m-d H:M Z"]
def output = format_datetime[dt, format, "America/New_York"]
"2018-03-11 01:00 EST"
For details on the format parameter, see the Julia documentation for Dates.DateFormat
.
Definition
@inline
def format_datetime = rel_primitive_format_datetime
formula_card_est
formula_card_est[R]
The cardinality estimate of a given relational abstraction R
as estimated by the physical query optimizer. The estimate is represented as a Float
. It is typically an upper bound on the actual cardinality and can suffer from a numeric overflow, e.g. if the formula R
is a big cross-product of relations. To avoid this, use the log-version log_card_est[R]
. See log_card_est
for more details.
Examples:
def card = formula_card_est[R]
def card = formula_card_est[a, b, c: R(a, b) and S(b, c) and T(a, c)]
Definition
@inline
def formula_card_est[R] = 2.0 ^ rel_log_card_est[R]
frequency
frequency(R, elem, c)
frequency[R, elem]
frequency[R]
Finds the frequency c
of elem
in (the last argument of) the relation R
.
Example:
def example = {(1, "a"); (2, "b"); (3, 123); (4, 12.5); (3, "b")}
def output = frequency[example, "b"]
2
def output = frequency[example]
("a", 1)
("b", 2)
(123, 1)
(12.5, 1)
Definition
@inline
def frequency[R, elem] = count[x...: R(x..., elem)]
function
function(R)
Given a relation R, true
if R is a function (from all but the last elements, to the last), and false
if not.
Examples:
def output = function[{(1, 2); (2, 5)}]
true
def output = function[{(1, 2); (1, 3)}]
false
def output = function[{(1,2,3) ; (1,3,4)}]
true
Definition
@inline
def function(R) =
forall(k..., v1, v2 where R(k..., v1) and R(k..., v2): v1 = v2)
geometric_mean
geometric_mean[R]
The geometric mean of the values of a relation.
Example:
def output = geometric_mean[{('a',1); ('b',2); ('a',3); ('b',4); ('c',5); ('d',6}]
2.99379516552
Definition
@inline
def geometric_mean[R] = power[product[R], 1 / count[R]]
gt
gt(x, y)
x > y
Definition
@inline
def gt(x, y) = rel_primitive_gt(x, y)
@inline
def (>)(x, y) = gt(x, y)
gt_eq
gt_eq(x, y)
x ≥ y
x >= y
Definition
@inline
def gt_eq(x, y) = rel_primitive_gt_eq(x, y)
@inline
def (≥)(x, y) = gt_eq(x, y)
@inline
def (>=)(x, y) = gt_eq(x, y)
harmonic_mean
harmonic_mean[R]
The harmonic mean of the values of a relation.
Example:
def output = harmonic_mean[{('a',1); ('b',2); ('a',3); ('b',4); ('c',5); ('d',6}]
2.44897959184
Definition
@inline
def harmonic_mean[R] = count[R] / sum[a..., v, i: R(a..., v) and i = 1 / v]
hash
hash[R]
see hash128
Definition
@inline
def hash = hash128
Hash
Hash(x)
Holds if x
is a RelationalAI HashValue generated by hash128
.
Examples:
Integrity constraint that tests whether x
is Hash (will throw if x
is not Hash):
def R = (9, "a")
def my_hash = hash128[R]
//three value relation with hash value as third value
ic hash_type_check{
subset(my_hash, (Any, Any, Hash))
}
Defines Hash as schema for a relation:
def R = (9, "a")
def my_hash = hash128[R]
def hashed_relation(x in Hash) = my_hash(_, _, x)
def output = hashed_relation
//output> -107770920621774551289984725953057040743
Definition
@inline
def Hash = rel_primitive_HashValue
hash128
hash128[R]
Hash128 each of the tuples in a relation.
hash128
takes a relation R(xs...)
and produces a relation R(xs..., i)
where i
is a HashValue
index. The index indicates the hash128 of each tuple of R
.
Example:
def output = hash128['a']
('a', HashValue(0xbd1defa3be4617e0))
def sample = {'a'; 'b'; 'c'}
def output = hash128[sample]
('a', HashValue(0xbd1defa3be4617e0))
('b', HashValue(0x2423a34d925fc9ed))
('c', HashValue(0xd2cffd2ea6c73a7))
Definition
@inline
def hash128 = rel_primitive_hash
hash_value_uint128_convert
hash_value_uint128_convert[v]
Convert a HashValue v
to UInt128.
Definition
@inline
def hash_value_uint128_convert[v] = rel_primitive_hash_value_uint128_convert[v]
haversine
haversine[r, x1, y1, x2, y2]
The great circle distance of two points, i.e and , on a sphere of radius , using the Haversine formula. The two points are specified by their latitude and longitude in radians.
Example:
def output = haversine[10, 0, 0, 0, pi_float64/2]
15.707963267948964
Definition
@inline
def haversine[r, x1, y1, x2, y2] =
2 * r * asin[sqrt[sin[(x2 - x1)/2] ^ 2 + cos[x1] * cos[x2] * sin[(y2 - y1) / 2] ^ 2]]
Hour
Hour[n]
A period that spans n
hours. The period is given in units of Hour
.
Example:
datetime_add[unix_epoch, Hour[10]]
Definition
@inline
def Hour = rel_primitive_hour
iff
F iff G
F ≡ G
F ⇔ G
If and only if (boolean equivalence), for boolean (arity 0, true or false) arguments F and G.
Definition
@inline
def iff(F, G) = (F implies G) and (G implies F)
@inline
def (≡)(F, G) = F iff G
@inline
def (⇔)(F, G) = F iff G
int
int[n, v]
The n
-bit signed integer value from the integer value v
.
Definition
@inline
def int = rel_primitive_int
int128
int128[v]
The 128
-bit signed integer value from the integer value v
.
Definition
@inline
def int128 = int[128]
int64
int64[v]
The 64
-bit signed integer value from the integer value v
.
Definition
@inline
def int64 = int[64]
int_float_convert
int_float_convert[x]
Conversion from int to float. Argument must be an int.
Example:
int_float_convert[3] = 3.0
int_float_convert[3.2] : error
Definition
@inline
def int_float_convert = rel_primitive_int_float_convert
int_spread_by_even
int_spread_by_even[R, value]
Spreads value
to the entries in the relation R
evenly as whole units. If the value to spread = 50
and R
has 5 elements, each element will get 10
. In case of uneven spread (e.g. spread 51 to 5 elements), the remainder gets distributed as single units in order starting from the first element after the relation has been sorted lexicographically (for details see sort
).
Empty if value
is negative.
Example:
def R = {("Atlanta", "Georgia");
("Seattle", "Washington");
("San Francisco", "California")}
def output = int_spread_by_even[R, 32]
("Atlanta", "Georgia", 11)
("San Francisco", "California", 11)
("Seattle", "Washington", 10)
Definition
@inline
def int_spread_by_even[R, val] = v..., s:
Int(val) and
R(v...) and
val >= 0 and
c = count[R] and
sort[R](i, v...) and
(
(i <= remainder[val, c] and s = trunc_divide[val, c] + 1) or
(i > remainder[val, c] and s = trunc_divide[val, c])
)
from i, c
int_spread_by_ratio
int_spread_by_ratio[R, value]
Spreads value
to the entries in the relation R
proportionally to their weight (which is the last element in each tuple), as whole units. For example, if the relation is {("X",90); ("Y",10)}
, "X"
will get 90%
of value
and "Y"
will get 10%
. Often, this split will result in a decimal number, but we want to spread whole units. To resolve this issue, we first distribute the floor value of the decimal number computed. The remainder gets distributed as single units in order starting from the element that had the highest decimal value.
Empty if value
is negative.
Example:
def R = {("Atlanta", "Georgia", 50);
("Seattle", "Washington", 80);
("San Francisco", "California", 10)}
def output = int_spread_by_ratio[R, 30]
("Atlanta", "Georgia", 11)
("San Francisco", "California", 2)
("Seattle", "Washington", 17)
Definition
@inline
def int_spread_by_ratio[R, val] = v..., s:
Int(val) and
R(v..., w) and
val >= 0 and
forall(t in last[R] : t > 0) and
s = floor_to_int[val * (w / sum[R])] + offset and
sort[(rr, vv...) : rr = remainder[R[vv...] * val, sum[R]]](i, r, v...) and
c = val - sum[vv... : floor_to_int[val * (R[vv...] / sum[R])]] and
(
(offset = 1 and i >= count[R] - c + 1) or
(offset = 0 and i < count[R] - c + 1)
)
from w, i, r, c, offset
intersect
intersect[R, S]
R ∩ S
Intersect two n-ary relations R and S
Definition
@inline
def intersect[R, S](x...) = R(x...) and S(x...)
@inline
def (∩) = intersect
is_Day
is_Day(x)
Holds if x
is a Day
period.
Definition
@inline
def is_Day = rel_primitive_Day
is_Hour
is_Hour(x)
Holds if x
is a Hour
period.
Definition
@inline
def is_Hour = rel_primitive_Hour
is_Microsecond
is_Microsecond(x)
Holds if x
is a Microsecond
period.
Definition
@inline
def is_Microsecond = rel_primitive_Microsecond
is_Millisecond
is_Millisecond(x)
Holds if x
is a Millisecond
period.
Definition
@inline
def is_Millisecond = rel_primitive_Millisecond
is_Minute
is_Minute(x)
Holds if x
is a Minute
period.
Definition
@inline
def is_Minute = rel_primitive_Minute
is_Month
is_Month(x)
Holds if x
is a Month
period.
Definition
@inline
def is_Month = rel_primitive_Month
is_Nanosecond
is_Nanosecond(x)
Holds if x
is a Nanosecond
period.
Definition
@inline
def is_Nanosecond = rel_primitive_Nanosecond
is_Second
is_Second(x)
Holds if x
is a Second
period.
Definition
@inline
def is_Second = rel_primitive_Second
is_Week
is_Week(x)
Holds if x
is a Week
period.
Definition
@inline
def is_Week = rel_primitive_Week
is_Year
is_Year(x)
Holds if x
is a Year
period.
Definition
@inline
def is_Year = rel_primitive_Year
jacobian
jacobian[S, R]
jacobian(S, R, J)
Compute the Jacobian (e.g. the derivative or gradient) of relation S
w.r.t. relation R
.
jacobian
computes the derivatives of one scalar- or vector-valued relation (R
) with respect to a second scalar- or vector-valued relation (S
). It returns a matrix of all of them (J
) using (relational) automatic differentiation techniques.
Both relations (S
and R
) must be key value pairs. This is true even of scalars such as x[1]
, which cannot be rendered as x
.
Examples
Derivative of a scalar-valued relation of one scalar variable:
def y[1] = 1.0
def z[1] = 3.0 * y[1]^2
def output = jacobian[z, y]
//output> 1,1,6.0
Derivative of a more complicated scalar-valued relation of one scalar variable:
def x[1] = 1.0
def y[1] = 3.0 * x[1]^2
def z[1] = y[1] / 2
def output = jacobian[z, x]
//output> 1,1,3.0
Gradient of a vector-valued relation with respect to a scalar input:
def y[1] = 2.0
def z[1] = 3.0 * y[1]^2
def w[1] = z[1]^2
def w[2] = y[1]
def output = jacobian[w, y]
//output> 1,1,288.0
// 2,2,1.0
Jacobian of a vector-valued relation with respect to a vector input:
def y[1] = 1.0
def y[2] = 4.0
def z[1] = 2*y[1] + 3*y[2]
def z[2] = y[1] - y[2]
def output = jacobian[z, y]
//output> 1,1,2.0
// 1,2,3.0
// 2,1,1.0
// 2,2,-1.0
Definition
@inline
def jacobian[S, R] = rel_primitive_jacobian[S, R]
json_string
json_string[json_relation]
The string representation of a relation that encodes JSON data.
Example:
def json_relation[:name] = "Amira"
def json_relation[:age] = 32
def json_relation[:height] = missing
def json_relation[:pets, :[], 1] = "dog"
def json_relation[:pets, :[], 2] = "rabbit"
def result = json_string[json_relation]
results in the following JSON:
{
"name": "Amira",
"age": 32,
"height": null,
"pets": [
"dog",
"rabbit"
]
}
Definition
@inline
def json_string = rel_primitive_json_string
last
last[R]
Projection to the last argument of the relation R
.
last
supports heterogeneous relations, both in arity and types.
Examples:
def output = last[(1, 2, 3); (4, 5, 6)]
3
6
def output = last[1; 2; 3]
1
2
3
def output = last[(1, 2); (3, "abc")]
2
"abc"
def output = last[(1, 2, 3); (4, 5)]
3
5
Definition
@inline
def last[R](y) = ∃(x... : R(x..., y))
left_override
left_override[R, S]
R <++ S
The (left) override operator is usually applied to relations of tuples (k..., v)
with a functional dependency from the initial (key) arguments (k...)
to the last (value) argument v
.
R <++ S
contains all the tuples of R
, plus all the tuples in S
whose key is not in R
. Often, S
specifies default values for keys that are missing in R
. In the result, a value in R
overrides a value in S
.
There are two equivalent ways to think about R <++ S
: (1) S
is providing default values for keys that are missing in R
. (2) The entries in R
are overriding the entries in S
.
Example | Value |
---|---|
2 <++ 3 | 2 |
1.5 <++ 3 | 1.5 |
(1,2) <++ (1,3) | (1,2) |
(3,4) <++ (1,2); (3,5) | (1,2); (3,4) |
(3,"abc") <++ (1,2); (3,5) | (1,2); (3,"abc") |
(1,2); (3,5) <++ (3,4); (6,7) | (1,2); (3,5); (6,7) |
Override can be applied to heterogeneous relations, notably JSON-like relations. The following examples show <++
operators applied to JSON inputs.
Left | Right | Value |
---|---|---|
{"a": 1, "b": 3} | {"a": 2 } | {"a": 1, "b": 3} |
{"a": 2 } | {"a": 1, "b": 3} | {"a": 2, "b": 3} |
{"a": 1, "b": 2} | {"c": 3} | {"a": 1, "b": 2, "c": 3} |
See also: default_value[D, F, c]
.
Definition
@inline
def left_override[R, S](xs...) = R(xs...)
@inline
def left_override[R, S](xs..., v) = S(xs..., v) and not R(xs..., _)
@inline
def (<++) = left_override
levenshtein
levenshtein[string1, string2]
Calculate the Levenshtein distance between two strings.
Example:
levenshtein["kitten", "sitting"]
3
Definition
@inline
def levenshtein = rel_primitive_levenshtein
like_match
like_match(like_string, string)
Match string with a SQL “LIKE” pattern; case-sensitive.
The ’%’ character is a wildcard, and is not escapable.
Example:
like_match("%PROMO%", "this is a PROMOtion")
()
Definition
@inline
def like_match = rel_primitive_like_match
lined_csv
lined_csv[R]
Given an already loaded CSV relation R
, lined_csv[R]
is a copy of the relation, but with the file position field replaced with a line number. Only lines containing data, including errors, are included in the numbering. Headers and empty lines are not counted. Thus, line number 1 is the first non-empty, non-header line of the CSV relation.
Example:
def csv = load_csv[config]
def csv_with_lines = lined_csv[csv]
Definition
@inline
def lined_csv[R](col, line, rest...) = sort[second[R]](line, pos) and R(col, pos, rest...) from pos
load_binary
load_binary[URI_string]
load_binary[R]
This is the main in-language API for loading files as binary string. To load a file, either provide the string URI where the data resides, or a configuration relation R
that describes the data load.
The relation R
, which must be an EDB or an IDB but not a formula, can configure the data load with the following options:
Required:
:path
: The path to the file, e.g."s3://storage.blob.core.net/container/image.jpeg"
.
Optional:
:integration
: A relation containing storage integration configuration.
Example:
def config[:path] = "azure://<account_name>.blob.core.windows.net/container/file.bin"
def config[:integration, :provider] = "azure"
def config[:integration, :credentials, :azure_sas_token] = "<azure_sas_token>"
def output = load_binary[config]
Definition
@inline
def load_binary = rel_primitive_load_binary
load_csv
load_csv[URI_string]
load_csv[R]
This is the main in-language API for loading CSV data. To load a CSV, either provide the string URI where the data resides, or a configuration relation R
that describes the data load.
In the first case, all columns are imported with type String
.
The relation R
, which must be an EDB or an IDB but not a formula, can configure the data load with the following options:
Required: Either
:path
: The path to the CSV file, e.g."azure://storage.blob.core.net/container/file.csv"
Or
:data
: A CSV formatted string value
Optional:
:schema
: A relation containing schema information.:syntax
: A relation containing syntax configuration.:integration
: A relation containing storage integration configuration.
The resulting relation is of the form (column_name, file_position, value)
. The file_position
uniquely identifies each row in the CSV.
Examples:
Loading from literal string value:
def config[:data] = """
A,B,C
1,2,3
"""
def my_data = load_csv[config]
Loading a CSV file using a constant path:
def y = load_csv["s3://mybucket/path/test.csv"]
def x = y[:A, _]
In the example above, all values for column :A
(“A” in the original file) are stored in x
.
This can also be used to only load specific columns: load_csv[path](:A, xs...)
.
Loading a CSV file using a configuration relation config
:
def config[:path] = "/path/to/file.csv"
def config[:syntax, :delim] = '_'
def config[:syntax, :header] = (1, :A); (2, :B); (3, :C)
def config[:schema, :A] = "int"
def config[:schema, :B] = "string"
def config[:schema, :C] = "decimal(64,2)"
def csv = load_csv[config]
Lastly, load_csv
also supports error handling. If the CSV can’t be read (e.g. if it is badly formatted), parsing errors are stored in the :load_errors
“column”, which can be accessed the same way as any other column. It has the format (:load_errors, file_position, error_column, raw_line)
. Note : Since the arities of regular columns and the error relations differ, use the splat operator (”…”) to make your program as resilient as possible (cf. examples above).
Definition
@inline
def load_csv = rel_primitive_load_csv
load_csv_row_wise
load_csv_row_wise[R]
Load a CSV file into a “row-wise” relation, with the position as the first argument, and the column name second. Included for backward compatibility, we now recommend using load_csv
instead, which puts the column name first.
Definition
@inline
def load_csv_row_wise[R][pos, col] = rel_primitive_load_csv[R][col, pos]
load_json
load_json[filename_or_url]
load_json[R]
The relation R
, which currently needs to either be an EDB or an IDB but no formula, can be used to configure the data load with the following options:
Required: Either
:path
: The path to the JSON file, e.g."azure://storage.blob.core.net/container/file.json"
Or
:data
: A JSON formatted string value
Optional:
:integration
: A relation containing storage integration configuration.
Examples:
Loading literal JSON values:
def data = "{\"name\": \"Anton\", \"age\": 56}"
def config[:data] = data
def json = load_json[config]
Loading a JSON file using a constant path:
def y = load_json["test.json"]
Loading a JSON file using a configuration relation:
def config[:path] = "/path/to/file.json"
def json = load_json[config]
Loading a JSON file from behind a private Azure container using a SAS token:
def config[:path] = "azure://<account_name>.blob.core.windows.net/container/file.json"
def config[:integration, :provider] = "azure"
def config[:integration, :credentials, :azure_sas_token] = "<azure_sas_token>"
def json = load_json[config]
Definition
@inline
def load_json = rel_primitive_load_json
load_jsonlines
load_jsonlines[filename_or_url]
load_jsonlines[R]
Parse and load a JSONLines file. JSONLines is a file format that considers each line of the text file as a separate JSON object.
Example:
def data = "{\"name\":\"Anton\",\"age\":56}\n{\"name\": \"Alex\", \"age\":44}"
def config[:data] = data
def output = load_jsonlines[config]
ic {equal(output[1,:name], "Anton")}
ic {equal(output[1,:age], 56)}
ic {equal(output[2,:name], "Alex")}
ic {equal(output[2,:age], 44)}
Definition
@inline
def load_jsonlines = rel_primitive_load_jsonlines
log
log[x, y]
Logarithm of y with given base x.
Defined for non-negative x
and y
.
Example:
log[2, 8] = 3.0
Definition
@inline
def log[x, y] = rel_primitive_log[x, y]
log10
log10[x]
Base 10 logarithm.
Defined for non-negative x
.
log10[1000] = 3.0
Definition
@inline
def log10[x] = rel_primitive_log10[x]
log_card_est
log_card_est[R]
The log base 2 of the cardinality estimate of a given relational abstraction R
.
Definition
@inline
def log_card_est[R] = rel_log_card_est[R]
lowercase
lowercase[string_or_char]
A string where all the characters are converted to lowercase. If a character is already lowercase or has no lowercase version, it remains unchanged.
Example:
def output = lowercase["aB1c"]
"ab1c"
def output = lowercase['Â']
'â'
lowercase
does not take a locale option and does not handle local-specific case mapping rules.
Definition
@inline
def lowercase = rel_primitive_lowercase
lt
lt(x, y)
x < y
Definition
@inline
def lt(x, y) = rel_primitive_lt(x, y)
@inline
def (<)(x, y) = lt(x, y)
lt_eq
lt_eq(x, y)
x ≤ y
x <= y
Definition
@inline
def lt_eq(x, y) = rel_primitive_lt_eq(x, y)
@inline
def (≤)(x, y) = lt_eq(x, y)
@inline
def (<=)(x, y) = lt_eq(x, y)
mae
mae[YHAT, Y]
Mean absolute error (MAE, L1)
Definition
@inline
def mae[YHAT,Y] = mean[x... : abs[Y[x...] - YHAT[x...]]]
max
max[R]
Maximum of (the last argument of) the relation R
, if non-empty. If R
is empty, max[R]
is false (empty). Use <++
(left override) if you need a specific value for the case where R
is empty.
Example:
max[{(2, 3); (1, 6)}] = 6
Definition
@inline
def max[R] = rel_primitive_reduce_noinit[maximum, R]
Max
Max[R]
Please use max[R]. Deprecates in near future
Definition
@inline
def Max = max
max_k
max_k[K, R]
The maximum K
distinct values of (the last column of) the relation R
, if non-empty. If R
contains fewer than K
distinct values (in its last column), then max_k
will have fewer than K
values too. If R
is empty, max_k[K, R]
is false. Currently, the maximum supported value of K is 10000.
Example:
max_k[2, {(2, 3); (1, 6); (3, 5); (1, 1); (4, 1); (2, 6)}]
is {5; 6}
.
Definition
@inline
def max_k[K, R] = rel_primitive_mink[R, K, 1]
maximum
maximum[x, y]
Maximum of two arguments. The arguments should have the same type.
Examples:
maximum[3, 4] = 4
maximum[3.0, 4.0] = 4.0
Definition
@inline
def maximum[x, y] = rel_primitive_max[x, y]
mean
mean[R]
average[R]
The arithmetic mean of (the last argument of) the relation R, if non-empty. Empty if R
is empty.
Definition
@inline
def mean[R] = sum[R] / count[R]
mean_normalization
mean_normalization[R]
Mean normalization:
Definition
@inline
def mean_normalization[R][x...] =
(R[x...] - mean[R]) / (max[R] - min[R]), (max[R] > min[R])
median
median[R]
The median of R
according to the sort order of the last element. If the median index is between two values, then median
takes the mean of the middle two values.
Example:
def output = median[{(1, 12); (2, 16); (3, 24)}]
16
Definition
@inline
def median[R] = percentile[R, 50]
metaphone
metaphone[string_value, length]
metaphone
returns an encoding of the phonetic representation of string_value
. metaphone
lets you compare words and sentences based on how they sound in English. The metaphone is considered as an improvement of the soundex
algorithm, and does a better job at matching words and names which sound similar.
string_value
could be any arbitrary long string. length
is the length of the encoding that is used for comparison purpose. A length of 4
is commonly used.
Examples:
def output = metaphone["Smith", 4]
"sm0"
def output = metaphone["Smythe", 4]
"sm0"
def output = metaphone["Christina", 4]
"xrst"
def output = metaphone["Cristine", 4]
"krst"
def output = metaphone["I like Music", 4]
"ilkm"
def output = metaphone["i loak Museek", 4]
"ilkm"
def output = metaphone["RelationalAI", 4]
"rlxn"
def output = metaphone["rellationalleAI", 4]
"rlxn"
Here a simple illustration. Consider the following:
def my_favorite_bands = { "Metallica"; "Iron Maiden"; "Wolfheart"; "Insomium" }
def do_I_like = "Mee ta li ka"
def output = {y in my_favorite_bands : metaphone(do_I_like, 4, z), metaphone[y, 4] = z from z}
the script above returns "Metallica"
since since "Mee ta li ka"
and "Metallica"
have the same metaphone code.
Definition
@inline
def metaphone = rel_primitive_metaphone
Microsecond
Microsecond[n]
A period that spans n
microseconds. The period is given in units of Microsecond
. (One second is microseconds.)
Example:
datetime_add[unix_epoch, Microsecond[5 * 10^6]]
gives
Dates.DateTime("1970-01-01T00:00:05")
Definition
@inline
def Microsecond = rel_primitive_microsecond
Millisecond
Millisecond[n]
A period that spans n
milliseconds. The period is given in units of Millisecond
. (One second is milliseconds.)
Example:
datetime_add[unix_epoch, Millisecond[20 * 10^3]]
gives
Dates.DateTime("1970-01-01T00:00:20")
Definition
@inline
def Millisecond = rel_primitive_millisecond
min
min[R]
Minimum of (the last argument of) the relation R
, if non-empty. If R
is empty, min[R]
is false. Use <++
(left override) if you need a different value in this case.
Example:
min[{(2, 3); (1, 6)}] = 3
Definition
@inline
def min[R] = rel_primitive_reduce_noinit[minimum, R]
Min
Min[R]
Please use min[R]. Deprecates in near future
Definition
@inline
def Min = min
min_k
min_k[K, R]
The minimum K
distinct values of (the last column of) the relation R
, if non-empty. If R
contains fewer than K
distinct values (in its last column), then min_k
will have fewer than K
values too. If R
is empty, min_k[K, R]
is false. Currently, the maximum supported value of K is 10000.
Example:
min_k[2, {(2, 3); (1, 6); (3, 5); (1, 1); (4, 1); (2, 6)}]
is {1; 3}
.
Definition
@inline
def min_k[K, R] = rel_primitive_mink[R, K, 0]
min_max_normalization
min_max_normalization[R]
Min-Max normalization: .
Also known as min-max scaling or rescaling.
Definition
@inline
def min_max_normalization[R][x...] =
(R[x...] - min[R]) / (max[R] - min[R]), (max[R] > min[R])
minimum
minimum[x, y]
Minimum of two arguments. The arguments should have the same type.
Examples:
minimum[3, 4] = 3
minimum[3.0, 4.0] = 3.0
Definition
@inline
def minimum[x, y] = rel_primitive_min[x, y]
Minute
Minute[n]
A period that spans n
minutes. The period is given in units of Minute
.
Example:
datetime_add[unix_epoch, Minute[100]]
Definition
@inline
def Minute = rel_primitive_minute
missing
missing
Singleton missing value.
Examples:
missing[] = missing
equal( (x : missing(x)) , {missing} )
Definition
@inline
def missing = rel_primitive_missing
Missing
Missing(x)
Tests whether x
has the data type Missing
.
The only data value that is of type Missing
is missing
. Used for imported data that contains explicit nulls (such as JSON data).
Note: Missing
(with a capital) refers to the data type, while missing
is the data itself.
Examples:
Query that returns all relations without missing
as the second argument:
def R = { (1, 2); (3, missing) }
def output(x, y) = R(x, y) and not Missing(y)
//output> (1, 2)
Definition
@inline
def Missing = rel_primitive_Missing
mode
mode[R]
Finds the most common value in (the last argument of) the relation R
. If there are multiple most common values, then mode
chooses the first one according to the sort order.
Examples:
def example = {(1, "a"); (2, "b"); (3, 123); (4, 12.5); (3, "b")}
mode[example] = "b"
def example = {(1, 1); (2, 1); (3, 1); (4, "b"); (5, "b"); (6, "c"); (7, "c")}
mode[example] = 1
Definition
@inline
def mode[R] = last[top[1, argmax[frequency[R]]]]
model_is_function
ic model_is_function
An integrity constraint ensuring that rel:catalog:model
has a functional dependency from the name to the actual value.
Definition
ic model_is_function(k) { function(rel:catalog:model[k]) }
modulo
modulo[x, y]
Modulus after division, in the range [0,y), if y is positive, or (y,0] if y is negative.
Satisfies x = floor_divide[x,y] * y + modulo[x, y]
.
Is false
(empty) when y = 0
for Integer arguments.
Examples:
modulo[8, 3] = 2
modulo[8, -3] = -1
modulo[-8, 3] = 1
empty(modulo[3,0])
modulo[x, y]
is defined when either the types of x
and y
are the same, or either operand is of type SignedInt[64]
or Floating[64]
. The return type is:
- the type of
x
when the types of both operands are the same. SignedInt[n]
orUnsignedInt[n]
wheren
is the greater of the number of bits in the types ofx
andy
and the signedness is chosen to match that of the type ofy
, and bothx
andy
are of integer types.Floating[64]
when either the type ofx
or the type ofy
isFloating[64]
.- The type of
x
whenx
is a non-integer type andy
is of typeSignedInt[64]
. - The type of
y
wheny
is a non-integer type andx
is of typeSignedInt[64]
. Rational[n]
wheren
is the greater of the number of bits in the types ofx
andy
, and one operand is a rational type and the other is aSignedInt[64]
.
Definition
@inline
def modulo[x, y] = rel_primitive_modulo[x, y]
Month
Month[n]
A period that spans n
months. The period is given in units of Month
.
Example:
datetime_add[unix_epoch, Month[10]]
Definition
@inline
def Month = rel_primitive_month
mse
mse[YHAT, Y]
Mean squared error (MSE, L2)
Definition
@inline
def mse[YHAT, Y] = Σ[x... : (Y[x...] - YHAT[x...]) ^ 2] / count[Y]
multiply
multiply[x, y]
x * y
Multiplication.
Examples:
3 * 2 = 6
equal(multiply(3.0, 2, 6.0), true)
Definition
@inline
def multiply[x, y] = rel_primitive_multiply[x, y]
@inline
def (*)(x, y, z) = multiply(x, y, z)
murmurhash3f
murmurhash3f[v]
Hash v
according to the MurmurHash3f algorithm with the default seed 0
, as a UInt128.
Equivalent to murmurhash3f_with_seed[0, v]
.
Definition
@inline
def murmurhash3f[v] = murmurhash3f_with_seed[int64[0], v]
murmurhash3f_with_seed
murmurhash3f_with_seed[seed in (Int64 ∪ UInt128), key]
Hash key
with seed
via the MurmurHash3F algorithm, and yield the result as a UInt128
.
The key
may be any singleton value supported in Rel, (e.g. [U]Int{8,16,32,64,128}
, Float{16,32,64}
, Char
, String
, RelName
, FilePos
, Missing
, …).
The seed
must be a value in (Int64 ∪ UInt128)
.
Note that the standard specification for MurmurHash3F does not include 128-bit seeds, so the result of hashing with a 128-bit seed will be different than a hash with the same value stored in a smaller integer type (i.e. hashing with the seeds uint64[1]
and uint128[1]
will result in different hashes).
This function is an implementation (in Julia) of the reference implementation of MurmurHash3F, i.e. MurmurHash3x64128. The original can be retrieved via:
https://github.com/aappleby/smhasher/blob/ 61a0530f28277f2e850bfc39600ce61d02b518de/src/MurmurHash3.cpp
The reference implementation was written by Austin Appleby and made public domain.
Note that this implementation departs from the standard, in order to provide different hashes for byte-identical values of different types. While this implementation’s internals should produce byte-identical results to the reference implementation, the primary entrypoint here, (murmurhash3f[seed, key]
) typically will not: To distinguish byte-identical values of different types, this function mixes type information into seeds prior to hashing.
Also note that the reference implementation supports only UInt32
seed
s; for such seed
s, this implementation’s internals should provide byte-identical results. But this implementation accepts seed
s up to and including 128-bit integer types. (For UInt32
seed
s, the reference implementation initializes h1
/h2
, the two UInt64
s in which it accumulates/mixes the digest, with copies of seed
. This implementation does the same for UInt32
seed
s, and also for all other seed
types that are less than or equal to 64 bits. For 128-bit seed
s, this implementation initializes h1
with the seed
’s low bits and h2
with the seed
’s high bits.)
Examples
// Without a seed
murmurhash3f["cat"] = 0x37322aa78b4b4ef4816da65505e8efaa
// With a seed
murmurhash3f_with_seed[8675309, "cat"] = 0x9fd1d18093ba94b6f454b48e58011643
murmurhash3f_with_seed[8675309, :bunny] = 0x56f800aed5ff3b7300ace61c691e7568
// byte-identical values of different types hash differently
murmurhash3f[0x01] = 0x6ed3439777f613f7df39df0849d45e09
murmurhash3f[1] = 0x9116cd3f0a651c49f1674100935b29bf
Definition
@inline
def murmurhash3f_with_seed[seed, v] = rel_primitive_murmurhash3f_with_seed[seed, v]
Nanosecond
Nanosecond[n]
A period that spans n
nanoseconds. The period is given in units of Nanosecond
. (One second is nanoseconds.)
Example:
datetime_add[unix_epoch, Nanosecond[5*10^9]]
gives
Dates.DateTime("1970-01-01T00:00:05")
Definition
@inline
def Nanosecond = rel_primitive_nanosecond
nanoseconds_to_datetime
nanoseconds_to_datetime[ns]
Convert nanoseconds (from the unix epoch) to datetime.
Analogous to datetime.datetime.utcfromtimestamp(seconds) in python.
Example:
nanoseconds_to_datetime[24 * 60 * 60 * 10^9]
is 1970-01-02T00:00:00
(one day after unix_epoch
).
Definition
@inline
def nanoseconds_to_datetime[nanoseconds] = datetime_add[unix_epoch, Nanosecond[nanoseconds]]
natural_exp
natural_exp[x]
Exponentiation with the base of the natural log, e.
Definition
@inline
def natural_exp[x] = rel_primitive_natural_exp[x]
natural_log
natural_log[x]
Natural logarithm (ln) (base e).
Defined for non-negative x
.
Definition
@inline
def natural_log[x] = rel_primitive_natural_log[x]
neq
neq(x, y)
x ≠ y
x != y
Examples:
1 != 2
istrue
1 != 1.0
istrue
1 != "a"
istrue
Definition
@inline
def neq(x, y) = rel_primitive_neq(x, y)
@inline
def (≠)(x, y) = neq(x, y)
@inline
def (!=)(x, y) = neq(x, y)
num_bytes
num_bytes[str]
The number of bytes in a string value.
If a string contains Unicode characters, it’s possible that some characters take up more than one byte, so num_bytes[str]
may be larger than num_chars[str]
.
Example:
num_bytes["abcd"] = 4
num_bytes["中文例子"] = 12
Definition
@inline
def num_bytes = rel_primitive_num_bytes
num_chars
num_chars[str]
The number of (Unicode) characters in a string, i.e. the length of the string.
A character is also known as a “Code Point” in the Unicode specification.
Example:
num_chars["abcd"] = 4
num_chars["中文例子"] = 4
Definition
@inline
def num_chars = rel_primitive_num_chars
Number
Number(x)
Holds if x
is a number (for example, Int
,Float
, or their fixed-width versions).
See also: Int
and Float
in intrinsics.rel
.
Example:
Integrity constraint that tests whether x
is of type Number
(throws if x
is not of type Number
):
def R = {1; 1.5; uint[8,3]; 4; float[32, 1.321]}
ic number_ic { subset(R, Number) }
Definition
@inline
def Number = rel_primitive_Number
numerator
numerator[x]
Numerator of a rational-like number.
Examples:
numerator[rational[64, 1, 3]] = 1
numerator[rational[64, 1, 100]] = 1
numerator[rational[64, 1, 100] + rational[64, 1, 3]] = 103
numerator[parse_decimal[64, 2, "3.14"]] = 314
numerator[parse_decimal[64, 5, "3.14159"]] = 314159
numerator[5] = 5
Definition
@inline
def numerator = rel_primitive_numerator
pack
pack(x₁, ..., xₙ, v)
Creates a packed structure v
storing the values (x₁, ..., xₙ)
in sequence.
pack
cannot be used with point-free syntax.
Direct usage of pack
should be avoided, as it serves primarily as a low-level interface for value types.
Definition
@inline
def pack = rel_primitive_pack_logical
parse_date
parse_date[d, format]
Parse Date.
This function is only defined for a valid combination of date string d
and format string format
, and invalid arguments will evaluate to false
(the empty relation, {}
).
Examples:
def output = parse_date["2018-06-12", "Y-m-d"]
2018-06-12
For details on the format parameter, see the Julia documentation for Dates.DateFormat
.
Definition
@inline
def parse_date = rel_primitive_parse_date
parse_datetime
parsedatetime[datestring, format] parsedatetime(datestring, format, datetime)
Maps a (date_string, format)
pair, where date_string
contains the datetime information in a String and format
specifies the datetime format, to a Datetime value type holding the corresponding datetime information.
This function is only defined for a valid combination of date string date_string
and format string format
. Invalid arguments will evaluate to false
(the empty relation, {}
).
Examples:
def output = parse_datetime["2018-06-12 13:00 +00:00", "YYYY-mm-dd HH:MM zzzz"]
//output> Dates.DateTime("2018-06-12T13:00:00")
def output = parse_datetime["2018-03-11 01:00 America/New_York", "Y-m-d H:M Z"]
//output> Dates.DateTime("2018-03-11T06:00:00")
For details on the supported date format specified in format
, see the Julia documentation for Dates.DateFormat
.
Definition
@inline
def parse_datetime = rel_primitive_parse_datetime
parse_decimal
parse_decimal[bits, digits, string]
Parse a string representation of a fixed-point decimal number to a decimal[a,b]
The implementation does not consider a locale. The decimal separator is always .
and the thousands separator is always ,
. The thousands separator is optional.
If the string has more digits after the decimal point than supported by the b
parameter, then the number is rounded to the nearest number.
This function is only defined for valid arguments. For invalid arguments it will be false
(the empty relation, {}
).
Examples:
parse_decimal[64, 2, "3.14"]
FixedDecimal{Int64,2}(3.14)
parse_decimal[64, 1, "4.27"]
FixedDecimal{Int64,1}(4.3)
parse_decimal[64, 2, ""]
FixedDecimal{Int64,2}(0.00)
parse_decimal[64, 2, "1,234.567"]
FixedDecimal{Int64,2}(1234.57)
parse_decimal[64, 2, "invalid"]
{}
Definition
@inline
def parse_decimal = rel_primitive_parse_decimal
parse_float
parse_float[string]
Parse a string representation of a float to a Floating[64]
Note that this currently does not consider a locale (decimal separator is always .
)
Is false
(empty) if the string fails to parse as an integer.
Examples:
parse_float["3.14"] = 3.14
parse_float["3"] = 3.0
empty(parse_int["hello"])
Definition
@inline
def parse_float = rel_primitive_parse_float
parse_int
parse_int[string]
Parse a string representation of an integer to a SignedInt[64]
Is false
(empty) if the string fails to parse as an integer.
Example:
parse_int["123"] = 123
empty(parse_int["hello"])
Definition
@inline
def parse_int = rel_primitive_parse_int
parse_json
parse_json[json_string_value]
Parses a JSON value directly from a string.
Example:
def json = parse_json["""{"name": "Anton", "age": 56}"""]
Definition
@inline
def parse_json = rel_primitive_parse_json
parse_uuid
parse_uuid[str]
Parse a UUID string (in the standard 8-4-4-4-12 format) to a UInt128 value.
Example:
parse_uuid["22b4a8a1-e548-4eeb-9270-60426d66a48e"] = 0x22b4a8a1e5484eeb927060426d66a48e
Definition
@inline
def parse_uuid = rel_primitive_parse_uuid
pattern_match
pattern_match(pattern, string)
Match string with a pattern (pre-compiled regular expression)
Example:
def p = regex_compile["^.*@.*$"]
def output = pattern_match(p, "foo@example.com")
true
Definition
@inline
def pattern_match = rel_primitive_pattern_match
percentile
percentile[R, p]
Select the element of R
corresponding to the p-th percentile, computed with linear interpolation.
If the p-th percentile does not correspond to a single element, then percentile_continuous
computes a weighted average of the two values surrounding the p-th percentile.
There are a number of different approaches for computing linearly interpolated percentiles. Here, the rank of the percentile is computed by x = p(N - 1) + 1
, which corresponds to the C = 1
variant in https://en.wikipedia.org/wiki/Percentile. This is also the approach used in Julia’s libraries and NumPy.
Note that percentile
is not defined for p <= 0
or p > 100
.
Example:
def output = percentile[{(1, 12); (2, 24); (3, 16)}, 60]
17.6
Definition
@inline
def percentile[R, p] = value :
// index is a float value, we multiply by count[R] - 1 first rather than divide by
// 100.0 first to avoid imprecision
index = (p * (count[R] - 1.0)) / 100.0 + 1.0 and
// compute the lower and upper bounds surrounding the value for the p-th percentile
value = sum[i, s:
// the value at index i contributes (1 - abs[i - index]) proportion of the final
// percentile
s = v * (1 - abs[i - index]) and
// sort[y, x...] instead of sort[last[R]] to handle duplicate values
sort[y, x...: R(x..., y)](i, v, rest...) and
// only for the values surrounding index
index - 1.0 <= 1.0 * i <= index + 1.0
from v, rest...
]
from index
percentile_nearest
percentile_nearest[R, p]
Select the element of R
corresponding to the p-th percentile.
The p-th percentile is the smallest value in the list such that no more than p
percent of the data is strictly less than the value and at least p
percent of the data is less than or equal to that value (nearest-rank method in https://en.wikipedia.org/wiki/Percentile). This function computes the p-th percentile based on ranking the last argument of R
.
Note that percentile_nearest
is not defined for p <= 0
or p > 100
.
Example, here 33% of the values are strictly less than 16, and 66% of the values are less than or equal to 16, so the 60-th percentile is 16:
def output = percentile_nearest[{(1, 12); (2, 24); (3, 16)}, 60]
16
Definition
@inline
def percentile_nearest[R, p] =
// we need to sort[y, x...] instead of sort[last[R]] to handle duplicates properly
first[(sort[y, x...: R(x..., y)][i] from i
where i = float_int_convert[ceil[(p / 100.0) * count[R]]])]
period_add
period_add[period1, period2]
Add two periods. For now, they need to be of the same type.
def output = period_add[Minute[3], Minute[4]]
Minute[7]
Definition
@inline
def period_add = rel_primitive_add
period_day_to_int
period_day_to_int[day]
Conversion from period Day
to int
.
In other words, returns the value of Day
data type as an integer.
Example:
def num_of_days = period_day_to_int[Day[3]]
ic { equal(num_of_days, 3) }
Definition
@inline
def period_day_to_int = rel_primitive_period_days
period_max
period_max[period1, period2]
Maximum of two periods. The result is expressed as a nanosecond period.
Example:
def output = period_max[Minute[3],Minute[4]] = Minute[4]
true
Definition
@inline
def period_max[x,y] = rel_primitive_max_period[x,y]
period_min
period_min[period1, period2]
Minimum of two periods. The result is expressed as a nanosecond period.
Example:
def output = period_min[Minute[3],Minute[4]] = Minute[3]
true
Definition
@inline
def period_min[x,y] = rel_primitive_min_period[x,y]
pi_float64
pi_float64
The constant pi.
Example:
cos[pi_float64] = -1.0
Definition
@inline
def pi_float64 = 3.14159265358979323846
pivot
pivot[R]
Associates each element of an input tuple with its index in the tuple. The result is always a binary relation.
pivot[(t1, ..., tn)]
generates pairs (1, t1);... (i, ti); ... (n, tn)
This is useful to match up long tuples with names (see zip
), for example.
Example:
def output = pivot[{(5, 60, 7, 2)}]
(1, 5)
(2, 60)
(3, 7)
(4, 2)
pivot[true]
and pivot[false]
are both false
(the empty relation) For unary relations, pivot[{(x)}]
is {(1,x)}
Definition
@inline
def pivot[R] = rel_primitive_pivot[R]
pop_standard_deviation
pop_standard_deviation[R]
pop_stddev[R]
Population standard deviation
Definition
@inline
def pop_standard_deviation[F] = sqrt[pop_variance[F]]
@inline
def pop_stddev = pop_standard_deviation
pop_variance
pop_variance[R]
Population variance
Definition
@inline
def pop_variance[F] = mean[squared_deviation[F]], count[F] > 1
pop_zscore_normalization
pop_zscore_normalization[R]
Z-score normalization (population) of (the last argument of) the relation R
.
Definition
@inline
def pop_zscore_normalization[R][x...] =
(R[x...] - mean[R]) / pop_stddev[R]
power
power[x, y]
x ^ y
Exponentiation: to the power of .
Examples:
power[2, 3] = 8
power[3.0, 2] = 9.0
power[9, 0.5] = 3.0
Definition
@inline
def power[x, y] = rel_primitive_power[x, y]
@inline
def (^)(x, y, z) = power(x, y, z)
prefix_join
prefix_join[R, S]
R <: S
The prefix join (or restriction) of S
to R
consists of the tuples (x..., y...)
in S
where the prefix (x...)
is in R
.
That is, R <: S
contains the tuples in S
that have a prefix in R
.
Example | Normalized |
---|---|
n <: edge | x, y: edge(x, y) and x = n |
female <: parent | x, y: female(x) and parent(x, y) |
t <: players | x, p: players(x, p) and t = x |
t.players <: age | p, v: players(t, p) and age(p, v) |
intern <: salary | p, v: intern(p) and salary(p, v) |
The restriction operator can also be used to select subsets of JSON-like relations:
:a <: json
wherejson
is{"a": {"b": 1, "c": 2}, "d": 3}
has value{"a": {"b": 1, "c": 2}}
(:[], _, :a) <: json
wherejson
is[ {"a": 1, "b": 2}, {"a": 3, "b": 4} ]
has value[ {"a": 1}, {"a": 3} ]
.
Definition
@inline
def prefix_join[R, S](x..., y...) = R(x...) and S(x..., y...)
@inline
def (<:) = prefix_join
product
product[R]
Product of (the last argument of) the relation R, if non-empty. If R is empty, product[R] is false
.
Example:
product[{(1, 4); (2, 5)}] = 20
Definition
@inline
def product[R] = rel_primitive_reduce_noinit[*, R]
proper_subset
proper_subset(R, S)
R ⊂ S
R
is a proper subset of S
if subset(R, S)
and R
and S
are not equal. This means there has to be at least one fact in S
that is not in R
.
Definition
@inline
def proper_subset(R, S) = subset(R, S) and not equal(R, S)
@inline
def (⊂) = proper_subset
proper_superset
proper_superset(R, S)
R ⊃ S
Inverse of propersubset. See propersubset.
Definition
@inline
def proper_superset(R, S) = superset(R, S) and not equal(R, S)
@inline
def (⊃) = proper_superset
rad2deg
rad2deg[x]
Convert radians to degrees.
Definition
@inline
def rad2deg[x] = x / pi_float64 * 180.0
random_mersenne_twister
random_mersenne_twister[seed, R]
Generates a pseudorandom number for every tuple in R
using the Mersenne Twister PRNG.
random_mersenne_twister
takes a relation R(xs...)
and produces a relation with the tuples (xs..., r)
where r
is a pseudorandom number generated by using the Mersenne Twister algorithm with the given seed
. The value r
is a Float64 in the range of [0, 1).
The seed is required to be a number convertible to an unsigned 128-bit integer (UInt128
) or a DateTime
. For a seed dt
of type DateTime
the value of epoch_milliseconds[dt]
is used as seed.
Please note that if seed
has more than one value, then the smallest value in seed
will be used as the seed (according to the native sort order that is used by sort
). If seed
is empty, then the result is empty.
Example:
def output = random_mersenne_twister[0, true]
(0.8236475079774124,)
def output = random_mersenne_twister[random_uint128, 'a']
('a', 0.590845)
def sample = {'a'; 'b'; 'c'}
def output = random_mersenne_twister[random_uint128, sample]
('a', 0.590845)
('b', 0.766797)
('c', 0.566237)
def output = random_mersenne_twister[datetime_now, true]
(0.09946255273771532,)
Definition
@inline
def random_mersenne_twister[SEED, R] =
rel_primitive_mersenne_twister[
// We specifically need a single value in the SEED, which is why we take the top-1.
// We need to take the conjunction with `R` because otherwise we may not get the
// same number of groups, which is not supported by the foreign function
// implementation.
(_internal_convert_seed[top[1, SEED, 1]], exists(R)),
// The restriction to the SEED is needed to make sure that the SEED is not empty for
// the current group. This is a limitation in the implementation of the foreign
// function.
(R, exists(_internal_convert_seed[top[1, SEED, 1]]))]
@inline
def _internal_convert_seed[x in Number] = uint128[x]
@inline
def _internal_convert_seed[x in DateTime] = uint128[epoch_milliseconds[x]]
random_threefry_float64
random_threefry_float64[key1, key2]
Generates a pseudorandom number of type Float64
between 0.0 and 1.0 using the Threefry algorithm.
The Threefry algorithm depends on two keys: key1
and key2
of type UInt64
or Int64
. One key can be used as the seed. The other key can be used as the stream position. Which key is used for what is up to the user.
Examples:
Getting a single random Float64
number:
def output = random_threefry_float64[1234, 1]
0.14708645730224323
Generating a sequence of random Float64
numbers using a user-specific seed:
def output = random_threefry_float64[1234, i] for i in range[1, 3, 1]
(1, 0.14708645730224323)
(2, 0.18124311325337028)
(3, 0.14463837673485513)
Using a randomly generated (but deterministic) seed via random_threefry_uint64
:
def seed = random_threefry_uint64[1234, 0]
def output = random_threefry_float64[seed, i] for i in range[1, 3, 1]
(1, 0.2959011588531637)
(2, 0.5971368640131505)
(3, 0.4259318598026802)
Generating a random Float64
number from the indeterministic random seed random_uint64
:
def output = random_threefry_float64[random_uint64, 0]
Definition
@inline
def random_threefry_float64[key1, key2] = rel_primitive_threefry[key1, key2] . rel_primitive_threefry_uniform
random_threefry_uint64
random_threefry_uint64[key1, key2]
Generates a pseudorandom number of type UInt64
using the Threefry algorithm.
The Threefry algorithm depends on two keys: key1
and key2
of type UInt64
or Int64
. One key can be used as the seed. The other key can be used as the stream position. Which key is used for what is up to the user.
Note that the output of random_threefry_uint64
can be used as a key again in a subsequent call. This capability allows the user to start a new random sequence from a previous sequence without the need to track which keys have already been used.
Examples:
Getting a single random UInt64
number:
def output = random_threefry_uint64[1234, 1]
0xbf025a77543cc31d
Generating a sequence of random UInt64
numbers:
def output = random_threefry_uint64[1234, i] for i in range[1, 3, 1]
(1, 0xbf025a77543cc31d)
(2, 0xe532e65f2dc0c673)
(3, 0x1bb25070549d29e7)
Generating two sequences of random UInt64
numbers by creating a new seed from the previous seed:
def seed1 = random_threefry_uint64[1234, 0]
def seed2 = random_threefry_uint64[seed1, 0]
def output:one = random_threefry_uint64[seed1, i] for i in range[1, 3, 1]
def output:two = random_threefry_uint64[seed2, i] for i in range[1, 3, 1]
(:one, 1, 0x2ca4bc02da81f726)
(:one, 2, 0x12898ddf6262c27b)
(:one, 3, 0xd606d09ded02d4dd)
(:two, 1, 0x48879dd755a393be)
(:two, 2, 0xc5683a27c1e876a7)
(:two, 3, 0xfdb025894557c350)
Definition
@inline
def random_threefry_uint64 = rel_primitive_threefry
random_uint128
random_uint128
Random number generated using the random device. It is an unsigned 128-bit integer.
A single random number is generated using the random device of the operating system for every transaction. Within the transaction, the random number does not change. The random number can be used as the seed of a pseudo-random number generator.
Examples:
def output = random_uint128
(0x648fa1de9056c1c7de7fc61bc138f5a8,)
Please note that random numbers need to be used with caution in views that need to be materialized or even incrementally maintained. If a view depends on a random number, then it needs to be recomputed every time the view is needed because the random number changes with every transaction and the view is always be consistent with its inputs.
If a random seed should not change with every transaction, then the state of a seed should be managed explicitly as an EDB. For example, a seed uses for a PRNG can be initialized in one transaction and then the same value is used from that point. In this way, materialized views involving random numbers can be reused.
Definition
@inline
def random_uint128 = rel_primitive_transaction_edb[:txn_random_uint128]
random_uint64
random_uint64
Random number generated using the random device. It is an unsigned 64-bit integer.
This is a truncation of random_uint128
to the lower 64 bits. It does not provide an additional source of random numbers.
Please check out the documentation for random_uint128
to get more information about how to use random numbers in various situations.
Example:
def output = random_uint64
(0xde7fc61bc138f5a8,)
Definition
@inline
def random_uint64 = uint128_uint64_truncate[random_uint128]
range
range(low, hi, stride, x)
Generate a relation with integer values between low
and high
, inclusive, advancing by stride
each time.
- If
low <= hi
andstride > 0
, thenrange[low, hi, stride]
has all the valuesx = low + i * stride <= hi
for non-negative integersi
. - Otherwise (i.e. if
low > hi
orstride <= 0
), thenrange[low, hi, stride]
is empty.
Example:
def output = range[1, 10, 4]
1
5
9
Definition
@inline
def range = rel_primitive_range
rational
rational[n, num, denom]
The n
-bit rational value from integer numerator num
and denominator denom
.
Definition
@inline
def rational = rel_primitive_rational
Rational
Rational(nbits, x)
Holds if x
is an nbits
rational constructed with rational[nbits, numerator, denominator]
.
Example:
Query that checks that x
is of type Rational with a bit size of nbits
, a numerator of -5
, and a denominator of -7
:
def my_rational = rational[16, -5, -7]
def output(x) = my_rational(x) and Rational(16, x)
//output> 5/7
Definition
@inline
def Rational = rel_primitive_Rational
rational_convert
rational_convert[n, x]
Convert a number x
to a rational with n
-bit integer representation.
Any type that supports numerator
and denominator
is supported.
Examples:
rational_convert[64, 3] = (3//1,)
rational_convert[64, decimal[64, 2, 0.75]] = (3//4,)
Definition
@inline
def rational_convert[n, a] =
rational[n, numerator[a], denominator[a]]
regex_compile
regex_compile(regex, pattern)
Compile a regular expression to a pattern. If regex
is a pattern, rather than a string, then pattern = regex
.
Example:
def p = regex_compile["^.*@.*$"]
pattern_match(p, "foo@example.com")
()
Definition
@inline
def regex_compile = rel_primitive_regex_compile
regex_match
regex_match(regex, string)
Match string with a regular expression (string or pattern).
Note that if the entire string needs to be matched, then a leading ^
and trailing $
are required in the regular expression.
Example:
regex_match("^.*@.*$", "a@b")
Definition
@inline
def regex_match = rel_primitive_regex_match
regex_match_all
regex_match_all[regex, input_string]
All the occurrences of a regular expression regex
(string or pattern) in string
, including all the matches and offsets.
Each match is a pair (i, s)
with the character index i
where that matching string starts and the matching string s
.
Example:
regex_match_all["(?<hour>\\d+):(?<minute>\\d+)",
"Appointment from 12:45 to 1:30"]
is equal to
{(19, "12:45"); (28, "1:30")}
Definition
@inline
def regex_match_all[re, str] = rel_primitive_regex_match_mv[regex_compile[re], str]
rel:integration:load_partitioned_csv
rel:integration:load_partitioned_csv[URI_string]
rel:integration:load_partitioned_csv[R]
This is the main in-language API for loading partitioned CSV data into a single logical relation.
For detailed information on CSV loading, see load_csv.
To load a partitioned CSV, specify the following:
:path
: The path to the first CSV partition,
e.g. "azure://storage.blob.core.net/container/file_1_of_24.csv"
. For more than one partition to get picked up, please note the naming schema: Name partitions using [...]_m_of_n[...]
, where m
is the current partition and n
the total number of partitions (e.g. mycsv_1_of_16.csv
, mycsv_2_of_16.csv
, etc.).
- Only the first partition should include header information.
- Make sure all partitions are stored under the same
:path
prefix.
The resulting relation is of the form (column_name, partition, line, value)
. partition, line
uniquely identifies each row in the CSV.
If errors occur, the resulting relation is of the form (:load_errors, partition, line, error_column, raw_line)
.
partition
and line
are UInt32s. To address them, please use uint[32, <number>]
.
If only a single partition is provided or data is directly passed into the load, the resulting relation looks just like a load_csv
call with an added partition = uint[32, 1]
.
Definition
@inline
def rel:integration:load_partitioned_csv = rel_primitive_load_part_csv
RelName
RelName(x)
Holds if x
is a relation name.
Examples:
Integrity constraint that specifies that a module always contains a RelName
and an integer:
def mymodule:f = 1
ic { subset(mymodule, (RelName, Int)) }
def output = mymodule
//output> (:f, 1)
Definition
@inline
def RelName = rel_primitive_RelName
relname_string
relnamestring[relationname] relnamestring(relationname, string)
Maps a RelName (relation_name
) to its corresponding String value (string
). Contains relation_name
and string
when relation_name
matches string
.
Relation names (aka RelNames), when treated as data, start with the symbol :
.
Can be used to
- Verify that a RelName and string match.
relname_string(relation_name, string)
evaluates totrue
ifrelation_name
matchesstring
. Ifrelation_name
does not matchstring
,relname_string(relation_name, string)
evaluates tofalse
(the empty relation,{}
). - Convert a RelName to a string (relnamestring[relationname] provides shorthand for this use).
Note: converting a string to a RelName is not supported.
Examples:
Verify that a RelName matches a string.
relname_string(:rel, "rel")
//evaluates to `true` because the RelName `:rel` matches the string `"rel"`.
Convert RelName :employees
to string:
def output = x: relname_string(:employees, x)
//output> "employees"
Convert RelName :employees
to string using equivalent shorthand:
def output = relname_string[:employees]
Definition
@inline
def relname_string = rel_primitive_relname_string
remainder
remainder[x, y]
x % y
Remainder from Euclidean division: a value with the same sign as x, and smaller in magnitude than y. This value is always exact.
Satisfies x = (x / y) * y + (x % y)
.
Is false
(empty) when y = 0
for Integer arguments.
Examples:
8 % 3 = 2
remainder[8, 3] = 2
remainder[8, -3] = 2
remainder[-8, 3] = -2
empty(remainder[6, 0])
remainder[x, y]
is defined when either the types of x
and y
are the same, or either operand is of type SignedInt[64]
or Floating[64]
. The return type is:
- the type of
x
when the types of both operands are the same. SignedInt[n]
orUnsignedInt[n]
wheren
is the greater of the number of bits in the types ofx
andy
and the signedness is chosen to match that of the type ofx
, and bothx
andy
are of integer types.Floating[64]
when either the type ofx
or the type ofy
isFloating[64]
.- The type of
x
whenx
is a non-integer type andy
is of typeSignedInt[64]
. - The type of
y
wheny
is a non-integer type andx
is of typeSignedInt[64]
. Rational[n]
wheren
is the greater of the number of bits in the types ofx
andy
, and one operand is a rational type and the other is aSignedInt[64]
.
Definition
@inline
def remainder[x, y] = rel_primitive_remainder[x, y]
@inline
def (%)(x, y, z) = remainder(x, y, z)
reverse_sort
reverse_sort[R]
Reverse Sort
Like sort
, except the ordering is reversed.
Example:
def sample = {'a'; 'b'; 'c'}
def output = reverse_sort[sample]
(1, 'c')
(2, 'b')
(3, 'a')
def sample = {(:g, 'a'); (:g, 'b'); (:g, 'c'); (:h, 'd'); (:h, 'e'); (:h, 'f')}
def output = x: reverse_sort[sample[x]]
(:g, 1, 'c')
(:g, 2, 'b')
(:g, 3, 'a')
(:h, 1, 'f')
(:h, 2, 'e')
(:h, 3, 'd')
Definition
@inline
def reverse_sort[R](ri, args...) =
sort[R](i, args...) and
ri = count[R] - i + 1
from i
right_override
right_override[R, S]
R ++> S
Right override is identical to left override, except the arguments are swapped.
R ++> S
contains all of S
, plus all the tuples in R
whose key is not in S
.
There are two equivalent ways to think about R ++> S
: (1) R
is providing default values for keys that are missing in S
. (2) The entries in S
are overriding the entries in R
.
A few examples to illustrate the difference with left-override:
Example | Value |
---|---|
2 ++> 3 | 3 |
(1,2) ++> (1,3) | (1,3) |
(3,4) ++> (1,2); (3,5) | (1,2); (3,5) |
(3,"abc") ++> (1,2); (3,5) | (1,2); (3,5) |
(1,2); (3,5) ++> (3,4); (6,7) | (1,2); (3,4); (6,7) |
Definition
@inline
def right_override[R, S](xs...) = S(xs...)
@inline
def right_override[R, S](xs..., v) = R(xs..., v) and not S(xs..., _)
@inline
def (++>) = right_override
rmse
rmse[YHAT, Y]
Root mean square error (RMSE)
Definition
@inline
def rmse[YHAT,Y] = sqrt[mse[YHAT,Y]]
round
round[mode, x]
Round to the nearest integer according to the given rounding mode.
Definition
@inline
def round(mode, x, rounded) = {
(mode = :ROUND_UP, rounded = rel_primitive_round_up[x]);
(mode = :ROUND_DOWN, rounded = rel_primitive_round_down[x]);
(mode = :ROUND_NEAREST, rounded = rel_primitive_round_nearest[x]);
(mode = :ROUND_NEAREST_TIES_AWAY, rounded = rel_primitive_round_nearest_ties_away[x]);
(mode = :ROUND_NEAREST_TIES_UP, rounded = rel_primitive_round_nearest_ties_up[x]);
(mode = :ROUND_TO_ZERO, rounded = rel_primitive_round_to_zero[x]) }
sample_standard_deviation
sample_standard_deviation[R]
sample_stddev[R]
Sample standard deviation of (the last argument of) the relation R
.
Definition
@inline
def sample_standard_deviation[F] = sqrt[sample_variance[F]]
sample_stddev
see: sample_standard_deviation
Definition
@inline
def sample_stddev = sample_standard_deviation
sample_variance
sample_variance[R]
Sample variance
Definition
@inline
def sample_variance[F] = sum[squared_deviation[F]] / (count[F] - 1), count[F] > 1
sample_zscore_normalization
sample_zscore_normalization[R]
Z-score normalization (sample) of (the last argument of) the relation R
.
Often simply referred to as ‘standardization’.
Definition
@inline
def sample_zscore_normalization[R][x...] = (R[x...] - mean[R]) / sample_stddev[R]
Second
Second[n]
A period that spans n
seconds. The period is given in units of Second
.
Example:
datetime_add[unix_epoch, Second[100]]
Definition
@inline
def Second = rel_primitive_second
second
second[R]
Projection to the second argument of R.
second
supports heterogeneous relations, both in arity and types.
Examples:
def output = second[(1, 2, 3); (4, 5)]
2
5
def output = second[(1, 2); (3, "abc")]
2
"abc"
Definition
@inline
def second[R](x) = ∃(y... : R(_, x, y...))
sign
sign[x]
The sign of x: either 0, -1, or 1.
Definition
@inline
def sign[x] = rel_primitive_sign[x]
SignedInt
SignedInt(nbits, x)
Holds if x
is an nbits
signed integer.
Examples:
Integrity constraint that tests whether x
is an 8-bit signed integer:
def R = int[8, 5]
ic unsigned_type_check(x in R) {
SignedInt(8, x)
}
Definition
@inline
def SignedInt = rel_primitive_SignedInt
sin
sin[x]
Sine of x (given in radians).
Defined for non-infinite x
.
Example:
sin[pi_float64/2] = 1.0
Definition
@inline
def sin[x] = rel_primitive_sin[x]
sinh
sinh[x]
Hyperbolic sine.
Definition
@inline
def sinh[x] = rel_primitive_sinh[x]
sort
sort[R]
Sort a relation.
sort
takes a relation R(xs...)
and produces a relation R(i, xs...)
where i
is an Int64
index starting at 1
. The index indicates the ordering of the tuples of R
.
The relation should have a fixed arity. Otherwise, the result is empty.
The ordering is the native sort order of values in the system and cannot be customized with a comparator. It is possible though to transform the input or output of sort to achieve a different sort order (see reverse_sort
for an example).
Example:
def sample = 'a'
def output = sort[sample]
(1, 'a')
def sample = {'a'; 'b'; 'c'}
def output = sort[sample]
(1, 'a')
(2, 'b')
(3, 'c')
Similar to aggregations, it is possible to separately sort per group. In the following example, the sort is grouped by free variable g
.
def sample = {(:g, 'a'); (:g, 'b'); (:g, 'c'); (:h, 'd'); (:h, 'e'); (:h, 'f')}
def output = x: sort[sample[x]]
(:g, 1, 'a')
(:g, 2, 'b')
(:g, 3, 'c')
(:h, 1, 'd')
(:h, 2, 'e')
(:h, 3, 'f')
Definition
@inline
def sort = rel_primitive_sort
soundex
soundex[string_value]
soundex
returns a 4-letter encoding the phonetic representation of string_value
. soundex
lets you compare words and sentences based on how they sound in English.
string_value
could be any arbitrary long string, but only the first few syllables are considered for the result
Examples:
def output = soundex["Smith"]
"s530"
def output = soundex["Smythe"]
"s530"
def output = soundex["Christina"]
"c623"
def output = soundex["Cristine"]
"c623"
Here a simple illustration. Consider the following:
def my_favorite_bands = { "Metallica"; "Iron Maiden"; "Wolfheart"; "Insomium" }
def do_I_like = "Mee ta li ka"
def output = exists({ y : soundex[my_favorite_bands][y] and equal(y, soundex[do_I_like]) })
the script above evaluates to true since "Mee ta li ka"
and "Metallica"
have the same soundex code.
Definition
@inline
def soundex = rel_primitive_soundex
spread
spread[mode, R, val]
This definition unifies the spread definitions on integers and FixedDecimal values using the two spreading strategies. Namely, this definition wraps FixedDecimal_spread
, int_spread_by_even
, and int_spread_by_ratio
.
mode
can be :even
for even spread, or :ratio
for ratio spread. R
is the relation to execute the spread on. val
is the value to spread.
Example:
def R = {("Atlanta", "Georgia", 50);
("Seattle", "Washington", 80);
("San Francisco", "California", 10)}
def out1 = spread[:even, R, decimal[64, 3, 31.0]]
("Atlanta", "Georgia", 50, 10.334)
("San Francisco", "California", 10, 10.333)
("Seattle", "Washington", 80, 10.333)
def out2 = spread[:even, R, 31]
("Atlanta", "Georgia", 50, 11)
("San Francisco", "California", 10, 10)
("Seattle", "Washington", 80, 10)
Definition
@inline
def spread[mode, R, val](v...) =
(
(Int(val) and mode = :even and int_spread_by_even[R, val](v...)) or
(Int(val) and mode = :ratio and int_spread_by_ratio[R, val](v...)) or
(
rel_primitive_decimal_type(val, decimal_bit_length[val], decimal_precision[val]) and
FixedDecimal_spread[mode, R, val](v...)
)
)
sqrt
sqrt[x]
The non-negative square root of x
.
Defined for non-negative x
.
Definition
@inline
def sqrt[x] = rel_primitive_sqrt[x]
squared
squared[R]
Square of a relation: the value of each last element is squared.
Examples:
def example = {(1, 2); (3, 4)}
equal(square[example], {(1, 4); (3, 16)}
Definition
@inline
def squared[F][x...] = F[x...] ^ 2
squared_deviation
squared_deviation[R]
Squared deviation: squared deviation from the mean of (the last argument of) the relation R
.
Definition
@inline
def squared_deviation[F][x...] = (F[x...] - mean[F]) ^ 2
string
string[x]
Convert ints, floats, dates, periods, to strings:
Examples:
string[1] = "1"
string[3.4] = "3.4"
string[unix_epoch] = "1970-01-01T00:00:00"
string[Hour[1]] = "1 hour"
string["a"] = "a"
string[:a] = "a"
Definition
@inline
def string = rel_primitive_string
String
String(x)
Holds if x
is a String
. Strings are specified by double quotes, for example, "abc"
.
Examples:
Integrity constraint that tests whether x
is of type String
:
def R = "foo"
ic string_type_check(x in R) {
String(x)
}
Schema defined in a relation using String
:
def myrelation(x in String, y in Int) {
x = "abc" and y = 123
}
def output = myrelation
//output> (abc, 123)
Definition
@inline
def String = rel_primitive_String
string_join
string_join[separator, R]
Concatenate together all the String values in the second argument of the binary relation R
, if non-empty, separated by the separator
string, following the ordering provided by the Ints in the first argument of R. If R is empty, string_join[sep, R]
is false
.
R
is required to be a binary relation, containing only (Int, String)
tuples, where the Int serves as the sort key for the concatenation. Otherwise, string_join[sep, R]
is false
. (Note: this restriction may be lifted in the future.)
Example:
string_join[", ", {(1, "a"); (2, "b"); (3, "c")}] = "a, b, c"
string_join[" ", {(1, "hello"); (2, "world")}] = "hello world"
string_join["->", sort[{"x"; "y"; "z"}]] = "x->y->z"
Definition
@inline
def string_join[sep, R] =
rel_primitive_reduce_noinit[_str_join2[sep], R],
// R is required to be a Binary relation with numeric keys to provide an ordering
R ⊆ (Int, String)
// This helper function is reduced over the values in R, with the provided separator.
@inline
def _str_join2[sep, a,b] = concat[concat[a, sep], b]
string_length
string_length[string]
DEPRECATED
This function is deprecated. Please prefer num_chars[string]
or num_bytes[string]
, depending on your need.
Example:
string_length["foo"] = 3
Definition
@inline
def string_length = num_chars
string_replace
string_replace[string, old_str, new_str]
Replace a string or character in string with the specific string or character.
Examples:
string_replace["One Two Three", "One", "1"] = "1 Two Three"
string_replace["Rel", 'R', 'r'] = "rel"
Definition
@inline
def string_replace = rel_primitive_replace
string_replace_multiple
string_replace_multiple[input, R]
Replaces all occurrences of old
string in input
with the corresponding new
string, for each tuple (old, new)
in relation R
.
Examples:
string_replace_multiple["One Two Three", {("One", "1"); ("Two", "2")}]
// "1 2 Three"
string_replace_multiple["One Two Three", {('O', 'o'); ('T', 't')}]
// "one two three"
The behavior is undefined if new
generates more occurrences of old
.
Definition
@inline
def string_replace_multiple[input, R] = rel_primitive_reduce[(x,y,z: z=string_replace[x,y,R[y]]), input, (x: R(x,_))]
string_split
string_split[delimiter, text]
Split the string text
into the substrings delimited by delimiter
(and the start/end of text
). The expression string_split[delimiter, string]
evaluates to a relation of the format (index, sub_string)
where index
is the substring’s position of type Int
and sub_string
is the matched substring of type String
. delimiter
can be a String
or a Char
.
Example:
string_split['@', "user@email.com"] = {(1, "user"); (2, "email.com")}
string_split[" ", "a b c"] = {(1, "a"); (2, "b"); (3, ""); (4, "c")}
Definition
@inline
def string_split(dlm, text, i, sub_string) =
dlm_string = string[dlm] and
_delims(dlm_string, text, i, pos) and
_delims(dlm_string, text, i + 1, next_pos) and
substring[text, pos + num_chars[dlm_string], next_pos - 1](sub_string) and
{String; Char}(dlm)
from dlm_string, pos, next_pos
// Helper gives us the positions of all delimeters in string `s`.
@inline
def _delims[dlm in String, s in String] =
sort[
-num_chars[dlm] + 1;
first[regex_match_all[escape_regex_metachars[dlm], s]];
num_chars[s] + 1
]
string_trim
string_trim[s]
To remove leading and tailing whitespaces in given string s
Example:
string_trim[" Good Day!!! "] = "Good Day!!!"
Definition
@inline
def string_trim[s] = string_replace[s, regex_compile["^\\s+|\\s+$"], ""]
subset
subset(R, S)
R ⊆ S
R
is a subset of S
if all facts in R
are also in S
.
Subset is the relational variant of implication, but note that there is a subtle difference for arity-overloaded relations due to usage of varargs.
forall(x, y: R(x, y) implies S(x, y))
is true if the implication is true for all binary facts ofR
. IfR
also contains facts of a different arity, then the implication remains true, even whenR ⊈ S
.subset(R, S)
, defined asforall(xs... where R(xs...) : S(xs...))
, is true only if there exists no fact inR
, of any arity, that is not also inS
.
Definition
@inline
def subset(R, S) = forall(x... where R(x...): S(x...))
@inline
def (⊆) = subset
substring
substring[string, index1, index2]
Substring of a string, selecting characters in range [index1:index2]
(both inclusive).
The index of the first character is 1.
When index1
and index2
are out of bounds on the same end, or index2
< index1
, the empty string is returned. Otherwise, both indexes are clamped in range.
Example:
substring["abcd", 2, 3] = "bc"
substring["abcd", 3, 2] = ""
substring["中文例子", 2, 3] = "文例"
substring["word", 0, 10] = "word"
Definition
@inline
def substring = rel_primitive_substring
substring_bytes
substring_bytes[string, index1, index2]
Extracts substring of a string
between the byte range [index1:index2]
(both inclusive). The byte index starts with 1.
When index1
and index2
are out of bounds on the same end, or index2
< index1
, the empty string is returned. Otherwise, both indexes are clamped in range.
Use substring
to extract substrings by character indexes.
Examples:
substring_bytes["abcd", 2, 3] = "bc"
substring_bytes["abcd", 3, 2] = ""
substring_bytes["中文例子", 2, 3] = """��"""
substring_bytes["word", 0, 10] = "word"
Definition
@inline
def substring_bytes = rel_primitive_substring_bytes
subtract
subtract[x, y]
x - y
Subtraction.
Examples:
8 - 5 = 3
subtract[2, 3] = -1
equal(subtract(8, 5, 3), true)
Definition
@inline
def subtract(x, y, z) = rel_primitive_subtract(x, y, z)
@inline
def (-)(x, y, z) = subtract(x, y, z)
suffix_join
suffix_join[R, S]
R :> S
The suffix join (or restriction) of R
to S
consists of the tuples (x..., y...)
in R
where the suffix (y...)
is in S
.
That is, R :> S
contains the tuples in R
that have a suffix in S
.
Example | Normalized |
---|---|
edge :> n | x, y: edge(x, y) and y = n |
parent :> female | x, y: parent(x, y) and female(y) |
account_balance :> positive | x, v: account_balance(x, v) and positive(v) |
brand_name :> "Tesla" | b, s: brand_name(b, s) and s = "Tesla" |
The restriction operator can also be used to select subsets of JSON-like relations:
json :> even
wherejson
is[ {"a": 1, "b": 2}, {"a": 3, "b": 4, "c": 6} ]
has value[ {"b": 2}, {"b": 4, "c": 6} ]
json :> (:b, even)
wherejson
is[ {"a": 1, "b": 2}, {"a": 3, "b": 4, "c": 6} ]
has value[ {"b": 2}, {"b": 4} ]
Definition
@inline
def suffix_join[R, S](x..., y...) = R(x..., y...) and S(y...)
@inline
def (:>) = suffix_join
sum
sum[R]
Σ[R]
Sum of (the last argument of) the relation R, if non-empty. If R is empty, sum[R]
is false
(empty). To get 0 instead, use left override, as in sum[R] <++ 0
.
Example:
def salary = {("John", 10) ; ("Mary", 20); ("Paul", 17) ; ("Peter", 15) }
def output = sum[salary]
62
def member = {("A", "John"); ("B", "Mary") ; ("A", "Paul") ; ("C" , "Peter") }
def department = {"A"; "B"; "C" }
def output = d in department: sum[salary[p] for p in member[d]]
("A", 27)
("B", 20)
("C", 15)
Definition
@inline
def sum[R] = rel_primitive_reduce_noinit[+, R]
sum_int
sum_int[R]
Integer sum of (the last argument of) the relation R, which should contain integers. sum_int[R] = 0
for empty R.
Definition
@inline
def sum_int[R] = rel_primitive_reduce[+, 0, R]
superset
superset(R, S)
R ⊇ S
Inverse of subset. See subset.
Definition
@inline
def superset(R, S) = forall(x... where S(x...): R(x...))
@inline
def (⊇) = superset
tan
tan[x]
Tangent of x (given in radians).
Defined for non-infinite x
.
Example:
tan[pi_float64/4] = 0.9999999999999999
Definition
@inline
def tan[x] = rel_primitive_tan[x]
tanh
tanh[x]
Hyperbolic tangent.
Definition
@inline
def tanh[x] = rel_primitive_tanh[x]
top
top[k, R]
Select the top k
facts of relation R
according to the sort order of R.
top
is sort
restricted to first k
tuples. top
takes a relation R(xs...)
and produces a relation R(i, xs...)
where i
is an Int64
index starting at 1
. The index indicates the ordering of the tuples of R
.
If k
is bigger than number of tuples in the relation R
, then top[k,R]
is the sorted relation itself. This means that the cardinality is the minimum of k
and the cardinality of R
.
Example:
def output = top[2, {'a'; 'b'; 'c'; 'd'}]
(1, 'a')
(2, 'b')
Note: Currently top[k, R]
only supports groupbys in the R
-expression but not in the k
-expression. For example, the following is supported:
def output[g] = top[K, R[g]] // `g` here is a groupby for `R`
However the following is NOT supported:
def output[g] = top[K[g], R] // `g` here is a groupby for `K`
Definition
@inline
def top[K, R] = rel_primitive_top[R, K]
total
total(D, F)
Given a domain D and a relation F, check that there is a value in F for every element of the domain D. The arity of F must be the arity of D + 1.
Definition
@inline
def total(D, F) =
forall(k... where D(k...): F(k..., _))
transpose
transpose[R]
Transpose a binary relation (swap the two arguments).
Example:
def output = transpose[{(1, 2); (3, 4)}]
(2, 1)
(4, 3)
Definition
@inline
def transpose[R](y, x) = R(x, y)
trunc
trunc[x]
Round toward zero to the nearest integer. Result has the same type as the argument.
Examples:
trunc[3.9] = 3.0
trunc[-4.9] = -4.0
trunc[4] = 4
Definition
@inline
def trunc[x] = round[:ROUND_TO_ZERO, x]
trunc_divide
trunc_divide[x, y]
Integer Division.
Quotient from Euclidean division. Computes x/y rounded towards zero.
Is false
(empty) when y = 0
for Integer arguments.
Examples:
trunc_divide[5, 2] = 2
trunc_divide[-5, 2] = -2
trunc_divide[5.1, 2] = 2.0
empty(trunc_divide[5, 0])
Definition
@inline
def trunc_divide[x, y] = rel_primitive_trunc_divide[x, y]
@inline
def (÷)(x, y, z) = trunc_divide(x, y, z)
trunc_to_int
trunc_to_int[x]
General float-to-int conversion, truncating. Argument must be a float.
Examples:
trunc_to_int[3.1] = 3
trunc_to_int[-3.1] = -3
trunc_to_int[3] : error
Definition
@inline
def trunc_to_int[x] = float_int_convert[trunc[x]]
uint
uint[n, v]
The n
-bit unsigned integer value from the integer value v
.
Definition
@inline
def uint = rel_primitive_uint
uint128
uint128[v]
The 128
-bit unsigned integer value from the integer value v
.
Definition
@inline
def uint128 = uint[128]
uint128_hash_value_convert
uint128_hash_value_convert[v]
Convert a provided UInt128 value v
to the HashValue type. This is an internal-facing function, used for the implementation of hashing.
Definition
@inline
def uint128_hash_value_convert[v] = rel_primitive_uint128_hash_value_convert[v]
uint128_uint64_truncate
uint128_uint64_truncate[v]
Truncates v
of type UInt128
to a UInt64
number.
Truncates the high order bits in v
and converts the remaining bits to UInt64. Truncate always succeeds. For values larger than 2^64, consists of only the lower bits.
Examples:
uint128_uint64_truncate[uint128[1]] = 1
uint128_uint64_truncate[uint128[2] ^ 64 + uint128[1]] = 1
Definition
@inline
def uint128_uint64_truncate = rel_primitive_uint128_uint64_truncate
uint64
uint64[v]
The 64
-bit unsigned integer value from the integer value v
.
Definition
@inline
def uint64 = uint[64]
union
union[R, S]
R ∪ S
Union two n-ary relations R and S
Definition
@inline
def union[R, S](x...) = R(x...) or S(x...)
@inline
def (∪) = union
unit_normalization
unit_normalization[R]
Unit vector normalization:
Definition
@inline
def unit_normalization[R][x...] = R[x...] / sqrt[sum[squared[R]]]
unix_epoch
unix_epoch
The beginning of the UNIX Epoch, midnight on 1 January 1970 (UTC).
Definition
@inline
def unix_epoch = parse_datetime["1970-01-01 00 UTC", "Y-m-d H Z"]
unpack
unpack(v, x₁, ..., xₙ)
Unpacks a packed structure v
into its arguments x₁, ..., xₙ
.
unpack
cannot be used with point-free syntax.
Direct usage of unpack
should be avoided, as it serves primarily as a low-level interface for value types.
Definition
@inline
def unpack(v, xs...) = rel_primitive_pack_logical(xs..., v)
UnsignedInt
UnsignedInt(nbits, x)
Holds if x
is an unsigned integer of the bit length nbits
.
Example:
Integrity constraint that tests whether x
is a 32-bit unsigned integer:
def my_unsigned_int = uint[32, 1234]
ic my_unsigned_ic(x) {
my_unsigned_int(x) implies UnsignedInt(32, x)
}
Definition
@inline
def UnsignedInt = rel_primitive_UnsignedInt
uppercase
uppercase[string_or_char]
A string where all the characters are converted to uppercase. If a character is already uppercase or has no uppercase version, it remains unchanged.
Example:
def output = uppercase["aB1c"]
"AB1C"
def output = uppercase['â']
'Â'
uppercase
does not take a locale option and does not handle local-specific case mapping rules.
Definition
@inline
def uppercase = rel_primitive_uppercase
uuid_string
uuid_string[v]
Convert a UInt128 value to the standard UUID format
Example:
uuid_string[0x22b4a8a1e5484eeb927060426d66a48e] = "22b4a8a1-e548-4eeb-9270-60426d66a48e"
Definition
@inline
def uuid_string[v in UInt128] = rel_primitive_uuid_string[v]
@inline
def uuid_string[v in Hash] = rel_primitive_uuid_string[hash_value_uint128_convert[v]]
Week
Week[n]
A period that spans n
weeks. The period is given in units of Week
.
Example:
datetime_add[unix_epoch, Week[10]]
Definition
@inline
def Week = rel_primitive_week
weighted_mean
weighted_mean[R, W]
The weighted mean of values in relation R
with weights defined by W
. The last attribute of the relation representing the weights is used to compute the weighted mean. When the arity of the relation that represents the weights is different from the arity of the input relation the user is expected to align the arities before passing them to weighted_mean
.(See examples 2 and 3)
Example 1 (The arity of both R
and W
are the same):
def inputs = {(1,'a',10); (2,'b',10); (3,'c',30); (4,'d',40); (5,'e',50)}
def weights = {(1,'a',11); (2,'b',12); (3,'c',13); (4,'d',14); (5,'e',15)}
def output = weighted_mean[inputs, weights]
29.692307692307693
Example 2 (The arity of R
and W
are different):
def inputs = {(1,'a',10); (2,'b',10); (3,'c',30); (4,'d',40); (5,'e',50)}
def weights = {(1,11); (2,12); (3,13); (4,14); (5,15)}
def output = weighted_mean[inputs, (n: _, weights[n])]
29.692307692307693
Example 3 (The arity of R
and W
are different):
def inputs = {(1,'a',100,10); (2,'a',200,10); (3,'b',300,30); (4,'c',400,40); (5,'b',500,50)}
def weights = {(1,'a',11); (2,'a',12); (3,'b',13); (4,'c',14); (5,'b',15)}
def output = weighted_mean[input, (n, c: _, weights[n,c])]
29.692307692307693
Definition
@inline
def weighted_mean[R,W] = sum[k...: R[k...] * W[k...]] / sum[domain[R] <: W]
xor
F xor G
F ≢ G
F ⇎ G
F ⊻ G
Exclusive OR, for boolean (arity 0, true or false) arguments F and G.
Definition
@inline
def xor(F, G) = not(F ≡ G)
@inline
def (⊻)(F, G) = F xor G
@inline
def (≢)(F, G) = F xor G
@inline
def (⇎)(F, G) = F xor G
Year
Year[n]
A period that spans n
years. The period is given in units of Year
.
Example:
datetime_add[unix_epoch, Year[10]]
Definition
@inline
def Year = rel_primitive_year
zip
zip[R, S]
Pair-wise combine two n
-ary tuples into a relation of n
pairs.
Example:
def foods = {(:hamburger, :chicken, :hot_dog, :fries, :macaroni, :pizza, :salad, :milk, :ice_cream)}
def cost = { (2.49, 2.89, 1.50, 1.89, 2.09, 1.99, 2.49, 0.89, 1.59) }
def output = zip[foods, cost]
(:chicken, 2.89)
(:fries, 1.89)
(:hamburger, 2.49)
(:hot_dog, 1.5)
(:ice_cream, 1.59)
(:macaroni, 2.09)
(:milk, 0.89)
(:pizza, 1.99)
(:salad, 2.49)
Definition
@inline def zip[R,S](x, y) = rel_primitive_pivot[R](i, x) and rel_primitive_pivot[S](i, y) from i
¬
not F
¬F
Logical negation, for boolean (arity 0, true or false) argument F.
Definition
@inline
def (¬)(F) = not F
Σ
sum[R]
Σ[R]
Sum of (the last argument of) the relation R, if non-empty. false
if R is empty.
Example:
d in department: Σ[salary[p] for p in member[d]]
Definition
@inline
def Σ = sum
⇒
F implies G
F ⇒ G
G ⇐ F
Logical implication, for boolean (arity 0, true or false) arguments F and G.
Definition
@inline
def (⇒)(F, G) = F implies G
@inline
def (⇐)(F, G) = G implies F
∧
F and G
F ∧ G
Logical and (conjunction).
Definition
@inline
def (∧)(F, G) = F and G
∨
F or G
F ∨ G
Logical or (disjunction), for boolean (arity 0, true or false) arguments F and G.
Definition
@inline
def (∨)(F, G) = F or G