The Math Optimization Library (mathopt)

A collection of math optimization tools.

Module: rel:mathopt

rel:mathopt

Integration of external mathematical optimization solvers within Rel.

This module implements a DSL that defines operators which allow for the specification of mathematical optimization problems within Rel. Additionally, it defines APIs to execute the specified models using different interpreters, which are different implementations of the DSL operators.

Expressing an Optimization Model

A mathematical optimization problem should be specified in 3 modules:

• a Data module defines the underlying data feeding into the problem.
• a Variables module declares variables to be solved for, usually based on the Data.
• a Model module uses the DSL operators to specify an objective function and a set of constraints that should be satisfied by a solution.

Note that you can use any name for those modules. A very simple (and very artificial) example could be the following:

module MyData
  // my data set has only a couple of integers
  def my_domain = 5 ; 7
end
 
@inline
module MyVariables[Data]
    with Data use my_domain
    // x is an integer variable
    def x:type = "integer"
    // create a variable for each value in the Data my_domain
    def x:keys = (my_domain)
end
 
@inline
module MyModel[Data, Variables, DSL]
    with Data use my_domain
    with Variables use x
    with DSL use sum, foreach, +, -, *, ≼, ≽, /, ∧, ∨, =
 
    // we will try to maximize the sum of the variables
    def objective = sum[x[d] for d in my_domain]
    // subject to an artificial constraint limiting their values
    def subject_to:artificial = foreach[my_domain, {d : d * x[d] ≼ 20 }]
end

With these definitions you can now use one of the interpreters. There are currently 3 implementations: solve, evaluate and debug.

Solve

rel:mathopt:solve uses an implementation that builds a representation of the model and sends it to an external, configurable solver. The results can be inspected with a call to rel:mathopt:extract. For example:

def result = rel:mathopt:solve[{}, MyModel, MyData, MyVariables]
def extracted = rel:mathopt:extract[result, MyData, MyVariables]

Note that currently it is necessary to materialize result, it cannot be @inlined. The extracted relation contains the objective function value and solution data, as well as various meta-data about the execution:

// the outcome of the execution (see below for possible values)
ic { extracted:outcome = 1 }
// the termination status of the execution (see below for possible values)
ic { extracted:termination_status = 1 }
// the time in seconds taken by the external solver to execute
ic { exists extracted:solver_time_seconds }
// the external solver version
ic { exists extracted:solver_version }

The returned outcome relation contains one of the following values:

• Model Error (0): some error occurred in the translation from Rel to solver, which usually means the model was incorrectly specified.
• Success (1): the Rel model was correctly translated into a solver model, the solver was invoked and it returned a termination status that indicated it reached a conclusion, which could be an OK or relaxed result. The termination_status field contains a more detailed value for the outcome.
• Solver Limit (2): translation was OK but the solver stopped without reaching a conclusion due to some limit imposed on it, such as a time or model size limit.
• Solver Error (3): translation was OK but the solver stopped due to some unexpected error.

If the outcome is not Model Error(0), then the termination_status contains additional information. Its value comes from MathOptInterface’s TerminationStatusCode (opens in a new tab).

Evaluate

rel:mathopt:evaluate uses a DSL implementation that evaluates the model against a proposed solution. This allows us to compute the objective function value for a specific solution, and check whether the constraints would be satisfied.

module ProposedSolution
    def x = MyData:my_domain, 2
end
def evaluated = rel:mathopt:evaluate[MyModel, MyData, ProposedSolution]
 
// the objective function value using ProposedSolution variable assignments
ic { evaluated:objective = 4 }
// the artificial constraint exists because it was satisfied by the assignments
ic { exists evaluated:subject_to:artificial }

Debug

Finally, rel:mathopt:debug can be used to compute debug information in the form of strings that contain S-expressions that represent the model’s objective function and constraints. This may be useful to understand issues with the specification.

// will output debug strings for objective function and constraints
def output = rel:mathopt:debug[MyModel, MyData, MyVariables]

Alternative Specification Pattern

The pattern suggested above, with Data, Variables and Model modules, is very flexible, allowing for multiple datasets to coexist in a database, reusing the same models. However, we pay a price in code complexity and, often, runtime performance.

The expression DSL can be reused with an alternative, simpler pattern. The simplified pattern has the advantage of minimizing the need for higher order modules, which makes specifications a bit simpler and faster. The drawback is that it is not possible to define group-bys based on different data sets.

The example above, using the simplified pattern, would look as follows:

// the dataset can be any Rel definition, not necessarily within a module
def my_domain = 5 ; 7
 
// variables are declared with rel:mathopt:variables and stored in a definition
def variables = rel:mathopt:variables[{
    :x, "integer", my_domain
}]
 
// we can then bring variables and the expression DSL definitions into scope
with variables use x
with mathopt:Solver:Solve use sum, foreach, +, -, *, ≼, ≽, /, =
 
// there is no need for higher order modules for objective function and constraints
def objective = sum[x[d] for d in my_domain]
 
// we use a first-order module to group the constraints
module constraints
    def artificial = foreach[my_domain, {d : d * x[d] ≼ 20 }]
end
 
 
// finally we can maximize and extract the results:
def result = rel:mathopt:maximize[objective, constraints, variables, {}]
 
// note that this is extract_result, not extract as before
def output = rel:mathopt:extract_result[result, variables]

Note that names are irrelevant for the library, you can use other names for objective, constraints, variables and result.

solve

rel:mathopt:solve[CONFIG, MODEL, DATA, VARS]

Call a mathematical optimization solver to search for a solution to the specified problem.

Inputs:

• CONFIG: the relation that contains configuration for the solver. mathopt:SolverConfigDefaults describes the accepted configuration parameters. The empty relation can be used when the defaults are appropriate, e.g. rel:mathopt:solve[{}, ...]
• MODEL: the relation that specifies the model to solve. It should contain the objective function, the constraints (under a subject_to sub-relation) and must be parameterizable with DATA, VARS and SOLVER higher order relations.
• DATA: the relation that contains data to populate the model. It must be first order.
• VARS: the relation that declares the model variables. It must be parametrizable with DATA.

Output:

• an opaque binary string containing the result of the solver. Use the rel:mathopt:extract function to extract detailed information from the result.

solve_instance

rel:mathopt:solve_instance[MODEL, DATA, VARS]

Compute the full instance of this model, when instantiated with this data and variables. This can be used to investigate and debug the model that is being instantiated by rel:mathopt:solve without attempting to call the solver.

Inputs are the same MODEL, DATA and VARS as rel:mathopt:solve.

Output:

• an instantiated model, which is the MODEL module instantiated with DATA and VARS, which should contain the concrete objective function and constraints that would be sent to the solver.

extract

rel:mathopt:extract[result, DATA, VARS]

Extract detailed information from the result of a call to a mathematical optimization solver. The result must be a materialized relation. This is equivalent to the union of rel:mathopt:extract_info and rel:mathopt:extract_solution; if you are only interested in the solution, using rel:mathopt:extract_solution is potentially more efficient.

Inputs:

• result: a relation that contains the result of a rel:mathopt:solve call. It must be materialized.
• DATA: the relation that contains data to populate the model. It must be grounded.
• VARS: the relation that declares the model variables. It must be parametrizable with DATA.

Output:

• a function from variable keys to the float value assigned by the solver, if any. The keys depend on the variable specification in VARS.

Example:

def result = rel:mathopt:solve[{}, Model, Data, Variables]
def extracted = rel:mathopt:extract[result, Data, Variables]

extract_info

rel:mathopt:extract_info[result]

Similar to rel:mathopt:extract, but only extract meta-information, such as termination status, solver time, etc.

extract_solution

rel:mathopt:extract_solution[result, DATA, VARS]

Similar to rel:mathopt:extract, but only extract the solution, which contains variable assignments.

evaluate

rel:mathopt:evaluate[MODEL, DATA, SOLUTION]

Evaluate a model against a specific solution.

This function uses the same MODEL and DATA modules that rel:mathopt:solve expects, but instead of searching for a solution, it evaluates the provided solution. This is useful to validate whether a solution satisfied the model constraints and to obtain the corresponding value for the objective function.

Example:

module CorrectSolution
def xMake = ("bands", 10) ; ("coils", 20)
end
def evaluated_model = rel:mathopt:evaluate[Model, Data, CorrectSolution]

debug

rel:mathopt:debug[MODEL, DATA, VARS]

Get debug information for a mathematical optimization model.

This function has the same interface as rel:mathopt:solve, but returns debug information in the form of strings that contain S-expressions that represent the model’s objective function and constraints.

Example:

def output = rel:mathopt:debug[Model, Data, Variables]

Module: mathopt:SolverConfigDefaults

Default values for the configuration parameters accepted by rel:mathopt:solve.

sense

Indicates whether the solver should “maximize” or “minimize” the objective function:

objective

A symbol with the name of the subrelation in MODEL that contains the objective function definition:

solver

Which solver backend to use. Currently “HiGHS” (MIP and QP formulations), “DAQP” (MIP and QP formulations), and “Pavito” (MIQP formulations) are accepted:

solver_attributes

Key value pairs of solver-specific attributes. These are sent directly to the solver backend. The key must be a string and the value must be either a number or a string. Example:

// provide a solution timeout limit and turn off presolve while using `HIGHS`.
def solver_attributes = {("time_limit", 60); ("presolve", "off")}

model_format

A string specifying the format of the optimization model to be returned as a single string. The available formats are “LP”, “MPS”, “MOF”, and “NL”.

Module: mathopt:Solver

Various implementations of a DSL for specifying mathematical optimization problems.

Module: Solve

Solve models using a mathematical optimization solver.

Module: Evaluate

Evaluate objective functions and constraints against variable assignments.

Module: Debug

Debug models by generating S-expressions for objective functions and constraints.

Module: rel:mathopt

variables

rel:mathopt:variables[R]

Declare variables without needing a module. The relation R must contain triples (Symbol, String, D) that represent the variable name, the variable type, and the domain, respectively.

Example:

    def vars = rel:mathopt:variables[{
        :x, "continuous", domain ;
        :y, "binary", ()
    }]

named_variables

rel:mathopt:named_variables[R]

Declare variables with name and key value pairs to be used in debugging, and also without needing a module. The relation R must contain triples (Symbol, String, D) that represent the variable name, the variable type, and the domain, respectively.

Example:

    def vars = rel:mathopt:named_variables[{
        :x, "continuous", domain ;
        :y, "binary", ()
    }]

maximize

rel:mathopt:maximize[OBJECTIVE, CONSTRAINTS, VARIABLES, CONFIG]

Execute a solver trying to maximize the OBJECTIVE function under these CONSTRAINTS, using these VARIABLES. Configure the solver using CONFIG.

Inputs:

• OBJECTIVE must be an expression created using the mathopt:Solver DSL.
• CONSTRAINTS must be a module that contains definitions for the constraints; those constraints also should be built with the mathopt:Solver DSL.
• VARIABLES must be a definition that contains the value returned from a call to rel:mathopt:variables, which defines variables.
• CONFIG is a relation to override values from mathopt:DefaultSolverConfig. Note that sense and objective are ignored by this function.

Output:

• an opaque binary string containing the result of the solver. Use the rel:mathopt:extract_result function to extract detailed information from the result.

minimize

rel:mathopt:minimize[OBJECTIVE, CONSTRAINTS, VARIABLES, CONFIG]

Execute a solver trying to minimize the OBJECTIVE function under these CONSTRAINTS, using these VARIABLES. Configure the solver using CONFIG.

See rel:mathopt:maximize for details.

export_model

rel:mathopt:export_model[OBJECTIVE, CONSTRAINTS, VARIABLES, CONFIG]

Export the model as a string representation of the file as specified in CONFIG, without optimizing the model. The default model returned is a LP format, the default sense is maximize, and the default solver is HiGHS. Any errors encountered while building the model are reported to the user instead of the model file string.

Inputs:

• OBJECTIVE: an expression created using the mathopt:Solver DSL.
• CONSTRAINTS: a module that contains definitions for the constraints, which should also be built with the mathopt:Solver DSL.
• VARIABLES: a definition that contains the value returned from a call to rel:mathopt:variables, or a call to rel:mathopt:named_variables, which defines variables.
• CONFIG: a relation to override values from mathopt:DefaultSolverConfig.

Output:

• An opaque binary string containing the result of the solver. Use the rel:mathopt:extract_result function to extract detailed information from the result.

Example:

def result = rel:mathopt:export_model[objective, constraints, variables, {}]
def extracted = rel:mathopt:extract_result[result, variables]

extract_result

rel:mathopt:extract_result[result, VARIABLES]

Extract detailed information from the result of a call to a mathematical optimization solver (maximize, minimize, or export_model). The result must be a materialized relation.

Inputs:

• result: a relation that contains the result of a rel:mathopt:maximize, rel:mathopt:minimize, orrel:mathopt:export_model call. It must be materialized.
• VARIABLES: the relation that declares the model variables, created with rel:mathopt:variables.

Output:

• a function from variable keys to the float value assigned by the solver, if any. The keys depend on the variable specification in VARS.

Example:

def result = rel:mathopt:maximize[objective, constraints, variables, {}]
def extracted = rel:mathopt:extract_result[result, variables]