relationalai.api.create_data_stream()
create_data_stream( object_reference OBJECT, model_name STRING, stream_name STRING, force_overwrite BOOLEAN DEFAULT FALSE)
Creates a new data stream from an object reference for a Snowflake table or view.
At least one engine with a READY
state must be available to create a data stream.
You must have SELECT
privileges and change tracking must be enabled on the source object.
See Supported Column Types for a list of column types supported in a data stream’s source table or view.
Requires the cdc_admin
application role.
Parameters
Section titled “Parameters”Name | Type | Description |
---|---|---|
object_reference | OBJECT | An object reference to a Snowflake table or view. |
model_name | STRING | The name of the model associated with the data stream. |
stream_name | STRING | The name of the data stream to create. Must be contain only characters A-Z , a-z , or 0-9 and may not start with a number. |
force_overwrite | BOOLEAN | If TRUE , overwrites data from a previously existing data stream with the same name. (Default: FALSE ) |
Returns
Section titled “Returns”STRING
Supported Column Types
Section titled “Supported Column Types”Data streams support the following Snowflake column types:
Tables or views with unsupported column types cannot be used as data stream sources.
Example
Section titled “Example”Data streams are created automatically by the Python API when a Snowflake source table or view is referenced by a RAI model. However, you may need to create a stream manually if the stream has been quaratined due to synchronization issues or has become stale.
Use the api.create_data_stream()
procedure to create a new data stream:
-- Replace the placeholders with your database, schema, and table/view names.SET obj_name = '<db>.<schema>.<table_or_view>';SET obj_type = 'TABLE'; -- Set to 'VIEW' if needed.SET obj_ref = relationalai.api.object_reference($obj_type, $obj_name);
-- Enable change tracking on the table or view-- (if you have a custom procedure handling this, since Snowflake lacks direct support).ALTER TABLE IDENTIFIER($obj_name) SET CHANGE_TRACKING = TRUE;
-- Stream the table or view into the model 'MyModel' with the name 'my_stream'.CALL relationalai.api.create_data_stream($obj_ref, 'MyModel', 'my_stream', TRUE);/*+----------------------------------+ | Datastream created successfully. | +----------------------------------+ */
Note that when you delete a data stream, models retain access to a snapshot of the most recent data processed by the stream.
If you are creating a data stream with the same name as a previously deleted stream, you must set the force_overwrite
parameter to TRUE
to overwrite the data in the snapshot, as shown in the above example.
See Data Management for more information data streams.