Experimental APIs

These features all require special permissions to be activated on your DataRobot account, and will not work otherwise. If you want to test a feature, please ask your DataRobot CFDS or account manager about enrolling in our preview program.

Classes in this list should be considered “experimental”, not fully released, and likely to change in future releases. Do not use them for production systems or other mission-critical uses.

class datarobot._experimental.models.model.Model

Experimental model class

continue_incremental_learning_from_incremental_model(chunk_definition_id, early_stopping_rounds=None)

Submit a job to the queue to perform the first incremental learning iteration training on an existing sample model. This functionality requires the SAMPLE_DATA_TO_START_PROJECT feature flag to be enabled.

Parameters:

  • chunk_definition_id (str) – The Mongo ID for the chunking service.

  • early_stopping_rounds (Optional[int]) – The number of chunks in which no improvement is observed that triggers the early stopping mechanism.

Returns:

job – The created job that is retraining the model

Return type:

ModelJob

datarobot._experimental.models.data_store.get_spark_session(self, db_token)

Returns a Spark session

Parameters:

db_token (str) – A personal access token.

Returns:

A spark session initialized with connection parameters taken from DataStore and provided db_token.

Return type:

SparkSession

Examples

>>> from datarobot._experimental.models.data_store import DataStore
>>> data_stores = DataStore.list(typ=DataStoreListTypes.DR_DATABASE_V1)
>>> data_stores
[DataStore('my_databricks_store_1')]
>>> db_connection = data_stores[0].get_spark_session('<token>')
>>> db_connection
<pyspark.sql.connect.session.SparkSession at 0x7f386068fbb0>
>>> df = session.read.table("samples.nyctaxi.trips")
>>> df.show()
class datarobot._experimental.models.data_store.DataStore

A data store. Represents database

Variables:

  • id (str) – The ID of the data store.

  • data_store_type (str) – The type of data store.

  • canonical_name (str) – The user-friendly name of the data store.

  • creator (str) – The ID of the user who created the data store.

  • updated (datetime.datetime) – The time of the last update.

  • params (DataStoreParameters) – A list specifying data store parameters.

  • role (str) – Your access role for this data store.

  • driver_class_type (str) – Your access role for this data store.

class datarobot._experimental.models.retraining.RetrainingUseCase

Retraining use case.

Variables:

  • id (str) – The ID of the use case.

  • name (str) – The name of the use case.

class datarobot._experimental.models.retraining.RetrainingPolicy

Retraining Policy.

Variables:

  • policy_id (str) – ID of the retraining policy

  • name (str) – Name of the retraining policy

  • description (str) – Description of the retraining policy

  • use_case (Optional[dict]) – Use case the retraining policy is associated with

classmethod list(deployment_id)

Lists all retraining policies associated with a deployment

Parameters:

deployment_id (str) – Id of the deployment

Returns:

policies – List of retraining policies associated with a deployment

Return type:

list

Examples

from datarobot import Deployment
from datarobot._experimental.models.retraining import RetrainingPolicy
deployment = Deployment.get(deployment_id='620ed0e37b6ce03244f19631')
RetrainingPolicy.list(deployment.id)
>>> [RetrainingPolicy('620ed248bb0a1f5889eb6aa7'), RetrainingPolicy('624f68be8828ed81bf487d8d')]
classmethod get(deployment_id, retraining_policy_id)

Retrieves a retraining policy associated with a deployment

Parameters:

  • deployment_id (str) – Id of the deployment

  • retraining_policy_id (str) – Id of the policy

Returns:

retraining_policy – Retraining policy

Return type:

Retraining Policy

Examples

from datarobot._experimental.models.retraining import RetrainingPolicy
policy = RetrainingPolicy.get(
    deployment_id='620ed0e37b6ce03244f19631',
    retraining_policy_id='624f68be8828ed81bf487d8d'
)
policy.id
>>>'624f68be8828ed81bf487d8d'
policy.name
>>>'PolicyA'
classmethod create(deployment_id, name, description=None, use_case_id=None)

Create a retraining policy associated with a deployment

Parameters:

  • deployment_id (str) – The ID of the deployment.

  • name (str) – The retraining policy name.

  • description (str) – The retraining policy description.

  • use_case_id (Optional[str]) – The ID of the Use Case that the retraining policy is associated with.

Returns:

retraining_policy – Retraining policy

Return type:

Retraining Policy

Examples

from datarobot._experimental.models.retraining import RetrainingPolicy
policy = RetrainingPolicy.create(
    deployment_id='620ed0e37b6ce03244f19631',
    name='Retraining Policy A',
    use_case_id='678114c41e9114cabca27044',
)
policy.id
>>>'624f68be8828ed81bf487d8d'
classmethod delete(deployment_id, retraining_policy_id)

Deletes a retraining policy associated with a deployment

Parameters:

  • deployment_id (str) – Id of the deployment

  • retraining_policy_id (str) – Id of the policy

Return type:

None

Examples

from datarobot._experimental.models.retraining import RetrainingPolicy
RetrainingPolicy.delete(
    deployment_id='620ed0e37b6ce03244f19631',
    retraining_policy_id='624f68be8828ed81bf487d8d'
)
update_use_case(use_case_id)

Update the use case associated with this retraining policy

Parameters:

use_case_id (str) – Id of the use case the retraining policy is associated with

Return type:

RetrainingPolicy

Examples

from datarobot._experimental.models.retraining import RetrainingPolicy
policy = RetrainingPolicy.get(
    deployment_id='620ed0e37b6ce03244f19631',
    retraining_policy_id='624f68be8828ed81bf487d8d'
)
updated = RetrainingPolicy.update_use_case(use_case_id='620ed0e37b6ce03244f19633')
updated.use_case.id
>>>'620ed0e37b6ce03244f19633'
class datarobot._experimental.models.retraining.RetrainingPolicyRun

Retraining policy run.

Variables:

  • policy_run_id (str) – ID of the retraining policy run

  • status (str) – Status of the retraining policy run

  • challenger_id (str) – ID of the challenger model retrieved after running the policy

  • error_message (str) – The error message if an error occurs during the policy run

  • model_package_id (str) – ID of the model package (version) retrieved after the policy is run

  • project_id (str) – ID of the project the deployment is associated with

  • start_time (datetime.datetime) – Timestamp of when the policy run starts

  • finish_time (datetime.datetime) – Timestamp of when the policy run finishes

classmethod list(deployment_id, retraining_policy_id)

Lists all the retraining policy runs of a retraining policy that is associated with a deployment.

Parameters:

  • deployment_id (str) – ID of the deployment

  • retraining_policy_id (str) – ID of the policy

Returns:

policy runs – List of retraining policy runs

Return type:

list

Examples

from datarobot._experimental.models.retraining import RetrainingPolicyRun
RetrainingPolicyRun.list(
    deployment_id='620ed0e37b6ce03244f19631',
    retraining_policy_id='62f4448f0dfd5699feae3e6e'
)
>>> [RetrainingPolicyRun('620ed248bb0a1f5889eb6aa7'), RetrainingPolicyRun('624f68be8828ed81bf487d8d')]
class datarobot._experimental.models.data_matching.DataMatching

Retrieves the closest data points for the input data.

This functionality is more than the simple lookup. In order to retrieve the closest data points data matching functionality will leverage DataRobot preprocessing pipeline first and then search for the closest data points. The returned values will be the closest data points at the point of entry to the model.

There are three sets of methods supported:

  1. Methods to build the index (for project, model, featurelist). The index needs to be built first in order to search for the closest data points. Once the index is built it will be reused.

  2. Methods to search for the closest data points (for project, model, featurelist). These methods will initialize the query, await its completion and then save the result as csv file with in the specified location.

  3. Additional methods to manually list history of queries and retrieve results for them.

get_query_url(url, number_of_data=None)

Returns formatted data matching query url

Return type:

str

get_closest_data(query_file_path, number_of_data=None, max_wait=600, build_index_if_missing=True)

Retrieves closest data points to the data point in input file. If the index is missing by default the method will try to build it.

Parameters:

  • query_file_path (str) – Path to file with the data point to search closest data points

  • number_of_data (int or None) – Number of results to search for. If no value specified, the default is 10.

  • max_wait (int) – Number of seconds to wait for the result. Default is 600.

  • build_index_if_missing (Optional[bool]) – Should the index be created if it is missing. If False is specified and the index is missing, an exception is thrown. Default True.

Returns:

df – Dataframe with query result

Return type:

pd.DataFrame

get_closest_data_for_model(model_id, query_file_path, number_of_data=None, max_wait=600, build_index_if_missing=True)

Retrieves closest data points to the data point in input file. If the index is missing by default the method will try to build it.

Parameters:

  • model_id (str) – Id of the model to search for the closest data points

  • query_file_path (str) – Path to file with the data point to search closest data points

  • number_of_data (int or None) – Number of results to search for. If no value specified, the default is 10.

  • max_wait (int) – Number of seconds to wait for the result. Default is 600.

  • build_index_if_missing (Optional[bool]) – Should the index be created if it is missing. If False is specified and the index is missing, an exception is thrown. Default True.

Returns:

df – Dataframe with query result

Return type:

pd.DataFrame

get_closest_data_for_featurelist(featurelist_id, query_file_path, number_of_data=None, max_wait=600, build_index_if_missing=True)

Retrieves closest data points to the data point in input file. If the index is missing by default the method will try to build it.

Parameters:

  • featurelist_id (str) – Id of the featurelist to search for the closest data points

  • query_file_path (str) – Path to file with the data point to search closest data points

  • number_of_data (int or None) – Number of results to search for. If no value specified, the default is 10.

  • max_wait (int) – Number of seconds to wait for the result. Default is 600.

  • build_index_if_missing (bool) – Should the index be created if it is missing. If False is specified and the index is missing, the exception is thrown. Default True.

Returns:

df – Dataframe with query result

Return type:

pd.DataFrame

build_index(max_wait=600)

Builds data matching index and waits for its completion.

Parameters:

max_wait (int or None) – Seconds to wait for the completion of build index operation. Default is 600. When the 0 or None value is passed then the method will exit without awaiting for the build index operation to complete.

Return type:

None

build_index_for_featurelist(featurelist_id, max_wait=600)

Builds data matching index for featurelist and waits for its completion.

Parameters:

  • featurelist_id (str) – Id of the featurelist to build the index for

  • max_wait (int or None) – Seconds to wait for the completion of build index operation. Default is 600. When the 0 or None value is passed then the method will exit without awaiting for the build index operation to complete.

Return type:

None

build_index_for_model(model_id, max_wait=600)

Builds data matching index for feature list and waits for its completion.

Parameters:

  • model_id (str) – Id of the model to build index for

  • max_wait (int or None) – Seconds to wait for the completion of build index operation. Default is 600. When the 0 or None value is passed then the method will exit without awaiting for the build index operation to complete.

Return type:

None

list()

Lists all data matching queries for the project. Results are sorted in descending order starting from the latest to the oldest.

Return type:

List[DataMatchingQuery]

class datarobot._experimental.models.data_matching.DataMatchingQuery

Data Matching Query object.

Represents single query for the closest data points. Once related query job is completed, its result can be retrieved and saved as csv file in specified location.

classmethod list(project_id)

Retrieves the list of queries.

Parameters:

project_id (str) – Project ID to retrieve data matching queries for

Return type:

List[DataMatchingQuery]

save_result(file_path)

Downloads the query result and saves it in file_path location.

Parameters:

file_path (str) – Path location where to save the query result

Return type:

None

get_result()

Returns the query result as dataframe.

Parameters:

df (pd.DataFrame) – Dataframe with query result

Return type:

DataFrame

class datarobot._experimental.models.model_lineage.FeatureCountByType

Contains information about a feature type and how many features in the dataset are of this type.

Variables:

  • feature_type (str) – The feature type grouped in this count.

  • count (int) – The number of features of this type.

class datarobot._experimental.models.model_lineage.User

Contains information about a user.

Variables:

  • Id (str) – Id of the user.

  • full_name (Optional[str]) – Full name of the user.

  • email (Optional[str]) – Email address of the user.

  • user_hash (Optional[str]) – User’s gravatar hash.

  • user_name (Optional[str]) – Username of the user.

class datarobot._experimental.models.model_lineage.ReferencedInUseCase

Contains information about the reference of a dataset in an Use Case.

Variables:

  • added_to_use_case_by (User) – User who added the dataset to the Use Case.

  • added_to_use_case_at (datetime.datetime) – Time when the dataset was added to the Use Case.

class datarobot._experimental.models.model_lineage.DatasetInfo

Contains information about the dataset.

Variables:

  • dataset_name (str) – Dataset name.

  • dataset_version_id (str) – Dataset version Id.

  • dataset_id (str) – Dataset Id.

  • number_of_rows (int) – Number of rows in the dataset.

  • file_size (int) – Size of the dataset as a CSV file, in bytes.

  • number_of_features (int) – Number of features in the dataset.

  • number_of_feature_by_type (List[FeatureCountByType]) – Number of features in the dataset, grouped by feature type.

  • referenced_in_use_case (Optional[ReferencedInUseCase]) – Information about the reference of this dataset in the Use Case. This information will only be present if the use_case_id was passed to ModelLineage.get.

class datarobot._experimental.models.model_lineage.FeatureWithMissingValues

Contains information about the number of missing values for one feature.

Variables:

  • feature_name (str) – Name of the feature.

  • number_of_missing_values (int) – Number of missing values for this feature.

class datarobot._experimental.models.model_lineage.FeaturelistInfo

Contains information about the featurelist.

Variables:

  • featurelist_name (str) – Featurelist name.

  • featurelist_id (str) – Featurelist Id.

  • number_of_features (int) – Number of features in the featurelist.

  • number_of_feature_by_type (List[FeatureCountByType]) – Number of features in the featurelist, grouped by feature type.

  • number_of_features_with_missing_values (int) – Number of features in the featurelist with at least one missing value.

  • number_of_missing_values (int) – Number of missing values across all features of the featurelist.

  • features_with_most_missing_values (List[FeatureWithMissingValues]) – List of features with the most missing values.

  • description (str) – Description of the featurelist.

class datarobot._experimental.models.model_lineage.TargetInfo

Contains information about the target.

Variables:

  • name (str) – Name of the target feature.

  • target_type (str) – Project type resulting from selected target.

  • positive_class_label (Optional[Union[str, int, float]]) – Positive class label. For every project type except Binary Classification, this value will be null.

  • mean (Optional[float]) – Mean of the target. This field will only be available for Binary Classification, Regression, and Min Inflated projects.

class datarobot._experimental.models.model_lineage.PartitionInfo

Contains information about project partitioning.

Variables:

  • validation_type (str) – Either CV for cross-validation or TVH for train-validation-holdout split.

  • cv_method (str) – Partitioning method used.

  • holdout_pct (float) – Percentage of the dataset reserved for the holdout set.

  • datetime_col (Optional[str]) – If a date partition column was used, the name of the column. Note that datetime_col applies to an old partitioning method no longer supported for new projects, as of API version v2.0.

  • datetime_partition_column (Optional[str]) – If a datetime partition column was used, the name of the column.

  • validation_pct (Optional[float]) – If train-validation-holdout split was used, the percentage of the dataset used for the validation set.

  • reps (Optional[float]) – If cross validation was used, the number of folds to use.

  • cv_holdout_level (Optional[Union[str, float, int]]) – If a user partition column was used with cross validation, the value assigned to the holdout set.

  • holdout_level (Optional[Union[str, float, int]]) – If a user partition column was used with train-validation-holdout split, the value assigned to the holdout set.

  • user_partition_col (Optional[str]) – If a user partition column was used, the name of the column.

  • training_level (Optional[Union[str, float, int]]) – If a user partition column was used with train-validation-holdout split, the value assigned to the training set.

  • partition_key_cols (Optional[List[str]]) – A list containing a single string - the name of the group partition column.

  • validation_level (Optional[Union[str, float, int]]) – If a user partition column was used with train-validation-holdout split, the value assigned to the validation set.

  • use_time_series (Optional[bool]) – A boolean value indicating whether a time series project was created by using datetime partitioning. Otherwise, datetime partitioning created an OTV project.

class datarobot._experimental.models.model_lineage.ProjectInfo

Contains information about the project.

Variables:

  • project_name (str) – Name of the project.

  • project_id (str) – Project Id.

  • partition (PartitionInfo) – Partitioning settings of the project.

  • metric (str) – Project metric used to select the best-performing models.

  • created_by (User) – User who created the project.

  • created_at (Optional[datetime.datetime]) – Time when the project was created.

  • target (Optional[TargetInfo]) – Information about the target.

class datarobot._experimental.models.model_lineage.ModelInfo

Contains information about the model.

Variables:

  • blueprint_tasks (List[str]) – Tasks that make up the blueprint.

  • blueprint_id (str) – Blueprint Id.

  • model_type (str) – Model type.

  • sample_size (Optional[int]) – Number of rows this model was trained on.

  • sample_percentage (Optional[float]) – Percentage of the dataset the model was trained on.

  • milliseconds_to_predict_1000_rows (Optional[float]) – Estimate of how many millisecond it takes to predict 1000 rows. The estimate is based on the time it took to predict the holdout set.’

  • serialized_blueprint_file_size (Optional[int]) – Size of the serialized blueprint, in bytes.

class datarobot._experimental.models.model_lineage.ModelLineage

Contains information about the lineage of a model.

Variables:

  • dataset (DatasetInfo) – Information about the dataset this model was created with.

  • featurelist (FeaturelistInfo) – Information about the featurelist used to train this model.

  • project (ProjectInfo) – Information about the project this model was created in.

  • model (ModelInfo) – Information about the model itself.

classmethod get(model_id, use_case_id=None)

Retrieve lineage information about a trained model. If you pass the optional use_case_id parameter, this class will contain additional information.

Parameters:

  • model_id (str) – Model Id.

  • use_case_id (Optional[str]) – Use Case Id.

Return type:

ModelLineage

class datarobot._experimental.models.notebooks.notebook.ManualRunPayload
class datarobot._experimental.models.notebooks.notebook.Notebook

Metadata for a DataRobot Notebook accessible to the user.

Variables:

  • id (str) – The ID of the Notebook.

  • name (str) – The name of the Notebook.

  • type (NotebookType) – The type of the Notebook. Can be “plain” or “codespace”.

  • permissions (List[NotebookPermissions]) – The permissions the user has for the Notebook.

  • tags (List[str]) – Any tags that have been added to the Notebook. Default is an empty list.

  • created (NotebookActivity) – Information on when the Notebook was created and who created it.

  • updated (NotebookActivity) – Information on when the Notebook was updated and who updated it.

  • last_viewed (NotebookActivity) – Information on when the Notebook was last viewed and who viewed it.

  • settings (NotebookSettings) – Information on global settings applied to the Notebook.

  • org_id (Optional[str]) – The organization ID associated with the Notebook.

  • tenant_id (Optional[str]) – The tenant ID associated with the Notebook.

  • description (Optional[str]) – The description of the Notebook. Optional.

  • session (Optional[NotebookSession]) – Metadata on the current status of the Notebook and its kernel. Optional.

  • use_case_id (Optional[str]) – The ID of the Use Case the Notebook is associated with. Optional.

  • use_case_name (Optional[str]) – The name of the Use Case the Notebook is associated with. Optional.

  • has_schedule (bool) – Whether or not the notebook has a schedule.

  • has_enabled_schedule (bool) – Whether or not the notebook has a currently enabled schedule.

get_uri()
Returns:

url – Permanent static hyperlink to this Notebook in its Use Case or standalone.

Return type:

str

classmethod get(notebook_id)

Retrieve a single notebook.

Parameters:

notebook_id (str) – The ID of the notebook you want to retrieve.

Returns:

notebook – The requested notebook.

Return type:

Notebook

Examples

from datarobot._experimental.models.notebooks.notebook import Notebook

notebook = Notebook.get(notebook_id='6556b00dcc4ea0bb7ea48121')
create_revision(name=None, notebook_path=None, is_auto=False)

Create a new revision for the notebook.

Parameters:

  • name (Optional[str]) – The name of the revision. Optional.

  • notebook_path (Optional[str]) – The path of the notebook to execute within the codespace. Required if notebook is in a codespace.

  • is_auto (bool) – Indicates whether the revision was auto-saved versus a user interaction. Default is False.

Returns:

notebook_revision – Information about the created notebook revision.

Return type:

NotebookRevision

download_revision(revision_id, file_path=None, filelike=None)

Downloads the notebook as a JSON (.ipynb) file for the specified revision.

Parameters:

  • file_path (string, optional) – The destination to write the file to.

  • filelike (file, optional) – A file-like object to write to. The object must be able to write bytes. The user is responsible for closing the object.

Return type:

None

Examples

from datarobot._experimental.models.notebooks.notebook import Notebook

notebook = Notebook.get(notebook_id='6556b00dcc4ea0bb7ea48121')
manual_run = notebook.run_as_job()
revision_id = manual_run.wait_for_completion()
notebook.download_revision(revision_id=revision_id, file_path="./results.ipynb")
delete()

Delete a single notebook :rtype: None

Examples

from datarobot._experimental.models.notebooks.notebook import Notebook

notebook = Notebook.get(notebook_id='6556b00dcc4ea0bb7ea48121')
notebook.delete()
classmethod list(created_before=None, created_after=None, order_by=None, tags=None, owners=None, query=None, use_cases=None)

List all Notebooks available to the user.

Parameters:

  • created_before (Optional[str]) – List Notebooks created before a certain date. Optional.

  • created_after (Optional[str]) – List Notebooks created after a certain date. Optional.

  • order_by (Optional[str]) – Property to sort returned Notebooks. Optional. Supported properties are “name”, “created”, “updated”, “tags”, and “lastViewed”. Prefix the attribute name with a dash to sort in descending order, e.g. order_by=’-created’. By default, the order_by parameter is None.

  • tags (Optional[List[str]]) – A list of tags that returned Notebooks should be associated with. Optional.

  • owners (Optional[List[str]]) – A list of user IDs used to filter returned Notebooks. The respective users share ownership of the Notebooks. Optional.

  • query (Optional[str]) – A specific regex query to use when filtering Notebooks. Optional.

  • use_cases (Optional[UseCase or List[UseCase] or str or List[str]]) – Filters returned Notebooks by a specific Use Case or Cases. Accepts either the entity or the ID. Optional. If set to [None], the method filters the notebook’s datasets by those not linked to a UseCase.

Returns:

notebooks – A list of Notebooks available to the user.

Return type:

List[Notebook]

Examples

from datarobot._experimental.models.notebooks.notebook import Notebook

notebooks = Notebook.list()
is_running()

Check if the notebook session is currently running.

Return type:

bool

get_session_status()

Get the status of the notebook session.

Return type:

NotebookStatus

start_session(is_triggered_run=False, parameters=None, open_file_paths=None, clone_repository=None)

Start a new session for the notebook.

Parameters:

  • is_triggered_run (bool) – Whether the session being started is considered an “interactive” or “triggered” session. Default is False.

  • parameters (Optional[List[StartSessionParameters]]) – A list of dictionaries in the format {“name”: “FOO”, “value”: “my_value”} representing environment variables propagated to the notebook session.

  • open_file_paths (Optional[List[str]]) – A list of file paths to open upon instantiation of the notebook session.

  • clone_repository (Optional[CloneRepositorySchema]) – Information used to clone a remote repository as part of the environment setup flow.

Returns:

notebook_session – The created notebook session.

Return type:

NotebookSession

Examples

from datarobot._experimental.models.notebooks.notebook import Notebook

notebook = Notebook.get(notebook_id='6556b00dcc4ea0bb7ea48121')
session = notebook.start_session()
stop_session()

Stop the current session for the notebook.

Returns:

notebook_session – The stopped notebook session.

Return type:

NotebookSession

Examples

from datarobot._experimental.models.notebooks.notebook import Notebook

notebook = Notebook.get(notebook_id='6556b00dcc4ea0bb7ea48121')
session = notebook.stop_session()
execute(notebook_path=None, cell_ids=None)

Execute the notebook. Assumes session is already started.

Parameters:

  • notebook_path (Optional[str]) – The path of the notebook to execute within the Codespace. Required if the notebook is in a Codespace.

  • cell_ids (Optional[List[str]]) – The list of cell IDs to execute for a notebook. Not supported if the notebook is in a Codespace. Optional. If not provided, the whole notebook will be executed.

Return type:

None

get_execution_status()

Get the execution status information of the notebook.

Returns:

execution_status – The notebook execution status information.

Return type:

NotebookExecutionStatus

is_finished_executing()

Check if the notebook is finished executing.

Return type:

bool

run_as_job(title=None, notebook_path=None, parameters=None, manual_run_type=ManualRunType.MANUAL)

Create a manual scheduled job that runs the notebook.

Notes

The notebook must be part of a Use Case. If the notebook is in a Codespace then notebook_path is required.

Parameters:

  • title (Optional[str]) – The title of the background job. Optional.

  • notebook_path (Optional[str]) – The path of the notebook to execute within the Codespace. Required if notebook is in a Codespace.

  • parameters (Optional[List[StartSessionParameters]]) – A list of dictionaries in the format {“name”: “FOO”, “value”: “my_value”} representing environment variables predefined in the notebook session. Optional.

  • manual_run_type (Optional[ManualRunType]) – The type of manual run being triggered. Defaults to “manual” as opposed to “pipeline”.

Returns:

notebook_scheduled_job – The created notebook schedule job.

Return type:

NotebookScheduledJob

Raises:

InvalidUsageError – If attempting to create a manual scheduled run for a Codespace without a notebook path.

Examples

from datarobot._experimental.models.notebooks.notebook import Notebook

notebook = Notebook.get(notebook_id='6556b00dcc4ea0bb7ea48121')
manual_run = notebook.run_as_job()

# Alternatively, with title and parameters:
# manual_run = notebook.run_as_job(title="My Run", parameters=[{"FOO": "bar"}])

revision_id = manual_run.wait_for_completion()
class datarobot._experimental.models.notebooks.execution_environment.ExecutionEnvironmentAssignPayload
class datarobot._experimental.models.notebooks.execution_environment.Image

Execution environment image information.

Variables:

  • id (str) – The ID of the image.

  • name (str) – The name of the image.

  • default (bool) – Whether the image is the default image.

  • description (str) – The description of the image.

  • environment_id (str) – The ID of the environment.

  • gpu_optimized (bool) – Whether the image is GPU optimized.

  • language (ImageLanguage) – The runtime language of the image. For example “Python” or “R”

  • language_version (str) – The version of the language. For example “3.11” or “4.3”

  • libraries (list[str]) – A list of pre-installed libraries on the image.

class datarobot._experimental.models.notebooks.execution_environment.Machine

Execution environment machine information.

Variables:

  • id (str) – The ID of the machine.

  • name (str) – The name of the machine. Values include “XS”, “S”, “M”, “L” etc.

  • default (bool) – Whether the machine is the default machine.

  • cpu (str) – The CPU of the machine. For example a value like “2000m”.

  • cpu_cores (int) – The number of CPU cores.

  • ephemeral_storage (str) – The ephemeral storage of the machine. For example a value like “15Gi”.

  • has_gpu (bool) – Whether the machine has a GPU.

  • memory (str) – The memory of the machine. For example a value like “8Gi”.

  • ram_gb (int) – The amount of RAM of the machine.

class datarobot._experimental.models.notebooks.execution_environment.ExecutionEnvironment

An execution environment associated with a notebook.

Variables:

  • image (Image) – The image associated with the execution environment.

  • machine (Machine) – The machine associated with the execution environment.

  • time_to_live (int) – The inactivity timeout for notebook session.

classmethod get(notebook_id)

Get a notebook execution environment by its notebook ID.

Parameters:

notebook_id (str) – The ID of the notebook.

Returns:

The notebook execution environment.

Return type:

ExecutionEnvironment

classmethod assign_environment(notebook_id, payload)

Assign execution environment values to a notebook.

Parameters:
Returns:

The assigned execution environment.

Return type:

ExecutionEnvironment

Examples

from datarobot._experimental.models.notebooks import ExecutionEnvironment, ExecutionEnvironmentAssignPayload

payload = ExecutionEnvironmentAssignPayload(machine_slug='medium', time_to_live=10)
exec_env = ExecutionEnvironment.assign_environment('67914bfab0279fd832dc3fd1', payload)
class datarobot._experimental.models.notebooks.kernel.NotebookKernel

A kernel associated with a codespace notebook.

Variables:

  • id (str) – The kernel ID.

  • name (str) – The kernel name.

  • language (RuntimeLanguage) – The kernel language. Supports Python and R.

  • running (bool) – Whether the kernel is running.

  • execution_state (KernelState) – The kernel execution state.

class datarobot._experimental.models.notebooks.revision.CreateRevisionPayload
class datarobot._experimental.models.notebooks.revision.NotebookRevision

Represents a notebook revision.

Variables:

  • revision_id (str) – The ID of the notebook revision.

  • notebook_id (str) – The ID of the notebook.

  • is_auto (bool) – Whether the revision was auto-saved.

classmethod create(notebook_id, payload)

Create a new notebook revision.

Parameters:

  • notebook_id (str) – The ID of the notebook.

  • payload (CreateRevisionPayload) – The payload to create the revision.

Returns:

Information about the created notebook revision.

Return type:

NotebookRevision

class datarobot._experimental.models.notebooks.scheduled_job.NotebookScheduledJob

DataRobot Notebook Schedule. A scheduled job that runs a notebook.

Variables:

  • id (str) – The ID of the scheduled notebook job.

  • enabled (bool) – Whether job is enabled or not.

  • run_type (RunType) – The type of the run - either manual (triggered via UI or API) or scheduled.

  • notebook_type (NotebookType) – The type of the notebook - either plain or codespace.

  • job_payload (ScheduledJobPayload) – The payload used for the background job.

  • next_run_time (Optional[str]) – The next time the job is scheduled to run (assuming it is enabled).

  • title (Optional[str]) – The title of the job. Optional.

  • schedule (Optional[str]) – Cron-like string to define how frequently job should be run. Optional.

  • schedule_localized (Optional[str]) – A human-readable localized version of the schedule. Example in English is ‘At 42 minutes past the hour’. Optional.

  • last_successful_run (Optional[str]) – The last time the job was run successfully. Optional.

  • last_failed_run (Optional[str]) – The last time the job failed. Optional.

  • last_run_time (Optional[str]) – The last time the job was run (failed or successful). Optional.

classmethod get(use_case_id, scheduled_job_id)

Retrieve a single notebook schedule.

Parameters:

scheduled_job_id (str) – The ID of the notebook schedule you want to retrieve.

Returns:

notebook_schedule – The requested notebook schedule.

Return type:

NotebookScheduledJob

Examples

from datarobot._experimental.models.notebooks.scheduled_job import NotebookScheduledJob

notebook_schedule = NotebookScheduledJob.get(
    use_case_id="654ad653c6c1e889e8eab12e",
    scheduled_job_id="65734fe637157200e28bf688",
)
get_most_recent_run()

Retrieve the most recent run for the notebook schedule.

Returns:

notebook_scheduled_run – The most recent run for the notebook schedule, or None if no runs have been made.

Return type:

Optional[NotebookScheduledRun]

Examples

from datarobot._experimental.models.notebooks.scheduled_job import NotebookScheduledJob

notebook_schedule = NotebookScheduledJob.get(
    use_case_id="654ad653c6c1e889e8eab12e",
    scheduled_job_id="65734fe637157200e28bf688",
)
most_recent_run = notebook_schedule.get_most_recent_run()
get_job_history()

Retrieve list of historical runs for the notebook schedule. Gets the most recent runs first.

Returns:

notebook_scheduled_runs – The list of historical runs for the notebook schedule.

Return type:

List[NotebookScheduledRun]

Examples

from datarobot._experimental.models.notebooks.scheduled_job import NotebookScheduledJob

notebook_schedule = NotebookScheduledJob.get(
    use_case_id="654ad653c6c1e889e8eab12e",
    scheduled_job_id="65734fe637157200e28bf688",
)
notebook_scheduled_runs = notebook_schedule.get_job_history()
wait_for_completion(max_wait=600)

Wait for the completion of a scheduled notebook and return the revision ID corresponding to the run’s output.

Parameters:

max_wait (int) – The number of seconds to wait before giving up.

Returns:

revision_id – Returns either revision ID or message describing current state.

Return type:

str

Examples

from datarobot._experimental.models.notebooks.notebook import Notebook

notebook = Notebook.get(notebook_id='6556b00dcc4ea0bb7ea48121')
manual_run = notebook.run_as_job()
revision_id = manual_run.wait_for_completion()
class datarobot._experimental.models.notebooks.scheduled_run.ScheduledJobParam

DataRobot Schedule Job Parameter.

Variables:

  • name (str) – The name of the parameter.

  • value (str) – The value of the parameter.

class datarobot._experimental.models.notebooks.scheduled_run.ScheduledJobPayload

DataRobot Schedule Job Payload.

Variables:

  • uid (str) – The ID of the user who created the notebook schedule.

  • org_id (str) – The ID of the user’s organization who created the notebook schedule.

  • use_case_id (str) – The ID of the Use Case that the notebook belongs to.

  • notebook_id (str) – The ID of the notebook being run on a schedule.

  • notebook_name (str) – The name of the notebook being run on a schedule.

  • run_type (RunType) – The type of the run - either manual (triggered via UI or API) or scheduled.

  • notebook_type (NotebookType) – The type of the notebook - either plain or codespace.

  • parameters (List[ScheduledJobParam]) – The parameters being used in the notebook schedule. Can be an empty list.

  • notebook_path (Optional[str]) – The path of the notebook to execute within the codespace. Optional. Required if notebook is in a codespace.

class datarobot._experimental.models.notebooks.scheduled_run.ScheduledRunRevisionMetadata

DataRobot Notebook Revision Metadata specifically for a scheduled run.

Both id and name can be null if for example the job is still running or has failed.

Variables:

  • id (Optional[str]) – The ID of the Notebook Revision. Optional.

  • name (Optional[str]) – The name of the Notebook Revision. Optional.

class datarobot._experimental.models.notebooks.scheduled_run.NotebookScheduledRun

DataRobot Notebook Scheduled Run. A historical run of a notebook schedule.

Variables:

  • id (str) – The ID of the scheduled notebook job.

  • use_case_id (str) – The Use Case ID of the scheduled notebook job.

  • status (str) – The status of the run.

  • payload (ScheduledJobPayload) – The payload used for the background job.

  • title (Optional[str]) – The title of the job. Optional.

  • start_time (Optional[str]) – The start time of the job. Optional.

  • end_time (Optional[str]) – The end time of the job. Optional.

  • revision (ScheduledRunRevisionMetadata) – Notebook revision data - ID and name.

  • duration (Optional[int]) – The job duration in seconds. May be None for example while the job is running. Optional.

  • run_type (Optional[RunType]) – The type of the run - either manual (triggered via UI or API) or scheduled. Optional.

  • notebook_type (Optional[NotebookType]) – The type of the notebook - either plain or codespace. Optional.

class datarobot._experimental.models.notebooks.session.CloneRepositorySchema
class datarobot._experimental.models.notebooks.session.StartSessionParameters
class datarobot._experimental.models.notebooks.session.StartSessionPayload
class datarobot._experimental.models.notebooks.session.NotebookExecutionStatus

Notebook execution status information.

Variables:

  • status (str) – The status of the notebook execution.

  • cell_id (Optional[bson.ObjectId]) – The ID of the cell being executed. Optional.

  • queued_cell_ids (Optional[List[bson.ObjectId]]) – The list of cell IDs that are queued for execution. Optional.

class datarobot._experimental.models.notebooks.session.CodespaceNotebookCell
class datarobot._experimental.models.notebooks.session.CodespaceNotebookState

Notebook state information for a codespace notebook.

Variables:

  • name (str) – The name of the notebook.

  • path (str) – The path of the notebook.

  • generation (int) – The generation of the notebook.

  • nbformat (int) – The notebook format version.

  • nbformat_minor (int) – The notebook format minor version.

  • metadata (dict) – The metadata of the notebook.

  • cells (List[CodespaceNotebookCell]) – The list of cells in the notebook.

  • kernel_id (Optional[str]) – The ID of the kernel. Optional.

class datarobot._experimental.models.notebooks.session.NotebookSession

Notebook session information.

Variables:

  • status (NotebookStatus) – The current status of the notebook kernel.

  • notebook_id (str) – The ID of the notebook.

  • session_id (str) – The ID of the session. Incorporates the notebook_id as part of this ID.

  • started_at (Optional[str]) – The date and time when the notebook was started. Optional.

  • session_type (Optional[SessionType]) – The type of the run - either manual (triggered via UI or API) or scheduled. Optional.

  • ephemeral_session_key (Optional[str]) – The ID specific to ephemeral session if being used. Optional.

classmethod get(notebook_id)

Get a notebook session by its notebook ID.

Parameters:

notebook_id (str) – The ID of the notebook.

Returns:

The notebook session information.

Return type:

NotebookSession

classmethod start(notebook_id, payload)

Start a notebook session.

Parameters:

  • notebook_id (str) – The ID of the notebook.

  • payload (StartSessionPayload) – The payload to start the session.

Returns:

The notebook session information.

Return type:

NotebookSession

classmethod stop(notebook_id)

Stop a notebook session.

Parameters:

notebook_id (str) – The ID of the notebook.

Returns:

The notebook session information.

Return type:

NotebookSession

classmethod execute_notebook(notebook_id, cell_ids=None)

Execute a notebook.

Parameters:

  • notebook_id (str) – The ID of the notebook.

  • cell_ids (Optional[List[bson.ObjectId]]) – The list of cell IDs to execute. Optional. If not provided, the whole notebook will be executed.

Return type:

None

classmethod execute_codespace_notebook(notebook_id, notebook_path, generation, cells)

Execute a notebook.

Parameters:

  • notebook_id (str) – The ID of the notebook.

  • notebook_path (str) – The path of the notebook.

  • generation (int) – The generation of the notebook.

  • cells (List[CodespaceNotebookCell]) – The list of cells to execute.

Return type:

None

classmethod get_execution_status(notebook_id)

Get the execution status information of a notebook.

Parameters:

notebook_id (str) – The ID of the notebook.

Returns:

The execution status information of the notebook.

Return type:

NotebookExecutionStatus

class datarobot._experimental.models.notebooks.settings.NotebookSettings

Settings for a DataRobot Notebook.

Variables:

  • show_line_numbers (bool) – Whether line numbers in cells should be displayed.

  • hide_cell_titles (bool) – Whether cell titles should be displayed.

  • hide_cell_outputs (bool) – Whether the cell outputs should be displayed.

  • show_scrollers (bool) – Whether scroll bars should be shown on cells.

  • hide_cell_footers (bool) – Whether footers should be shown on cells.

  • highlight_whitespace (bool) – Whether whitespace should be highlighted or not.

class datarobot._experimental.models.notebooks.user.NotebookUser

A user associated with a Notebook.

Variables:

  • id (str) – The ID of the user.

  • activated (bool) – Whether or not the user is enabled.

  • username (str) – The username of the user, usually their email address.

  • first_name (str) – The first name of the user.

  • last_name (str) – The last name of the user.

  • gravatar_hash (Optional[str]) – The gravatar hash of the user. Optional.

  • tenant_phase (Optional[str]) – The phase that the user’s tenant is in. Optional.

class datarobot._experimental.models.notebooks.user.NotebookActivity

A record of activity (i.e. last run, updated, etc.) in a Notebook.

Variables:

  • at (str) – The time of the activity in the notebook.

  • by (NotebookUser) – The user who performed the activity.

class datarobot._experimental.models.chunking_service.ChunkStorage

The chunk storage location for the data chunks.

Variables:

  • storage_reference_id (str) – The ID of the storage entity.

  • chunk_storage_type (str) – The type of the chunk storage.

  • version_id (str) – The catalog version ID. This will only be used if the storage type is “AI Catalog”.

class datarobot._experimental.models.chunking_service.Chunk

Data chunk object that holds metadata about a chunk.

Variables:

  • id (str) – The ID of the chunk entity.

  • chunk_definition_id (str) – The ID of the dataset chunk definition the chunk belongs to.

  • limit (int) – The number of rows in the chunk.

  • offset (int) – The offset in the dataset to create the chunk.

  • chunk_index (str) – The index of the chunk if chunks are divided uniformly. Otherwise, it is None.

  • data_source_id (str) – The ID of the data request used to create the chunk.

  • chunk_storage (ChunkStorage) – A list of storage locations where the chunk is stored.

get_chunk_storage_id(storage_type)

Get storage location ID for the chunk.

Parameters:

storage_type (ChunkStorageType) – The storage type where the chunk is stored.

Returns:

storage_reference_id – An ID that references the storage location for the chunk.

Return type:

str

get_chunk_storage_version_id(storage_type)

Get storage version ID for the chunk.

Parameters:

storage_type (ChunkStorageType) – The storage type where the chunk is stored.

Returns:

storage_reference_id – A catalog version ID associated with the AI Catalog dataset ID.

Return type:

str

class datarobot._experimental.models.chunking_service.DatasourceDefinition

Data source definition that holds information of data source for API responses. Do not use this to ‘create’ DatasourceDefinition objects directly, use DatasourceAICatalogInfo and DatasourceDataWarehouseInfo.

Variables:

  • id (str) – The ID of the data source definition.

  • data_store_id (str) – The ID of the data store.

  • credentials_id (str) – The ID of the credentials.

  • table (str) – The data source table name.

  • schema (str) – The offset into the dataset to create the chunk.

  • catalog (str) – The database or catalog name.

  • storage_origin (str) – The origin data source or data warehouse (e.g., Snowflake, BigQuery).

  • data_source_id (str) – The ID of the data request used to generate sampling and metadata.

  • total_rows (str) – The total number of rows in the dataset.

  • source_size (str) – The size of the dataset.

  • estimated_size_per_row (str) – The estimated size per row.

  • columns (str) – The list of column names in the dataset.

  • order_by_columns (List[str]) – A list of columns used to sort the dataset.

  • is_descending_order (bool) – Orders the direction of the data. Defaults to False, ordering from smallest to largest.

  • select_columns (List[str]) – A list of columns to select from the dataset.

  • datetime_partition_column (str) – The datetime partition column name used in OTV projects.

  • validation_pct (float) – The percentage threshold between 0.1 and 1.0 for the first chunk validation.

  • validation_limit_pct (float) – The percentage threshold between 0.1 and 1.0 for the validation kept.

  • validation_start_date (datetime.datetime) – The start date for validation.

  • validation_end_date (datetime.datetime) – The end date for validation.

  • training_end_date (datetime.datetime) – The end date for training.

  • latest_timestamp (datetime.datetime) – The latest timestamp.

  • earliest_timestamp (datetime.datetime) – The earliest timestamp.

class datarobot._experimental.models.chunking_service.DatasourceDataWarehouseInfo

Data source information used at creation time with dataset chunk definition. Data warehouses supported: Snowflake, BigQuery, Databricks

Variables:

  • name (str) – The optional custom name of the data source.

  • table (str) – The data source table name or AI Catalog dataset name.

  • storage_origin (str) – The origin data source or data warehouse (e.g., Snowflake, BigQuery).

  • data_store_id (str) – The ID of the data store.

  • credentials_id (str) – The ID of the credentials.

  • schema (str) – The offset into the dataset to create the chunk.

  • catalog (str) – The database or catalog name.

  • data_source_id (str) – The ID of the data request used to generate sampling and metadata.

  • order_by_columns (List[str]) – A list of columns used to sort the dataset.

  • is_descending_order (bool) – Orders the direction of the data. Defaults to False, ordering from smallest to largest.

  • select_columns (List[str]) – A list of columns to select from the dataset.

  • datetime_partition_column (str) – The datetime partition column name used in OTV projects.

  • validation_pct (float) – The percentage threshold between 0.1 and 1.0 for the first chunk validation.

  • validation_limit_pct (float) – The percentage threshold between 0.1 and 1.0 for the validation kept.

  • validation_start_date (datetime.datetime) – The start date for validation.

  • validation_end_date (datetime.datetime) – The end date for validation.

  • training_end_date (datetime.datetime) – The end date for training.

  • latest_timestamp (datetime.datetime) – The latest timestamp.

  • earliest_timestamp (datetime.datetime) – The earliest timestamp.

class datarobot._experimental.models.chunking_service.DatasourceAICatalogInfo

AI Catalog data source information used at creation time with dataset chunk definition.

Variables:

  • name (str) – The optional custom name of the data source.

  • table (str) – The data source table name or AI Catalog dataset name.

  • storage_origin (str) – The origin data source, always AI Catalog type.

  • catalog_id (str) – The ID of the AI Catalog dataset.

  • catalog_version_id (str) – The ID of the AI Catalog dataset version.

  • order_by_columns (List[str]) – A list of columns used to sort the dataset.

  • is_descending_order (bool) – Orders the direction of the data. Defaults to False, ordering from smallest to largest.

  • select_columns (List[str]) – A list of columns to select from the dataset.

  • datetime_partition_column (str) – The datetime partition column name used in OTV projects.

  • validation_pct (float) – The percentage threshold between 0.1 and 1.0 for the first chunk validation.

  • validation_limit_pct (float) – The percentage threshold between 0.1 and 1.0 for the validation kept.

  • validation_start_date (datetime.datetime) – The start date for validation.

  • validation_end_date (datetime.datetime) – The end date for validation.

  • training_end_date (datetime.datetime) – The end date for training.

  • latest_timestamp (datetime.datetime) – The latest timestamp.

  • earliest_timestamp (datetime.datetime) – The earliest timestamp.

class datarobot._experimental.models.chunking_service.DatasetChunkDefinition

Dataset chunking definition that holds information about how to chunk the dataset.

Variables:

  • id (str) – The ID of the dataset chunk definition.

  • user_id (str) – The ID of the user who created the definition.

  • name (str) – The name of the dataset chunk definition.

  • project_starter_chunk_size (int) – The size, in bytes, of the project starter chunk.

  • user_chunk_size (int) – Chunk size in bytes.

  • datasource_definition_id (str) – The data source definition ID associated with the dataset chunk definition.

  • chunking_type (ChunkingType) –

    The type of chunk creation from the dataset. All possible chunking types can be found under ChunkingType enum, that can be imported from datarobot._experimental.models.enums Types include:

    • INCREMENTAL_LEARNING for non-time aware projects that use a chunk index to create chunks.

    • INCREMENTAL_LEARNING_OTV for OTV projects that use a chunk index to create chunks.

    • SLICED_OFFSET_LIMIT for any dataset in which user provides offset and limit to create chunks.

    SLICED_OFFSET_LIMIT has no indexed based chunks aka method create_by_index() not supported.

classmethod get(dataset_chunk_definition_id)

Retrieve a specific dataset chunk definition metadata.

Parameters:

dataset_chunk_definition_id (str) – The ID of the dataset chunk definition.

Returns:

dataset_chunk_definition – The queried instance.

Return type:

DatasetChunkDefinition

classmethod list(limit=50, offset=0)

Retrieves a list of dataset chunk definitions

Parameters:

  • limit (int) – The maximum number of objects to return. Default is 50.

  • offset (int) – The starting offset of the results. Default is 0.

Returns:

dataset_chunk_definitions – The list of dataset chunk definitions.

Return type:

List[DatasetChunkDefinition]

classmethod create(name, project_starter_chunk_size, user_chunk_size, datasource_info, chunking_type=ChunkingType.INCREMENTAL_LEARNING)

Create a dataset chunk definition. Required for both index-based and custom chunks.

In order to create a dataset chunk definition, you must first:

  • Create a data connection to the target data source via dr.DataStore.create()

  • Create credentials that must be attached to the data connection via dr.Credential.create()

If you have an existing data connections and credentials:

  • Retrieve the data store ID by the canonical name via:

    • [ds for ds in dr.DataStore.list() if ds.canonical_name == <name>][0].id

  • Retrieve the credential ID by the name via:

    • [cr for cr in dr.Credential.list() if ds.name == <name>][0].id

You must create the required ‘datasource_info’ object with the datasource information that corresponds to your use case:

  • DatasourceAICatalogInfo for AI catalog datasets.

  • DatasourceDataWarehouseInfo for Snowflake, BigQuery, or other data warehouse.

Parameters:

  • name (str) – The name of the dataset chunk definition.

  • project_starter_chunk_size (int) – The size, in bytes, of the first chunk. Used to start a DataRobot project.

  • user_chunk_size (int) – The size, in bytes, of the user-defined incremental chunk.

  • datasource_info (Union[DatasourceDataWarehouseInfo, DatasourceAICatalogInfo]) – The object that contains the information of the data source.

  • chunking_type (ChunkingType) –

    The type of chunk creation from the dataset. All possible chunking types can be found under ChunkingType enum, that can be imported from datarobot._experimental.models.enums Types include:

    • INCREMENTAL_LEARNING for non-time aware projects that use a chunk index to create chunks.

    • INCREMENTAL_LEARNING_OTV for OTV projects that use a chunk index to create chunks.

    • SLICED_OFFSET_LIMIT for any dataset in which user provides offset and limit to create chunks.

    SLICED_OFFSET_LIMIT has no indexed based chunks aka method create_by_index() not supported. The default type is ChunkingType.INCREMENTAL_LEARNING

Returns:

dataset_chunk_definition – An instance of a created dataset chunk definition.

Return type:

DatasetChunkDefinition

classmethod get_datasource_definition(dataset_chunk_definition_id)

Retrieves the data source definition associated with a dataset chunk definition.

Parameters:

dataset_chunk_definition_id (str) – id of the dataset chunk definition

Returns:

datasource_definition – an instance of created datasource definition

Return type:

DatasourceDefinition

classmethod get_chunk(dataset_chunk_definition_id, chunk_id)

Retrieves a specific data chunk associated with a dataset chunk definition

Parameters:

  • dataset_chunk_definition_id (str) – id of the dataset chunk definition

  • chunk_id (str) – id of the chunk

Returns:

chunk – an instance of created chunk

Return type:

Chunk

classmethod list_chunks(dataset_chunk_definition_id)

Retrieves all data chunks associated with a dataset chunk definition

Parameters:

dataset_chunk_definition_id (str) – id of the dataset chunk definition

Returns:

chunks – a list of chunks

Return type:

List[Chunk]

analyze_dataset(max_wait_time=600)

Analyzes the data source to retrieve and compute metadata about the dataset.

Depending on the size of the data set, adding order_by_columns to the dataset chunking definition will increase the execution time to create the data chunk. Set the max_wait_time for the appropriate wait time.

Parameters:

max_wait_time (int) – maximum time to wait for completion

Returns:

datasource_definition – an instance of created datasource definition

Return type:

DatasourceDefinition

create_chunk(limit, offset=0, storage_type=ChunkStorageType.DATASTAGE, max_wait_time=600)

Creates a data chunk using the limit and offset. By default, the data chunk is stored in data stages.

Depending on the size of the data set, adding order_by_columns to the dataset chunking definition will increase the execution time to retrieve or create the data chunk. Set the max_wait_time for the appropriate wait time.

Parameters:

  • limit (int) – The maximum number of rows.

  • offset (int) – The offset into the dataset (where reading begins).

  • storage_type (ChunkStorageType) – The storage location of the chunk.

  • max_wait_time (int) – maximum time to wait for completion

Returns:

chunk – An instance of a created or updated chunk.

Return type:

Chunk

create_chunk_by_index(index, storage_type=ChunkStorageType.DATASTAGE, max_wait_time=600)

Creates a data chunk using the limit and offset. By default, the data chunk is stored in data stages.

Depending on the size of the data set, adding order_by_columns to the dataset chunking definition will increase the execution time to retrieve or create the data chunk. Set the max_wait_time for the appropriate wait time.

Parameters:

  • index (int) – The index of the chunk.

  • storage_type (ChunkStorageType) – The storage location of the chunk.

  • max_wait_time (int) – maximum time to wait for completion

Returns:

chunk – An instance of a created or updated chunk.

Return type:

Chunk

classmethod patch_validation_dates(dataset_chunk_definition_id, validation_start_date, validation_end_date)

Updates the data source definition validation dates associated with a dataset chunk definition. In order to set the validation dates appropriately, both start and end dates should be specified. This method can only be used for INCREMENTAL_LEARNING_OTV dataset chunk definitions and its associated datasource definition.

Parameters:

  • dataset_chunk_definition_id (str) – The ID of the dataset chunk definition.

  • validation_start_date (datetime.datetime) – The start date of validation scoring data. Internally converted to format ‘%Y-%m-%d %H:%M:%S’, the timezone defaults to UTC.

  • validation_end_date (datetime.datetime) – The end date of validation scoring data. Internally converted to format ‘%Y-%m-%d %H:%M:%S’, the timezone defaults to UTC.

Returns:

datasource_definition – An instance of created datasource definition.

Return type:

DatasourceDefinition