Unsupervised Projects (Clustering)¶

Use clustering when data is not labelled and the problem can be interpreted as grouping a set of objects in such a way that objects in the same group (called a cluster) are more similar to each other than to those in other groups (clusters). It is a common task in data exploration when finding groups and similarities is needed.

Creating Unsupervised Projects¶

To create an unsupervised project, set unsupervised_mode to True when setting the target. To specify clustering, set unspervised_type to CLUSTERING.

Example:

from datarobot import Project
from datarobot.enums import UnsupervisedTypeEnum

project = Project.create("dataset.csv", project_name="unsupervised clustering")
project.set_target(
    unsupervised_mode=True,
    unsupervised_type=UnsupervisedTypeEnum.CLUSTERING)

You can optionally specify list of explicit cluster numbers. To do this, pass a list of integer values to optional autopilot_cluster_list parameter using the set_target() method.

project.set_target(
    unsupervised_mode=True,
    unsupervised_type=UnsupervisedTypeEnum.CLUSTERING,
    autopilot_cluster_list=[7, 9, 11, 15, 19]

You can also do both in one step using the Project.start() method:

from datarobot import Project
from datarobot.enums import UnsupervisedTypeEnum

project = Project.start("dataset.csv",
              unsupervised_mode=True,
              project_name="unsupervised clustering project",
              unsupervised_type=UnsupervisedTypeEnum.CLUSTERING)

Unsupervised Clustering Project Metric¶

Unsupervised clustering projects use the Silhouette Score metric for model ranking (instead of using it for model optimization). It measures the average similarity of objects within a cluster and their distance to the other objects in the other clusters.

Retrieving information about Clusters¶

In a trained model, you can retrieve information about clusters in along with standard model information. To do this, when training completes, retrieve a model and view basic clustering information:

n_clusters : number of clusters for model

is_n_clusters_dynamically_determined : how clustering model picks number of clusters

Here is a code snippet to retrieve information about the number of clusters for model:

from datarobot import ClusteringModel
model = ClusteringModel.get(project_id, model_id)
print("{} clusters found".format(model.n_clusters))

You can retrieve more details about clusters and their data using cluster insights.

Working with Clusters Insights¶

You can compute insights to gain deep insights into clusters and their characteristics. This process will perform calculations and return detailed information about each feature and its importance, as well as a detailed per-cluster breakdown.

To compute and retrieve cluster insights, use the ClusteringModel and its compute_insights method. The method starts the cluster insights compute job, waits for its completion for the number of seconds specified in the optional parameter max_wait (default: 600), and returns results when insights are ready.

If clusters are already computed, access them using the insights property of the ClusteringModel method.

from datarobot import ClusteringModel
model = ClusteringModel.get(project_id, model_id)
insights = model.compute_insights()

This call, with the specified wait_time, will run and wait for specified time:

from datarobot import ClusteringModel
model = ClusteringModel.get(project_id, model_id)
insights = model.compute_insights(max_wait=60)

If computation fails to finish before max_wait expires, the method will raise an AsyncTimeoutError. You can retrieve cluster insights after jobs computation finishes.

To retrieve cluster insights already computed:

from datarobot import ClusteringModel
model = ClusteringModel.get(project_id, model_id)
for insight in model.insights:
    print(insight)

Working with Clusters¶

By default, DataRobot names clusters “Cluster 1”, “Cluster 2”, … , “Cluster N” . You can retrieve these names and alter them according to preference. When retrieving clusters before computing insights, clusters will contain only names. After insight computation completes, each cluster will also hold information about the percentage of data that is represented by the Cluster.

For example:

from datarobot import ClusteringModel
model = ClusteringModel.get(project_id, model_id)

# helper function
def print_summary(name, percent):
    if not percent:
        percent = "?"
    print("'{}' holds {} % of data".format(name, percent))

for cluster in model.clusters:
    print_summary(cluster.name, cluster.percent)
model.compute_insights()
for cluster in model.clusters:
    print_summary(cluster.name, cluster.percent)

For a model with three clusters, the code snippet will output:

'Cluster 1' holds ? % of data
'Cluster 2' holds ? % of data
'Cluster 3' holds ? % of data
-- Cluster insights computation finished --
'Cluster 1' holds 27.1704180064 % of data
'Cluster 2' holds 36.9131832797 % of data
'Cluster 3' holds 35.9163987138 % of data

Use the following methods of ClusteringModel class to alter cluster names:

update_cluster_names - changes multiple cluster names using mapping in dictionary
update_cluster_name - changes one cluster name

After update, each method will return a list of clusters with changed names.

For example:

from datarobot import ClusteringModel
model = ClusteringModel.get(project_id, model_id)

# update multiple
cluster_name_mappings = [
    ("Cluster 1", "AAA"),
    ("Cluster 2", "BBB"),
    ("Cluster 3", "CCC")
]
clusters = model.update_cluster_names(cluster_name_mappings)

# update single
clusters = model.update_cluster_name("CCC", "DDD")

Clustering Classes Reference¶

ClusteringModel¶

class datarobot.models.model.ClusteringModel(id=None, processes=None, featurelist_name=None, featurelist_id=None, project_id=None, sample_pct=None, training_row_count=None, training_duration=None, training_start_date=None, training_end_date=None, model_type=None, model_category=None, is_frozen=None, is_n_clusters_dynamically_determined=None, blueprint_id=None, metrics=None, project=None, data=None, monotonic_increasing_featurelist_id=None, monotonic_decreasing_featurelist_id=None, n_clusters=None, has_empty_clusters=None, supports_monotonic_constraints=None, is_starred=None, prediction_threshold=None, prediction_threshold_read_only=None, model_number=None, parent_model_id=None, use_project_settings=None, supports_composable_ml=None)¶

ClusteringModel extends Model class. It provides provides properties and methods specific to clustering projects.

compute_insights(max_wait=600)¶

Compute and retrieve cluster insights for model. This method awaits completion of job computing cluster insights and returns results after it is finished. If computation takes longer than specified max_wait exception will be raised.

Parameters:	project_id: str Project to start creation in. model_id: str Project’s model to start creation in. max_wait: int Maximum number of seconds to wait before giving up
Returns:	List of ClusterInsight
Raises:	ClientError Server rejected creation due to client error. Most likely cause is bad `project_id` or `model_id`. AsyncFailureError If any of the responses from the server are unexpected AsyncProcessUnsuccessfulError If the cluster insights computation has failed or was cancelled. AsyncTimeoutError If the cluster insights computation did not resolve in time

insights¶

Return actual list of cluster insights if already computed.

Returns:	List of ClusterInsight

clusters¶

Return actual list of Clusters.

Returns:	List of Cluster

update_cluster_names(cluster_name_mappings)¶

Change many cluster names at once based on list of name mappings.

Parameters:	cluster_name_mappings: List of tuples Cluster names mapping consisting of current cluster name and old cluster name. Example: cluster_name_mappings = [ ("current cluster name 1", "new cluster name 1"), ("current cluster name 2", "new cluster name 2")]
Returns:	List of Cluster
Raises:	datarobot.errors.ClientError Server rejected update of cluster names. Possible reasons include: incorrect format of mapping, mapping introduces duplicates.

update_cluster_name(current_name, new_name)¶

Change cluster name from current_name to new_name.

Parameters:	current_name: str Current cluster name. new_name: str New cluster name.
Returns:	List of Cluster
Raises:	datarobot.errors.ClientError Server rejected update of cluster names.

Cluster¶

class datarobot.models.model.Cluster(**kwargs)¶

Representation of a single cluster.

Attributes:	name: str Current cluster name percent: float Percent of data contained in the cluster. This value is reported after cluster insights are computed for the model.

classmethod list(project_id, model_id)¶

Retrieve a list of clusters in the model.

Parameters:	project_id: str ID of the project that the model is part of. model_id: str ID of the model.
Returns:	List of clusters

classmethod update_multiple_names(project_id, model_id, cluster_name_mappings)¶

Update many clusters at once based on list of name mappings.

Parameters:	project_id: str ID of the project that the model is part of. model_id: str ID of the model. cluster_name_mappings: List of tuples Cluster name mappings, consisting of current and previous names for each cluster. Example: cluster_name_mappings = [ ("current cluster name 1", "new cluster name 1"), ("current cluster name 2", "new cluster name 2")]
Returns:	List of clusters
Raises:	datarobot.errors.ClientError Server rejected update of cluster names. ValueError Invalid cluster name mapping provided.

classmethod update_name(project_id, model_id, current_name, new_name)¶

Change cluster name from current_name to new_name

Parameters:	project_id: str ID of the project that the model is part of. model_id: str ID of the model. current_name: str Current cluster name new_name: str New cluster name
Returns:	List of Cluster

ClusterInsight¶

class datarobot.models.model.ClusterInsight(**kwargs)¶

Holds data on all insights related to feature as well as breakdown per cluster.

Parameters:

feature_name: str: Name of a feature from the dataset.
feature_type: str: Type of feature.
insights : List of classes (ClusterInsight): List provides information regarding the importance of a specific feature in relation to each cluster. Results help understand how the model is grouping data and what each cluster represents.
feature_impact: float: Impact of a feature ranging from 0 to 1.

classmethod compute(project_id, model_id, max_wait=600)¶

Starts creation of cluster insights for the model and if successful, returns computed ClusterInsights. This method allows calculation to continue for a specified time and if not complete, cancels the request.

Parameters:	project_id: str ID of the project to begin creation of cluster insights for. model_id: str ID of the project model to begin creation of cluster insights for. max_wait: int Maximum number of seconds to wait canceling the request.
Returns:	ClusterInsights
Raises:	ClientError Server rejected creation due to client error. Most likely cause is bad `project_id` or `model_id`. AsyncFailureError Indicates whether any of the responses from the server are unexpected. AsyncProcessUnsuccessfulError Indicates whether the cluster insights computation failed or was cancelled. AsyncTimeoutError Indicates whether the cluster insights computation did not resolve within the specified time limit (max_wait).