External Scores and Insights
- class datarobot.ExternalScores(project_id, scores, model_id=None, dataset_id=None, actual_value_column=None)
Metric scores on prediction dataset with target or actual value column in unsupervised case. Contains project metrics for supervised and special classification metrics set for unsupervised projects.
Added in version v2.21.
Examples
List all scores for a dataset
from datarobot.models.external_dataset_scores_insights.external_scores import ExternalScores scores = ExternalScores.list(project_id, dataset_id=dataset_id)
- Attributes:
- project_id: str
id of the project the model belongs to
- model_id: str
id of the model
- dataset_id: str
id of the prediction dataset with target or actual value column for unsupervised case
- actual_value_column: str, optional
For unsupervised projects only. Actual value column which was used to calculate the classification metrics and insights on the prediction dataset.
- scores: list of dicts in a form of {‘label’: metric_name, ‘value’: score}
Scores on the dataset.
- classmethod create(project_id, model_id, dataset_id, actual_value_column=None)
Compute an external dataset insights for the specified model.
- Parameters:
- project_idstr
id of the project the model belongs to
- model_idstr
id of the model for which insights is requested
- dataset_idstr
id of the dataset for which insights is requested
- actual_value_columnstr, optional
actual values column label, for unsupervised projects only
- Returns:
- jobJob
an instance of created async job
- Return type:
- classmethod list(project_id, model_id=None, dataset_id=None, offset=0, limit=100)
Fetch external scores list for the project and optionally for model and dataset.
- Parameters:
- project_id: str
id of the project
- model_id: str, optional
if specified, only scores for this model will be retrieved
- dataset_id: str, optional
if specified, only scores for this dataset will be retrieved
- offset: int, optional
this many results will be skipped, default: 0
- limit: int, optional
at most this many results are returned, default: 100, max 1000. To return all results, specify 0
- Returns:
- A list of
External Scoresobjects
- A list of
- Return type:
List[ExternalScores]
- classmethod get(project_id, model_id, dataset_id)
Retrieve external scores for the project, model and dataset.
- Parameters:
- project_id: str
id of the project
- model_id: str
if specified, only scores for this model will be retrieved
- dataset_id: str
if specified, only scores for this dataset will be retrieved
- Returns:
External Scoresobject
- Return type:
- class datarobot.ExternalLiftChart(dataset_id, bins)
Lift chart for the model and prediction dataset with target or actual value column in unsupervised case.
Added in version v2.21.
LiftChartBinis a dict containing the following:actual(float) Sum of actual target values in binpredicted(float) Sum of predicted target values in binbin_weight(float) The weight of the bin. For weighted projects, it is the sum of the weights of the rows in the bin. For unweighted projects, it is the number of rows in the bin.
- Attributes:
- dataset_id: str
id of the prediction dataset with target or actual value column for unsupervised case
- bins: list of dict
List of dicts with schema described as
LiftChartBinabove.
- classmethod list(project_id, model_id, dataset_id=None, offset=0, limit=100)
Retrieve list of the lift charts for the model.
- Parameters:
- project_id: str
id of the project
- model_id: str
if specified, only lift chart for this model will be retrieved
- dataset_id: str, optional
if specified, only lift chart for this dataset will be retrieved
- offset: int, optional
this many results will be skipped, default: 0
- limit: int, optional
at most this many results are returned, default: 100, max 1000. To return all results, specify 0
- Returns:
- A list of
ExternalLiftChartobjects
- A list of
- Return type:
List[ExternalLiftChart]
- classmethod get(project_id, model_id, dataset_id)
Retrieve lift chart for the model and prediction dataset.
- Parameters:
- project_id: str
project id
- model_id: str
model id
- dataset_id: str
prediction dataset id with target or actual value column for unsupervised case
- Returns:
ExternalLiftChartobject
- Return type:
- class datarobot.ExternalRocCurve(dataset_id, roc_points, negative_class_predictions, positive_class_predictions)
ROC curve data for the model and prediction dataset with target or actual value column in unsupervised case.
Added in version v2.21.
- Attributes:
- dataset_id: str
id of the prediction dataset with target or actual value column for unsupervised case
- roc_points: list of dict
List of precalculated metrics associated with thresholds for ROC curve.
- negative_class_predictions: list of float
List of predictions from example for negative class
- positive_class_predictions: list of float
List of predictions from example for positive class
- classmethod list(project_id, model_id, dataset_id=None, offset=0, limit=100)
Retrieve list of the roc curves for the model.
- Parameters:
- project_id: str
id of the project
- model_id: str
if specified, only lift chart for this model will be retrieved
- dataset_id: str, optional
if specified, only lift chart for this dataset will be retrieved
- offset: int, optional
this many results will be skipped, default: 0
- limit: int, optional
at most this many results are returned, default: 100, max 1000. To return all results, specify 0
- Returns:
- A list of
ExternalRocCurveobjects
- A list of
- Return type:
List[ExternalRocCurve]
- classmethod get(project_id, model_id, dataset_id)
Retrieve ROC curve chart for the model and prediction dataset.
- Parameters:
- project_id: str
project id
- model_id: str
model id
- dataset_id: str
prediction dataset id with target or actual value column for unsupervised case
- Returns:
ExternalRocCurveobject
- Return type: