LLM Generation

class datarobot.models.genai.custom_model_validation.CustomModelValidation

Bases: APIObject

Validation record checking the ability of the deployment to serve as a custom model LLM or vector database.

Variables:

prompt_column_name (str) – The column name that the deployed model expects as the input.
target_column_name (str) – The target name that the deployed model will output.
deployment_id (str) – ID of the deployment.
model_id (str) – ID of the underlying deployment model. Can be found from the API as Deployment.model[“id”].
validation_status (str) – Can be TESTING, FAILED, or PASSED. Only PASSED is allowed for use.
deployment_access_data (dict, optional) – Data that will be used for accessing deployment prediction server. Only available for deployments that pass validation. Dict fields: - prediction_api_url - URL for deployment prediction server. - datarobot_key - First of two auth headers for the prediction server. - authorization_header - Second of two auth headers for the prediction server. - input_type - Either JSON or CSV - input type model expects. - model_type - Target type of deployed custom model.
tenant_id (str) – Creating user’s tenant ID.
error_message (Optional[str]) – Additional information for errored validation.
deployment_name (Optional[str]) – The name of the deployment that is validated.
user_name (Optional[str]) – The name of the user
use_case_id (Optional[str]) – The ID of the Use Case associated with the validation.
prediction_timeout (int) – The timeout, in seconds, for the prediction API used in this custom model validation.

classmethod get(validation_id)

Get the validation record by id.

Parameters:: validation_id (Union[CustomModelValidation, str]) – The CustomModelValidation to retrieve, either CustomModelValidation or validation ID.
Return type:: CustomModelValidation

classmethod get_by_values(prompt_column_name, target_column_name, deployment_id, model_id)

Get the validation record by field values.

Parameters:

prompt_column_name (str) – The column name the deployed model expect as the input.
target_column_name (str) – The target name deployed model will output.
deployment_id (str) – ID of the deployment.
model_id (str) – ID of the underlying deployment model.

Return type:

CustomModelValidation

classmethod list(prompt_column_name=None, target_column_name=None, deployment=None, model=None, use_cases=None, playground=None, completed_only=False, search=None, sort=None)

List the validation records by field values.

Parameters:

prompt_column_name (Optional[str], optional) – The column name the deployed model expects as the input.
target_column_name (Optional[str], optional) – The target name that the deployed model will output.
deployment (Optional[Union[Deployment, str]], optional) – The returned validations are filtered to those associated with a specific deployment if specified, either Deployment or deployment ID.
model_id (Optional[Union[Model, str]], optional) – The returned validations are filtered to those associated with a specific model if specified, either Model or model ID.
use_cases (Optional[list[Union[UseCase, str]]], optional) – The returned validations are filtered to those associated with specific Use Cases if specified, either UseCase or Use Case IDs.
playground_id (Optional[Union[Playground, str]], optional) – The returned validations are filtered to those used in a specific playground if specified, either Playground or playground ID.
completed_only (Optional[bool]) – Whether to retrieve only completed validations.
search (Optional[str], optional) – String for filtering validations. Validations that contain the string in name will be returned.
sort (Optional[str], optional) – Property to sort validations by. Prefix the attribute name with a dash to sort in descending order, e.g. sort=’-name’. Currently supported options are listed in ListCustomModelValidationsSortQueryParams but the values can differ with different platform versions. By default, the sort parameter is None which will result in validations being returned in order of creation time descending.

Return type:

List[CustomModelValidation]

classmethod create(prompt_column_name, target_column_name, deployment_id, model=None, use_case=None, name=None, wait_for_completion=False, prediction_timeout=None)

Start the validation of deployment to serve as a vector database or LLM.

Parameters:

prompt_column_name (str) – The column name the deployed model expect as the input.
target_column_name (str) – The target name that the deployed model will output.
deployment_id (Union[Deployment, str]) – The deployment to validate, either Deployment or deployment ID.
model (Optional[Union[Model, str]], optional) – The specific model within the deployment, either Model or model ID. If not specified, the underlying model ID will be derived from the deployment info automatically.
use_case (Optional[Union[UseCase, str]], optional) – The Use Case to link the validation to, either UseCase or Use Case ID.
name (Optional[str], optional) – The name of the validation.
wait_for_completion (bool) – If set to True code will wait for the validation job to complete before returning the result (up to 10 minutes, raising timeout error after that). Otherwise, you can check current validation status by using CustomModelValidation.get with returned ID.
prediction_timeout (Optional[int], optional) – The timeout, in seconds, for the prediction API used in this custom model validation.

Return type:

CustomModelValidation

classmethod revalidate(validation_id)

Revalidate an unlinked custom model vector database or LLM. This method is useful when a deployment used as vector database or LLM is accidentally replaced with another model that stopped complying with the vector database or LLM requirements. Replace the model back and call this method instead of creating a new custom model validation from scratch. Another use case for this is when the API token used to create a validation record got revoked and no longer can be used by vector database / LLM to call custom model deployment. Calling revalidate will update the validation record with the token currently in use.

Parameters:: validation_id (str) – The ID of the CustomModelValidation for revalidation.
Return type:: CustomModelValidation

update(name=None, prompt_column_name=None, target_column_name=None, deployment=None, model=None, prediction_timeout=None)

Update a custom model validation.

Parameters:

name (Optional[str], optional) – The new name of the custom model validation.
prompt_column_name (Optional[str], optional) – The new name of the prompt column.
target_column_name (Optional[str], optional) – The new name of the target column.
deployment (Optional[Union[Deployment, str]], optional) – The new deployment to validate.
model (Optional[Union[Model, str]], optional) – The new model within the deployment to validate.
prediction_timeout (Optional[int], optional) – The new timeout, in seconds, for the prediction API used in this custom model validation.

Return type:

CustomModelValidation

delete()

Delete the custom model validation.

Return type:: None

class datarobot.models.genai.custom_model_llm_validation.CustomModelLLMValidation

Bases: CustomModelValidation

Validation record checking the ability of the deployment to serve as a custom model LLM.

Variables:

prompt_column_name (str) – The column name the deployed model expect as the input.
target_column_name (str) – The target name that the deployed model will output.
deployment_id (str) – ID of the deployment.
model_id (str) – ID of the underlying deployment model. Can be found from the API as Deployment.model[“id”].
validation_status (str) – Can be TESTING, FAILED, or PASSED. Only PASSED is allowed for use.
deployment_access_data (dict, optional) – Data that will be used for accessing deployment prediction server. Only available for deployments that passed validation. Dict fields: - prediction_api_url - URL for deployment prediction server. - datarobot_key - first of 2 auth headers for the prediction server. - authorization_header - second of 2 auth headers for the prediction server. - input_type - Either JSON or CSV - the input type that the model expects. - model_type - Target type of the deployed custom model.
tenant_id (str) – Creating user’s tenant ID.
error_message (Optional[str]) – Additional information for errored validation.
deployment_name (Optional[str]) – The name of the deployment that is validated.
user_name (Optional[str]) – The name of the user
use_case_id (Optional[str]) – The ID of the Use Case associated with the validation.
prediction_timeout (int) – The timeout in seconds for the prediction API used in this custom model validation.

class datarobot.models.genai.llm_blueprint.LLMBlueprint

Bases: APIObject

Metadata for a DataRobot GenAI LLM blueprint.

Variables:

id (str) – The LLM blueprint ID.
name (str) – The LLM blueprint name.
description (str) – A description of the LLM blueprint.
is_saved (bool) – Whether the LLM blueprint is saved (meaning the settings are locked and blueprint is eligible for use with ComparisonPrompts).
is_starred (bool) – Whether the LLM blueprint is starred.
playground_id (str) – The ID of the playground associated with the LLM blueprint.
llm_id (str or None) – The ID of the LLM type. If not None, this must be one of the IDs returned by LLMDefinition.list for this user.
llm_name (str or None) – The name of the LLM.
llm_settings (dict or None) – The LLM settings for the LLM blueprint. The specific keys allowed and the constraints on the values are defined in the response from LLMDefinition.list but this typically has dict fields: - system_prompt - The system prompt that tells the LLM how to behave. - max_completion_length - The maximum number of tokens in the completion. - temperature - Controls the variability in the LLM response. - top_p - Whether the model considers next tokens with top_p probability mass. Or - system_prompt - The system prompt that tells the LLM how to behave. - validation_id - The ID of the external model LLM validation. - external_llm_context_size - The external LLM’s context size, in tokens, for external model LLM blueprints.
creation_date (str) – The date the playground was created.
creation_user_id (str) – The ID of the user creating the playground.
creation_user_name (str) – The name of the user creating the playground.
last_update_date (str) – The date the playground was last updated.
last_update_user_id (str) – The ID of the user who most recently updated the playground.
prompt_type (PromptType) – The prompting strategy for the LLM blueprint. Currently supported options are listed in PromptType.
vector_database_id (str or None) – The ID of the vector database, if any, associated with the LLM blueprint.
vector_database_settings (VectorDatabaseSettings or None) – The settings for the vector database, if any, associated with the LLM blueprint.
vector_database_name (str or None) – The name of the vector database associated with the LLM blueprint, if any.
vector_database_status (str or None) – The status of the vector database, if any, associated with the LLM blueprint.
vector_database_error_message (str or None) – The error message for the vector database, if any, associated with the LLM blueprint.
vector_database_error_resolution (str or None) – The resolution for the vector database error, if any, associated with the LLM blueprint.
custom_model_llm_validation_status (str or None) – The status of the custom model LLM validation if the llm_id is ‘custom-model’.
custom_model_llm_error_message (str or None) – The error message for the custom model LLM, if any.
custom_model_llm_error_resolution (str or None) – The resolution for the custom model LLM error, if any.

classmethod create(playground, name, prompt_type=PromptType.CHAT_HISTORY_AWARE, description='', llm=None, llm_settings=None, vector_database=None, vector_database_settings=None)

Create a new LLM blueprint.

Parameters:

playground (Playground or str) – The playground associated with the created LLM blueprint. Accepts playground or playground ID.
name (str) – The LLM blueprint name.
prompt_type (PromptType, optional) – Prompting type of the LLM blueprint, by default PromptType.CHAT_HISTORY_AWARE.
description (Optional[str]) – An optional description for the LLM blueprint, otherwise null.
llm (LLMDefinition, str, or None, optional) – The LLM to use for the blueprint, either LLMDefinition or LLM ID.
llm_settings (dict or None) – The LLM settings for the LLM blueprint. The specific keys allowed and the constraints on the values are defined in the response from LLMDefinition.list but this typically has dict fields: - system_prompt - The system prompt that tells the LLM how to behave. - max_completion_length - The maximum number of tokens in the completion. - temperature - Controls the variability in the LLM response. - top_p - Whether the model considers next tokens with top_p probability mass. Or - system_prompt - The system prompt that tells the LLM how to behave. - validation_id - The ID of the custom model LLM validation for custom model LLM blueprints.
vector_database (VectorDatabase, str, or None, optional) – The vector database to use with this LLM blueprint, either VectorDatabase or vector database ID.
vector_database_settings (VectorDatabaseSettings or None, optional) – The settings for the vector database, if any.

Returns:

llm_blueprint – The created LLM blueprint.

Return type:

LLMBlueprint

classmethod create_from_llm_blueprint(llm_blueprint, name, description='')

Create a new LLM blueprint from an existing LLM blueprint.

Parameters:

llm_blueprint (LLMBlueprint or str) – The LLM blueprint to use to create the new LLM blueprint. Accepts LLM blueprint or LLM blueprint ID.
name (str) – LLM blueprint name.
description (Optional[str]) – Description of the LLM blueprint, by default “”.

Returns:

llm_blueprint – The created LLM blueprint.

Return type:

LLMBlueprint

classmethod get(llm_blueprint_id)

Retrieve a single LLM blueprint.

Parameters:: llm_blueprint_id (str) – The ID of the LLM blueprint you want to retrieve.
Returns:: llm_blueprint – The requested LLM blueprint.
Return type:: LLMBlueprint

classmethod list(playground=None, llms=None, vector_databases=None, is_saved=None, is_starred=None, sort=None)

Lists all LLM blueprints available to the user. If the playground is specified, then the results are restricted to the LLM blueprints associated with the playground. If the LLMs are specified, then the results are restricted to the LLM blueprints using those LLM types. If vector_databases are specified, then the results are restricted to the LLM blueprints using those vector databases.

Parameters:

playground (Optional[Union[Playground, str]], optional) – The returned LLM blueprints are filtered to those associated with a specific playground if it is specified. Accepts either the entity or the ID.
llms (Optional[list[Union[LLMDefinition, str]]], optional) – The returned LLM blueprints are filtered to those associated with the LLM types specified. Accepts either the entity or the ID.
vector_databases (Optional[list[Union[VectorDatabase, str]]], optional) – The returned LLM blueprints are filtered to those associated with the vector databases specified. Accepts either the entity or the ID.
is_saved (Optional[bool], optional) – The returned LLM blueprints are filtered to those matching is_saved.
is_starred (Optional[bool], optional) – The returned LLM blueprints are filtered to those matching is_starred.
sort (Optional[str]) – Property to sort LLM blueprints by. Prefix the attribute name with a dash to sort in descending order, e.g. sort=’-creationDate’. Currently supported options are listed in ListLLMBlueprintsSortQueryParams but the values can differ with different platform versions. By default, the sort parameter is None which will result in LLM blueprints being returned in order of creation time descending.

Returns:

playgrounds – A list of playgrounds available to the user.

Return type:

list[Playground]

update(name=None, description=None, llm=None, llm_settings=None, vector_database=None, vector_database_settings=None, is_saved=None, is_starred=None, prompt_type=None, remove_vector_database=False)

Update the LLM blueprint.

Parameters:

name (str or None, optional) – The new name for the LLM blueprint.
description (str or None, optional) – The new description for the LLM blueprint.
llm (Optional[Union[LLMDefinition, str]], optional) – The new LLM type for the LLM blueprint.
llm_settings (Optional[dict], optional) – The new LLM settings for the LLM blueprint. These must match the LLMSettings returned from the LLMDefinition.list method for the LLM type used for this LLM blueprint but this typically has dict fields: - system_prompt - The system prompt that tells the LLM how to behave. - max_completion_length - The maximum number of tokens in the completion. - temperature - Controls the variability in the LLM response. - top_p - Whether the model considers next tokens with top_p probability mass. Or - system_prompt - The system prompt that tells the LLM how to behave. - validation_id - The ID of the custom model LLM validation for custom model LLM blueprints.
vector_database (Optional[Union[VectorDatabase, str]], optional) – The new vector database for the LLM blueprint.
vector_database_settings (Optional[VectorDatabaseSettings], optional) – The new vector database settings for the LLM blueprint.
is_saved (Optional[bool], optional) – The new is_saved attribute for the LLM blueprint (meaning the settings are locked and blueprint is eligible for use with ComparisonPrompts).
is_starred (Optional[bool], optional) – The new setting for whether the LLM blueprint is starred.
prompt_type (PromptType, optional) – The new prompting type of the LLM blueprint.
remove_vector_database (Optional[bool], optional) – Whether to remove the vector database from the LLM blueprint.

Returns:

llm_blueprint – The updated LLM blueprint.

Return type:

LLMBlueprint

delete()

Delete the single LLM blueprint.

Return type:: None

register_custom_model(prompt_column_name=None, target_column_name=None, llm_test_configuration_ids=None)

Create a new CustomModelVersion. This registers a custom model from the LLM blueprint.

Parameters:

prompt_column_name (Optional[str]) – The column name of the prompt text.
target_column_name (Optional[str]) – The column name of the response text.
llm_test_configuration_ids (List[str], optional) – The IDs of the LLM test configurations to execute.

Returns:

custom_model – The registered custom model.

Return type:

CustomModelVersion

class datarobot.models.genai.llm.LLMDefinition

Bases: APIObject

Metadata for a DataRobot GenAI LLM.

Variables:

id (str) – Language model type ID.
name (str) – Language model name.
description (str) – Description of the language model.
vendor (str) – Name of the vendor for this model.
license (str) – License for this model.
supported_languages (str) – Languages supported by this model.
settings (list of LLMSettingDefinition) – Settings for this model
context_size (int) – The context size for this model

classmethod list(use_case=None, as_dict=True)

List all large language models (LLMs) available to the user.

Parameters:: use_case (Optional[UseCase or str], optional) – The returned LLMs, including external LLMs, available for the specified Use Case. Accepts either the entity or the Use CaseID.
Returns:: llms – A list of large language models (LLMs) available to the user.
Return type:: list[LLMDefinition] or list[LLMDefinitionDict]

to_dict()

Return type:: LLMDefinitionDict

class datarobot.models.genai.llm.LLMDefinitionDict

Bases: dict

id: str

name: str

description: str

vendor: str

license: str

supported_languages: str

settings: List[LLMSettingDefinitionDict]

context_size: Optional[int]