Vector Databases

class datarobot.models.genai.vector_database.CustomModelVectorDatabaseValidation

Bases: CustomModelValidation

Validation record checking the ability of the deployment to serve as a vector database.

Variables:

prompt_column_name (str) – The column name the deployed model expect as the input.
target_column_name (str) – The target name deployed model will output.
deployment_id (str) – ID of the deployment.
model_id (str) – ID of the underlying deployment model. Can be found from the API as Deployment.model[“id”].
validation_status (str) – Can be TESTING, FAILED and PASSED. Only PASSED allowed for use.
deployment_access_data (dict, optional) – Data that will be used for accessing deployment prediction server. Only available for deployments that passed validation. Dict fields: - prediction_api_url - URL for deployment prediction server. - datarobot_key - first of 2 auth headers for prediction server. - authorization_header - second of 2 auth headers for prediction server. - input_type - Either JSON or CSV - input type model expects. - model_type - Target type of deployed custom model.
tenant_id (str) – Creating user’s tenant ID.
error_message (Optional[str]) – Additional information for errored validation.
deployment_name (Optional[str]) – The name of the deployment that is validated.
user_name (Optional[str]) – The name of the user
use_case_id (Optional[str]) – The ID of the use case associated with the validation.

class datarobot.models.genai.vector_database.SupportedEmbeddings

Bases: APIObject

All supported embedding models including the recommended default model.

Variables:

embedding_models (list[EmbeddingModel]) – All supported embedding models.
default_embedding_model (str) – Name of the default recommended text embedding model. Currently supported options are listed in VectorDatabaseEmbeddingModel but the values can differ with different platform versions.
custom_model_embedding_validations (List[str]) – External embedding models that have been validated

class datarobot.models.genai.vector_database.SupportedTextChunkings

Bases: APIObject

Supported text chunking configurations which includes a set of recommended chunking parameters for each supported embedding model.

Variables:: text_chunking_configs – All supported text chunking configurations.

class datarobot.models.genai.vector_database.VectorDatabase

Bases: APIObject

Metadata for a DataRobot vector database accessible to the user.

Variables:

id (str) – Vector database ID.
name (str) – Vector database name.
size (int) – Size of the vector database assets in bytes.
use_case_id (str) – Linked use case ID.
dataset_id (str) – ID of the dataset used for creation.
embedding_model (str) – Name of the text embedding model. Currently supported options are listed in VectorDatabaseEmbeddingModel but the values can differ with different platform versions.
chunking_method (str or None) – Name of the method to split dataset documents. Currently supported options are listed in VectorDatabaseChunkingMethod but the values can differ with different platform versions.
chunk_size (int or None) – Size of each text chunk in number of tokens.
chunk_overlap_percentage (int or None) – Overlap percentage between chunks.
chunks_count (int) – Total number of text chunks.
custom_chunking (bool) – Determines if the chunking is custom. With custom chunking, dataset rows are not split into chunks automatically; instead, the user provides the chunks.
separators (list[string] or None) – Separators for document splitting.
creation_date (str) – Date when the database was created.
creation_user_id (str) – ID of the creating user.
organization_id (str) – Creating user’s organization ID.
tenant_id (str) – Creating user’s tenant ID.
last_update_date (str) – Last update date for the database.
execution_status (str) – Database execution status. Currently supported options are listed in VectorDatabaseExecutionStatus but the values can differ with different platform versions.
playgrounds_count (int) – Number of using playgrounds.
dataset_name (str) – Name of the used dataset.
user_name (str) – Name of the creating user.
source (str) – Source of the vector database. Currently supported options are listed in VectorDatabaseSource but the values can differ with different platform versions.
validation_id (Optional[str]) – ID of custom model vector database validation. Only filled for external vector databases.
error_message (Optional[str]) – Additional information for errored vector database.
embedding_validation_id (Optional[str]) – ID of the custom embedding validation, if any.
is_separator_regex (bool) – Whether the separators should be treated as regular expressions.

classmethod get_supported_embeddings(dataset_id=None, use_case=None)

Get all supported and the recommended embedding models.

Parameters:

dataset_id (Optional[str]) – ID of a dataset for which the recommended model is returned based on the detected language of that dataset.
use_case (Optional[UseCase, str]) – May be Use Case ID or the Use Case entity.

Returns:

supported_embeddings – The supported embedding models.

Return type:

SupportedEmbeddings

submit_export_dataset_job()

Submit the vector database dataset export job.

Returns:: result – The result of the vector database dataset export job containing the exported dataset id.
Return type:: VectorDatabaseDatasetExportJob

classmethod get_supported_retrieval_settings()

Get supported retrieval settings.

Returns:: supported_retrieval_settings – The supported retriever settings.
Return type:: SupportedRetrievalSettings

classmethod create(dataset_id, chunking_parameters=None, use_case=None, name=None, parent_vector_database_id=None, update_llm_blueprints=None, update_deployments=None)

Create a new vector database.

Parameters:

dataset_id (str) – ID of the dataset used for creation.
chunking_parameters (ChunkingParameters) – Parameters defining how documents are split and embedded.
use_case (Optional[Union[UseCase, str]], optional) – Use case to link to the created vector database.
name (Optional[str]) – Vector database name, by default None which leads to the default name ‘Vector Database for <dataset name>’.
parent_vector_database_id (Optional[str]) – ID of the parent vector database to base the update on.
update_llm_blueprints (Optional[bool]) – Whether to update LLM blueprints related to the parent vector database.
update_deployments (Optional[bool]) – Whether to update deployments related to the parent vector database.

Returns:

vector database – The created vector database with execution status ‘new’.

Return type:

VectorDatabase

classmethod create_from_custom_model(name, use_case=None, validation_id=None, prompt_column_name=None, target_column_name=None, deployment_id=None, model_id=None)

Create a new vector database from validated custom model deployment.

Parameters:

name (str) – Vector database name.
use_case (Optional[Union[UseCase, str]], optional) – Use case to link to the created vector database.
validation_id (Optional[str]) – ID of CustomModelVectorDatabaseValidation for the deployment. Alternatively, you can specify ALL the following fields.
prompt_column_name (Optional[str]) – The column name the deployed model expect as the input.
target_column_name (Optional[str]) – The target name deployed model will output.
deployment_id (Optional[str]) – ID of the deployment.
model_id (Optional[str]) – ID of the underlying deployment model. Can be found from the API as Deployment.model[“id”].

Returns:

vector database – The created vector database.

Return type:

VectorDatabase

classmethod get(vector_database_id)

Retrieve a single vector database.

Parameters:: vector_database_id (str) – The ID of the vector database you want to retrieve.
Returns:: vector database – The requested vector database.
Return type:: VectorDatabase

classmethod list(use_case=None, playground=None, search=None, sort=None, completed_only=None)

List all vector databases associated with a specific use case available to the user.

Parameters:

use_case (Optional[UseCaseLike], optional) – The returned vector databases are filtered to those associated with a specific Use Case or Cases if specified or can be inferred from the Context. Accepts either the entity or the ID.
playground (Optional[Union[Playground, str]], optional) – The returned vector databases are filtered to those associated with a specific playground if it is specified. Accepts either the entity or the ID.
search (Optional[str]) – String for filtering vector databases. Vector databases that contain the string in name will be returned. If not specified, all vector databases will be returned.
sort (Optional[str]) – Property to sort vector databases by. Prefix the attribute name with a dash to sort in descending order, e.g. sort=’-creationDate’. Currently supported options are listed in ListVectorDatabasesSortQueryParams but the values can differ with different platform versions. By default, the sort parameter is None which will result in vector databases being returned in order of creation time descending.
completed_only (Optional[bool]) – A filter to retrieve only vector databases that have been successfully created. By default, all vector databases regardless of execution status are retrieved.

Returns:

vectorbases – A list of vector databases available to the user.

Return type:

list[VectorDatabase]

update(name)

Update the vector database.

Parameters:: name (str) – The new name for the vector database.
Returns:: vector database – The updated vector database.
Return type:: VectorDatabase

delete()

Delete the vector database.

Return type:: None

classmethod get_supported_text_chunkings()

Get all supported text chunking configurations which includes a set of recommended chunking parameters for each supported embedding model.

Returns:: supported_text_chunkings – The supported text chunking configurations.
Return type:: SupportedTextChunkings

download_text_and_embeddings_asset(file_path=None)

Download a parquet file with text chunks and corresponding embeddings created by a vector database.

Parameters:: file_path (Optional[str]) – File path to save the asset. By default, it saves in the current directory autogenerated by server name.
Return type:: None

class datarobot.models.genai.vector_database.SupportedRetrievalSetting

Bases: APIObject

A single supported retrieval setting.

Variables:

name (str) – The name of the setting.
type (str or list[str]) – The type of the setting.
description (str) – The description of the setting.
title (str) – The title of the setting.
default (str, int, bool, or None) – The default value of the setting.
minimum (int or None) – The minimum value of the setting.
maximum (int or None) – The maximum value of the setting.
enum (list[str] or None) – The enum values of the setting.
settings (list[SupportedRetrievalSetting] or None) – The supported retriever settings.
group_id (str or None) – The group ID of the setting.

class datarobot.models.genai.vector_database.VectorDatabaseDatasetExportJob

Bases: APIObject

Response for the vector database dataset export job.

Variables:

job_id (str) – ID of the export job.
vector_database_id (str) – ID of the vector database.
export_dataset_id (str) – ID of the exported dataset.

class datarobot.models.genai.chat_prompt.Citation

Bases: APIObject

Citation for documents retrieved from a vector database.

Variables:

text (str) – The text retrieved from a vector database.
source (str or None, optional) – The source of the retrieved text.
similarity_score – The similarity score between the citation and the user prompt.
metadata (dict or None, optional) – Additional metadata for the citation.

class datarobot.models.genai.llm_blueprint.VectorDatabaseSettings

Bases: APIObject

Settings for a DataRobot GenAI vector database associated with an LLM blueprint.

Variables:

max_documents_retrieved_per_prompt (int or None, optional) – The maximum number of documents to retrieve for each prompt.
max_tokens (int or None, optional) – The maximum number of tokens to retrieve for each document.
retriever (VectorDatabaseRetrievers) – The vector database retriever name.
add_neighbor_chunks – Whether to add neighboring documents to the retrieved documents.

to_dict()

Return type:: VectorDatabaseSettingsDict

class datarobot.models.genai.vector_database.ChunkingParameters

Bases: APIObject

Parameters defining how documents are split and embedded.

Variables:

embedding_model (Optional[str]) – Name of the text embedding model. Currently supported options are listed in VectorDatabaseEmbeddingModel but the values can differ with different platform versions.
chunking_method (str) – Name of the method to split dataset documents. Currently supported options are listed in VectorDatabaseChunkingMethod but the values can differ with different platform versions.
chunk_size (int) – Size of each text chunk in number of tokens.
chunk_overlap_percentage (int) – Overlap percentage between chunks.
separators (list[str]) – Strings used to split documents into text chunks.
embedding_validation (Optional[CustomModelEmbeddingValidation, SupportedCustomModelEmbedding, str]) – ID or object for custom embedding validation.
custom_chunking (bool) – Determines if the chunking is custom. With custom chunking, dataset rows are not split into chunks automatically; instead, the user provides the chunks.