Em Invoker

Modules concerning the embedding model invokers used in Gen AI applications.

AzureOpenAIEMInvoker(azure_endpoint, azure_deployment, api_key=None, model_kwargs=None, default_hyperparameters=None, retry_config=None, truncation_config=None, vector_fuser=None)

Bases: OpenAIEMInvoker

An embedding model invoker to interact with Azure OpenAI embedding models.

Examples:

em_invoker = AzureOpenAIEMInvoker(
    azure_endpoint="https://<your-azure-openai-endpoint>.openai.azure.com/openai/v1",
    azure_deployment="<your-azure-openai-deployment>",
)
result = await em_invoker.invoke("Hi there!")

Supported features

1. Basic invocation
2. Batch invocation
3. Text input only
4. Text truncation
5. Vector fusion
6. Retry and timeout

Attributes:

Name	Type	Description
model_id	str	The model ID of the embedding model.
model_provider	str	The provider of the embedding model.
model_name	str	The name of the Azure OpenAI embedding model deployment.
client_kwargs	dict[str, Any]	The keyword arguments for the Azure OpenAI client.
default_hyperparameters	dict[str, Any]	Default hyperparameters for invoking the embedding model.
retry_config	RetryConfig	The retry configuration for the embedding model.
truncation_config	TruncationConfig | None	The truncation configuration for the embedding model.
vector_fuser	BaseVectorFuser | None	The vector fuser to handle mixed content.

Initializes a new instance of the AzureOpenAIEMInvoker class.

Parameters:

Name	Type	Description	Default
azure_endpoint	str	The endpoint of the Azure OpenAI service.	required
azure_deployment	str	The deployment name of the Azure OpenAI service.	required
api_key	str | None	The API key for authenticating with Azure OpenAI. Defaults to None, in which case the AZURE_OPENAI_API_KEY environment variable will be used.	None
model_kwargs	dict[str, Any] | None	Additional model parameters. Defaults to None.	None
default_hyperparameters	dict[str, Any] | None	Default hyperparameters for invoking the model. Defaults to None.	None
retry_config	RetryConfig | None	The retry configuration for the embedding model. Defaults to None, in which case a default config with no retry and 30.0 seconds timeout will be used.	None
truncation_config	TruncationConfig | None	Configuration for text truncation behavior. Defaults to None, in which case no truncation is applied.	None
vector_fuser	BaseVectorFuser | VectorFuserType | None	The vector fuser to handle mixed content. Defaults to None, in which case handling the mixed modality content depends on the EM's capabilities.	None

BedrockEMInvoker(model_name, access_key_id=None, secret_access_key=None, region_name='us-east-1', model_kwargs=None, default_hyperparameters=None, retry_config=None, truncation_config=None, vector_fuser=None)

Bases: BaseEMInvoker

An embedding model invoker to interact with AWS Bedrock embedding models.

Examples:

em_invoker = BedrockEMInvoker(model_name="cohere.embed-english-v3")
result = await em_invoker.invoke("Hi there!")

Supported features

1. Basic invocation
2. Batch invocation
3. Multimodal input a. Text b. Image (For Marengo models only)
4. Text truncation
5. Vector fusion
6. Retry and timeout

Attributes:

Name	Type	Description
model_id	str	The model ID of the embedding model.
model_provider	str	The provider of the embedding model.
model_name	str	The name of the embedding model.
session	Session	The Bedrock client session.
client_kwargs	dict[str, Any]	The Bedrock client kwargs.
default_hyperparameters	dict[str, Any]	Default hyperparameters for invoking the embedding model.
retry_config	RetryConfig	The retry configuration for the embedding model.
truncation_config	TruncationConfig | None	The truncation configuration for the embedding model.
vector_fuser	BaseVectorFuser | None	The vector fuser to handle mixed content.

Initializes a new instance of the BedrockEMInvoker class.

Parameters:

Name	Type	Description	Default
model_name	str	The name of the Bedrock embedding model to be used.	required
access_key_id	str | None	The AWS access key ID. Defaults to None, in which case the AWS_ACCESS_KEY_ID environment variable will be used.	None
secret_access_key	str | None	The AWS secret access key. Defaults to None, in which case the AWS_SECRET_ACCESS_KEY environment variable will be used.	None
region_name	str	The AWS region name. Defaults to "us-east-1".	'us-east-1'
model_kwargs	dict[str, Any] | None	Additional keyword arguments for the Bedrock client. Defaults to None.	None
default_hyperparameters	dict[str, Any] | None	Default hyperparameters for invoking the model. Defaults to None.	None
retry_config	RetryConfig | None	The retry configuration for the embedding model. Defaults to None, in which case a default config with no retry and 30.0 seconds timeout will be used.	None
truncation_config	TruncationConfig | None	Configuration for text truncation behavior. Defaults to None, in which case no truncation is applied.	None
vector_fuser	BaseVectorFuser | VectorFuserType | None	The vector fuser to handle mixed content. Defaults to None, in which case handling the mixed modality content depends on the EM's capabilities.	None

Raises:

Type	Description
ValueError	If the model name is not supported.
ValueError	If access_key_id or secret_access_key is neither provided nor set in the AWS_ACCESS_KEY_ID or AWS_SECRET_ACCESS_KEY environment variables, respectively.

CohereEMInvoker(model_name, api_key=None, base_url=None, model_kwargs=None, default_hyperparameters=None, retry_config=None, truncation_config=None, vector_fuser=None, input_type=CohereInputType.SEARCH_DOCUMENT)

Bases: BaseEMInvoker

An embedding model invoker to interact with Cohere embedding models.

Examples:

em_invoker = CohereEMInvoker(model_name="embed-english-v4.0")
result = await em_invoker.invoke("Hi there!")

Supported features

1. Basic invocation
2. Batch invocation
3. Multimodal input a. Text b. Image
4. Text truncation
5. Vector fusion
6. Retry and timeout

Attributes:

Name	Type	Description
model_id	str	The model ID of the embedding model.
model_provider	str	The provider of the embedding model (Cohere).
model_name	str	The name of the Cohere embedding model.
client	AsyncClient	The asynchronous client for the Cohere API.
default_hyperparameters	dict[str, Any]	Default hyperparameters for invoking the embedding model.
retry_config	RetryConfig	The retry configuration for the embedding model.
truncation_config	TruncationConfig | None	The truncation configuration for the embedding model.
vector_fuser	BaseVectorFuser | None	The vector fuser to handle mixed content.
input_type	CohereInputType	The input type for the embedding model. Supported values include: 1. CohereInputType.SEARCH_DOCUMENT, 2. CohereInputType.SEARCH_QUERY, 3. CohereInputType.CLASSIFICATION, 4. CohereInputType.CLUSTERING, 5. CohereInputType.IMAGE.

Initializes a new instance of the CohereEMInvoker class.

Parameters:

Name	Type	Description	Default
model_name	str	The name of the Cohere embedding model to be used.	required
api_key	str | None	The API key for authenticating with Cohere. Defaults to None, in which case the COHERE_API_KEY environment variable will be used.	None
base_url	str | None	The base URL for a custom Cohere-compatible endpoint. Defaults to None, in which case Cohere's default URL will be used.	None
model_kwargs	dict[str, Any] | None	Additional keyword arguments for the Cohere client. Defaults to None.	None
default_hyperparameters	dict[str, Any] | None	Default hyperparameters for invoking the model. Defaults to None.	None
retry_config	RetryConfig | None	The retry configuration for the embedding model. Defaults to None, in which case a default config with no retry and 30.0 seconds timeout will be used.	None
truncation_config	TruncationConfig | None	Configuration for text truncation behavior. Defaults to None, in which case no truncation is applied.	None
vector_fuser	BaseVectorFuser | VectorFuserType | None	The vector fuser to handle mixed content. Defaults to None, in which case handling the mixed modality content depends on the EM's capabilities.	None
input_type	CohereInputType	The input type for the embedding model. Defaults to CohereInputType.SEARCH_DOCUMENT. Valid values are: "search_document", "search_query", "classification", "clustering", and "image". For more information about, please refer to https://docs.cohere.com/docs/embeddings#the-input_type-parameter.	SEARCH_DOCUMENT

GoogleEMInvoker(model_name, api_key=None, credentials_path=None, project_id=None, location='us-central1', model_kwargs=None, default_hyperparameters=None, retry_config=None, truncation_config=None, vector_fuser=None)

Bases: BaseEMInvoker

An embedding model invoker to interact with Google embedding models.

Examples:

em_invoker = GoogleEMInvoker(model_name="gemini-embedding-001")
result = await em_invoker.invoke("Hi there!")

Initialization

The GoogleEMInvoker can use either Google Gen AI or Google Vertex AI.

Google Gen AI is recommended for quick prototyping and development. It requires a Gemini API key for authentication.

Usage example:

em_invoker = GoogleEMInvoker(
    model_name="gemini-embedding-001",
    api_key="your_api_key"
)

Google Vertex AI is recommended to build production-ready applications. It requires a service account JSON file for authentication.

Usage example:

em_invoker = GoogleEMInvoker(
    model_name="gemini-embedding-001",
    credentials_path="path/to/service_account.json"
)

If neither api_key nor credentials_path is provided, Google Gen AI will be used by default. The GOOGLE_API_KEY environment variable will be used for authentication.

Supported features

1. Basic invocation
2. Batch invocation
3. Multimodal input a. Text b. Image c. Audio d. Video e. Document
4. Text truncation
5. Vector fusion
6. Retry and timeout

Attributes:

Name	Type	Description
model_id	str	The model ID of the embedding model.
model_provider	str	The provider of the embedding model.
model_name	str	The name of the embedding model.
client_params	dict[str, Any]	The Google client instance init parameters.
default_hyperparameters	dict[str, Any]	Default hyperparameters for invoking the embedding model.
retry_config	RetryConfig	The retry configuration for the embedding model.
truncation_config	TruncationConfig | None	The truncation configuration for the embedding model.
vector_fuser	BaseVectorFuser | None	The vector fuser to handle mixed content.

Initializes a new instance of the GoogleEMInvoker class.

Parameters:

Name	Type	Description	Default
model_name	str	The name of the model to use.	required
api_key	str | None	Required for Google Gen AI authentication. Cannot be used together with credentials_path. Defaults to None.	None
credentials_path	str | None	Required for Google Vertex AI authentication. Path to the service account credentials JSON file. Cannot be used together with api_key. Defaults to None.	None
project_id	str | None	The Google Cloud project ID for Vertex AI. Only used when authenticating with credentials_path. Defaults to None, in which case it will be loaded from the credentials file.	None
location	str	The location of the Google Cloud project for Vertex AI. Only used when authenticating with credentials_path. Defaults to "us-central1".	'us-central1'
model_kwargs	dict[str, Any] | None	Additional keyword arguments for the Google client. Defaults to None.	None
default_hyperparameters	dict[str, Any] | None	Default hyperparameters for invoking the model. Defaults to None.	None
retry_config	RetryConfig | None	The retry configuration for the embedding model. Defaults to None, in which case a default config with no retry and 30.0 seconds timeout will be used.	None
vector_fuser	BaseVectorFuser | VectorFuserType | None	The vector fuser to handle mixed content. Defaults to None, in which case handling the mixed modality content depends on the EM's capabilities.	None
truncation_config	TruncationConfig | None	Configuration for text truncation behavior. Defaults to None, in which case no truncation is applied.	None

Note

If neither api_key nor credentials_path is provided, Google Gen AI will be used by default. The GOOGLE_API_KEY environment variable will be used for authentication.

JinaEMInvoker(model_name, api_key=None, base_url=JINA_DEFAULT_URL, model_kwargs=None, default_hyperparameters=None, retry_config=None, truncation_config=None, vector_fuser=None)

Bases: BaseEMInvoker

An embedding model invoker to interact with Jina AI embedding models.

Examples:

em_invoker = JinaEMInvoker(model_name="jina-embeddings-v2-large")
result = await em_invoker.invoke("Hi there!")

Supported features

1. Basic invocation
2. Batch invocation
3. Multimodal input a. Text b. Image
4. Text truncation
5. Vector fusion
6. Retry and timeout

Attributes:

Name	Type	Description
model_id	str	The model ID of the embedding model.
model_provider	str	The provider of the embedding model.
model_name	str	The name of the embedding model.
client	AsyncClient	The client for the Jina AI API.
default_hyperparameters	dict[str, Any]	Default hyperparameters for invoking the embedding model.
retry_config	RetryConfig	The retry configuration for the embedding model.
truncation_config	TruncationConfig | None	The truncation configuration for the embedding model.
vector_fuser	BaseVectorFuser | None	The vector fuser to handle mixed content.

Initializes a new instance of the JinaEMInvoker class.

Parameters:

Name	Type	Description	Default
model_name	str	The name of the Jina embedding model to be used.	required
api_key	str | None	The API key for authenticating with Jina AI. Defaults to None, in which case the JINA_API_KEY environment variable will be used.	None
base_url	str	The base URL for the Jina AI API. Defaults to "https://api.jina.ai/v1".	JINA_DEFAULT_URL
model_kwargs	dict[str, Any] | None	Additional keyword arguments for the HTTP client. Defaults to None.	None
default_hyperparameters	dict[str, Any] | None	Default hyperparameters for invoking the model. Defaults to None.	None
retry_config	RetryConfig | None	The retry configuration for the embedding model. Defaults to None, in which case a default config with no retry and 30.0 seconds timeout will be used.	None
truncation_config	TruncationConfig | None	Configuration for text truncation behavior. Defaults to None, in which case no truncation is applied.	None
vector_fuser	BaseVectorFuser | VectorFuserType | None	The vector fuser to handle mixed content. Defaults to None, in which case handling the mixed modality content depends on the EM's capabilities.	None

Raises:

Type	Description
ValueError	If neither api_key nor JINA_API_KEY environment variable is provided.

LangChainEMInvoker(model=None, model_class_path=None, model_name=None, model_kwargs=None, default_hyperparameters=None, retry_config=None, truncation_config=None, vector_fuser=None)

Bases: BaseEMInvoker

An embedding model invoker to interact with LangChain's Embeddings.

Examples:

em_invoker = LangChainEMInvoker(
    model_class_path="langchain_openai.OpenAIEmbeddings",
    model_name="text-embedding-3-small",
)
result = await em_invoker.invoke("Hi there!")

Initialization

The LangChainEMInvoker can be initialized by either passing:

1. A LangChain's Embeddings instance: Usage example:

from langchain_openai import OpenAIEmbeddings

model = OpenAIEmbeddings(model="text-embedding-3-small", api_key="your_api_key")
em_invoker = LangChainEMInvoker(model=model)

1. A model path in the format of ".": Usage example:

em_invoker = LangChainEMInvoker(
    model_class_path="langchain_openai.OpenAIEmbeddings",
    model_name="text-embedding-3-small",
    model_kwargs={"api_key": "your_api_key"}
)

For the list of supported providers, please refer to the following table: https://docs.langchain.com/oss/python/integrations/providers/overview#featured-providers

Supported features

1. Basic invocation
2. Batch invocation
3. Text input only
4. Text truncation
5. Vector fusion
6. Retry and timeout

Attributes:

Name	Type	Description
model_id	str	The model ID of the embedding model.
model_provider	str	The provider of the embedding model.
model_name	str	The name of the embedding model.
em	Embeddings	The instance to interact with an embedding model defined using LangChain's Embeddings.
retry_config	RetryConfig	The retry configuration for the embedding model.
truncation_config	TruncationConfig | None	The truncation configuration for the embedding model.
vector_fuser	BaseVectorFuser | None	The vector fuser to handle mixed content.

Initializes a new instance of the LangChainEMInvoker class.

Parameters:

Name	Type	Description	Default
model	Embeddings | None	The LangChain's Embeddings instance. If provided, will take precedence over the model_class_path parameter. Defaults to None.	None
model_class_path	str | None	The LangChain's Embeddings class path. Must be formatted as "." (e.g. "langchain_openai.OpenAIEmbeddings"). Ignored if model is provided. Defaults to None.	None
model_name	str | None	The model name. Only used if model_class_path is provided. Defaults to None.	None
model_kwargs	dict[str, Any] | None	The additional keyword arguments. Only used if model_class_path is provided. Defaults to None.	None
default_hyperparameters	dict[str, Any] | None	Default hyperparameters for invoking the model. Defaults to None.	None
retry_config	RetryConfig | None	The retry configuration for the embedding model. Defaults to None, in which case a default config with no retry and 30.0 seconds timeout will be used.	None
truncation_config	TruncationConfig | None	Configuration for text truncation behavior. Defaults to None, in which case no truncation is applied.	None
vector_fuser	BaseVectorFuser | VectorFuserType | None	The vector fuser to handle mixed content. Defaults to None, in which case handling the mixed modality content depends on the EM's capabilities.	None

OpenAIEMInvoker(model_name, api_key=None, base_url=OPENAI_DEFAULT_URL, model_kwargs=None, default_hyperparameters=None, retry_config=None, truncation_config=None, vector_fuser=None)

Bases: BaseEMInvoker

An embedding model invoker to interact with OpenAI embedding models.

Examples:

em_invoker = OpenAIEMInvoker(model_name="text-embedding-3-small")
result = await em_invoker.invoke("Hi there!")

Supported features

1. Basic invocation
2. Batch invocation
3. Text input only
4. Text truncation
5. Vector fusion
6. Retry and timeout

OpenAI compatible endpoints

This class can interact with endpoints that are compatible with OpenAI's Embeddings API schema. This includes but are not limited to: 1. Text Embeddings Inference (https://github.com/huggingface/text-embeddings-inference) 2. vLLM (https://vllm.ai/) To do this, simply set base_url to the endpoint URL. Supported features may vary between endpoints. Unsupported features will result in an error.

Attributes:

Name	Type	Description
model_id	str	The model ID of the embedding model.
model_provider	str	The provider of the embedding model.
model_name	str	The name of the embedding model.
client_kwargs	dict[str, Any]	The keyword arguments for the OpenAI client.
default_hyperparameters	dict[str, Any]	Default hyperparameters for invoking the embedding model.
retry_config	RetryConfig	The retry configuration for the embedding model.
truncation_config	TruncationConfig | None	The truncation configuration for the embedding model.
vector_fuser	BaseVectorFuser | None	The vector fuser to handle mixed content.

Initializes a new instance of the OpenAIEMInvoker class.

Parameters:

Name	Type	Description	Default
model_name	str	The name of the OpenAI embedding model to be used.	required
api_key	str | None	The API key for authenticating with OpenAI. Defaults to None, in which case the OPENAI_API_KEY environment variable will be used. If the endpoint does not require an API key, a dummy value can be passed (e.g. "").	None
base_url	str	The base URL of a custom endpoint that is compatible with OpenAI's Embeddings API schema. Defaults to OpenAI's default URL.	OPENAI_DEFAULT_URL
model_kwargs	dict[str, Any] | None	Additional keyword arguments for the OpenAI client. Defaults to None.	None
default_hyperparameters	dict[str, Any] | None	Default hyperparameters for invoking the model. Defaults to None.	None
retry_config	RetryConfig | None	The retry configuration for the embedding model. Defaults to None, in which case a default config with no retry and 30.0 seconds timeout will be used.	None
truncation_config	TruncationConfig | None	Configuration for text truncation behavior. Defaults to None, in which case no truncation is applied.	None
vector_fuser	BaseVectorFuser | VectorFuserType | None	The vector fuser to handle mixed content. Defaults to None, in which case handling the mixed modality content depends on the EM's capabilities.	None

TwelveLabsEMInvoker(model_name, api_key=None, model_kwargs=None, default_hyperparameters=None, retry_config=None, truncation_config=None, vector_fuser=None)

Bases: BaseEMInvoker

An embedding model invoker to interact with TwelveLabs embedding models.

Examples:

em_invoker = TwelveLabsEMInvoker(model_name="Marengo-retrieval-2.7")
result = await em_invoker.invoke("Hi there!")

Supported features

1. Basic invocation
2. Batch invocation
3. Multimodal input a. Text b. Audio c. Image d. Video
4. Text truncation
5. Vector fusion
6. Retry and timeout

Video embedding

For advanced video embedding customization, you can pass hyperparameters to control the embedding process:

hyperparameters={
    "video_embed_create_params": {}, # optional parameters for EmbedClient.TasksClient.create(),
    "video_embed_retrieve_params": {}, # optional parameters for EmbedClient.TasksClient.retrieve()
    "wait_for_done_params": {}, # optional parameters for EmbedClient.TasksClient.wait_for_done()
}

video = Attachment.from_path("path/to/local/video.mp4")
result = await em_invoker.invoke(video, hyperparameters)

For video related hyperparameters, see https://docs.twelvelabs.io/sdk-reference/python/create-embeddings-v-1/create-video-embeddings.

Furthermore, when embedding video: 1. The video format should meet the requirements of TwelveLabs embedding model. For more details, see Marengo requirements 2. VectorFuser is required to embed video content.

Attributes:

Name	Type	Description
model_id	str	The model ID of the embedding model.
model_provider	str	The provider of the embedding model.
model_name	str	The name of the embedding model.
client	Client	The client for the TwelveLabs API.
default_hyperparameters	dict[str, Any]	Default hyperparameters for invoking the embedding model.
retry_config	RetryConfig	The retry configuration for the embedding model.
truncation_config	TruncationConfig | None	The truncation configuration for the embedding model.
vector_fuser	BaseVectorFuser | None	The vector fuser to handle mixed content.

Initializes a new instance of the TwelveLabsEMInvoker class.

Parameters:

Name	Type	Description	Default
model_name	str	The name of the TwelveLabs embedding model to be used.	required
api_key	str | None	The API key for the TwelveLabs API. Defaults to None, in which case the TWELVELABS_API_KEY environment variable will be used.	None
model_kwargs	dict[str, Any] | None	Additional keyword arguments for the TwelveLabs client. Defaults to None.	None
default_hyperparameters	dict[str, Any] | None	Default hyperparameters for invoking the model. Defaults to None.	None
retry_config	RetryConfig | None	The retry configuration for the embedding model. Defaults to None, in which case a default config with no retry and 30.0 seconds timeout will be used.	None
truncation_config	TruncationConfig | None	Configuration for text truncation behavior. Defaults to None, in which case no truncation is applied.	None
vector_fuser	BaseVectorFuser | VectorFuserType | None	The vector fuser to handle mixed content. Defaults to None, in which case handling the mixed modality content depends on the EM's capabilities.	None

VoyageEMInvoker(model_name, api_key=None, model_kwargs=None, default_hyperparameters=None, retry_config=None, truncation_config=None, vector_fuser=None)

Bases: BaseEMInvoker

An embedding model invoker to interact with Voyage embedding models.

Examples:

em_invoker = VoyageEMInvoker(model_name="voyage-3.5-lite")
result = await em_invoker.invoke("Hi there!")

Supported features

1. Basic invocation
2. Batch invocation
3. Multimodal input a. Text b. Image
4. Text truncation
5. Vector fusion
6. Retry and timeout

Attributes:

Name	Type	Description
model_id	str	The model ID of the embedding model.
model_provider	str	The provider of the embedding model.
model_name	str	The name of the embedding model.
client	Client	The client for the Voyage API.
default_hyperparameters	dict[str, Any]	Default hyperparameters for invoking the embedding model.
retry_config	RetryConfig	The retry configuration for the embedding model.
truncation_config	TruncationConfig | None	The truncation configuration for the embedding model.
vector_fuser	BaseVectorFuser | None	The vector fuser to handle mixed content.

Initializes a new instance of the VoyageEMInvoker class.

Parameters:

Name	Type	Description	Default
model_name	str	The name of the Voyage embedding model to be used.	required
api_key	str | None	The API key for the Voyage API. Defaults to None, in which case the VOYAGE_API_KEY environment variable will be used.	None
model_kwargs	dict[str, Any] | None	Additional keyword arguments for the Voyage client. Defaults to None.	None
default_hyperparameters	dict[str, Any] | None	Default hyperparameters for invoking the model. Defaults to None.	None
retry_config	RetryConfig | None	The retry configuration for the embedding model. Defaults to None, in which case a default config with no retry and 30.0 seconds timeout will be used.	None
truncation_config	TruncationConfig | None	Configuration for text truncation behavior. Defaults to None, in which case no truncation is applied.	None
vector_fuser	BaseVectorFuser | VectorFuserType | None	The vector fuser to handle mixed content. Defaults to None, in which case handling the mixed modality content depends on the EM's capabilities.	None