Openai em invoker

Defines a module to interact with OpenAI embedding models.

Authors

Henry Wicaksono (henry.wicaksono@gdplabs.id)

References

[1] https://platform.openai.com/docs/api-reference/embeddings/create

`OpenAIEMInvoker(model_name, api_key=None, base_url=OPENAI_DEFAULT_URL, model_kwargs=None, default_hyperparameters=None, retry_config=None, truncation_config=None, vector_fuser=None)`

Bases: BaseEMInvoker

An embedding model invoker to interact with OpenAI embedding models.

Attributes:

Name	Type	Description
`model_id`	`str`	The model ID of the embedding model.
`model_provider`	`str`	The provider of the embedding model.
`model_name`	`str`	The name of the embedding model.
`client_kwargs`	`dict[str, Any]`	The keyword arguments for the OpenAI client.
`default_hyperparameters`	`dict[str, Any]`	Default hyperparameters for invoking the embedding model.
`retry_config`	`RetryConfig`	The retry configuration for the embedding model.
`truncation_config`	`TruncationConfig \| None`	The truncation configuration for the embedding model.
`vector_fuser`	`BaseVectorFuser \| None`	The vector fuser to handle mixed content.

Basic usage

The OpenAIEMInvoker can be used as follows:

em_invoker = OpenAIEMInvoker(model_name="text-embedding-3-small")
result = await em_invoker.invoke("Hi there!")

OpenAI compatible endpoints

The OpenAIEMInvoker can also be used to interact with endpoints that are compatible with OpenAI's Embeddings API schema. This includes but are not limited to: 1. Text Embeddings Inference (https://github.com/huggingface/text-embeddings-inference) 2. vLLM (https://vllm.ai/) Please note that the supported features and capabilities may vary between different endpoints and language models. Using features that are not supported by the endpoint will result in an error.

This customization can be done by setting the base_url parameter to the base URL of the endpoint:

em_invoker = OpenAIEMInvoker(
    model_name="<model-name>",
    api_key="<your-api-key>",
    base_url="<https://base-url>",
)
result = await em_invoker.invoke("Hi there!")

Input types

The OpenAIEMInvoker only supports text inputs.

Output format

The OpenAIEMInvoker can embed either: 1. A single content. 1. A single content is a single text. 2. The output will be a Vector, representing the embedding of the content.

# Example 1: Embedding a text content. python text = "This is a text" result = await em_invoker.invoke(text)

The above examples will return a Vector with a size of (embedding_size,).

A list of contents.
A list of contents is a list of texts.
The output will be a list[Vector], where each element is a Vector representing the embedding of each single content.

# Example: Embedding a list of contents. python text1 = "This is a text" text2 = "This is another text" text3 = "This is yet another text" result = await em_invoker.invoke([text1, text2, text3])

The above examples will return a list[Vector] with a size of (3, embedding_size).

Vector fusion

The OpenAIEMInvoker supports vector fusion, which allows fusing multiple results into a single vector. This feature allows the module to embed mixed modality contents, represented as tuples of contents. This feature can be enabled by providing a vector fuser to the vector_fuser parameter.

Usage example:

em_invoker = OpenAIEMInvoker(..., vector_fuser=SumVectorFuser())  # Using a vector fuser class
em_invoker = OpenAIEMInvoker(..., vector_fuser="sum")  # Using a vector fuser type

text = "What animal is in this image?"
image = Attachment.from_path("path/to/local/image.png")
mix_content = (text, image)
result = await em_invoker.invoke([text, image, mix_content])

The above example will return a list[Vector] with a size of (3, embedding_size).

Retry and timeout

The OpenAIEMInvoker supports retry and timeout configuration. By default, the max retries is set to 0 and the timeout is set to 30.0 seconds. They can be customized by providing a custom RetryConfig object to the retry_config parameter.

Retry config examples:

retry_config = RetryConfig(max_retries=0, timeout=None)  # No retry, no timeout
retry_config = RetryConfig(max_retries=5, timeout=10.0)  # 5 max retries, 10.0 seconds timeout

Usage example:

em_invoker = OpenAIEMInvoker(..., retry_config=retry_config)

Initializes a new instance of the OpenAIEMInvoker class.

Parameters:

Name	Type	Description	Default
`model_name`	`str`	The name of the OpenAI embedding model to be used.	required
`api_key`	`str \| None`	The API key for authenticating with OpenAI. Defaults to None, in which case the `OPENAI_API_KEY` environment variable will be used. If the endpoint does not require an API key, a dummy value can be passed (e.g. "").	`None`
`base_url`	`str`	The base URL of a custom endpoint that is compatible with OpenAI's Embeddings API schema. Defaults to OpenAI's default URL.	`OPENAI_DEFAULT_URL`
`model_kwargs`	`dict[str, Any] \| None`	Additional keyword arguments for the OpenAI client. Defaults to None.	`None`
`default_hyperparameters`	`dict[str, Any] \| None`	Default hyperparameters for invoking the model. Defaults to None.	`None`
`retry_config`	`RetryConfig \| None`	The retry configuration for the embedding model. Defaults to None, in which case a default config with no retry and 30.0 seconds timeout will be used.	`None`
`truncation_config`	`TruncationConfig \| None`	Configuration for text truncation behavior. Defaults to None, in which case no truncation is applied.	`None`
`vector_fuser`	`BaseVectorFuser \| VectorFuserType \| None`	The vector fuser to handle mixed content. Defaults to None, in which case handling the mixed modality content depends on the EM's capabilities.	`None`