Skip to content

Openai em invoker

Defines a module to interact with OpenAI embedding models.

OpenAIEMInvoker(model_name, api_key=None, base_url=OPENAI_DEFAULT_URL, model_kwargs=None, default_hyperparameters=None, retry_config=None, truncation_config=None, vector_fuser=None)

Bases: BaseEMInvoker

An embedding model invoker to interact with OpenAI embedding models.

Examples:

em_invoker = OpenAIEMInvoker(model_name="text-embedding-3-small")
result = await em_invoker.invoke("Hi there!")
Supported features
  1. Basic invocation
  2. Batch invocation
  3. Text input only
  4. Text truncation
  5. Vector fusion
  6. Retry and timeout
OpenAI compatible endpoints

This class can interact with endpoints that are compatible with OpenAI's Embeddings API schema. This includes but are not limited to: 1. Text Embeddings Inference (https://github.com/huggingface/text-embeddings-inference) 2. vLLM (https://vllm.ai/) To do this, simply set base_url to the endpoint URL. Supported features may vary between endpoints. Unsupported features will result in an error.

Attributes:

Name Type Description
model_id str

The model ID of the embedding model.
model_provider str

The provider of the embedding model.
model_name str

The name of the embedding model.
client_kwargs dict[str, Any]

The keyword arguments for the OpenAI client.
default_hyperparameters dict[str, Any]

Default hyperparameters for invoking the embedding model.
retry_config RetryConfig

The retry configuration for the embedding model.
truncation_config TruncationConfig | None

The truncation configuration for the embedding model.
vector_fuser BaseVectorFuser | None

The vector fuser to handle mixed content.

Initializes a new instance of the OpenAIEMInvoker class.

Parameters:

Name Type Description Default
model_name str

The name of the OpenAI embedding model to be used.

 required
api_key str | None

The API key for authenticating with OpenAI. Defaults to None, in which case the OPENAI_API_KEY environment variable will be used. If the endpoint does not require an API key, a dummy value can be passed (e.g. "").

 None
base_url str

The base URL of a custom endpoint that is compatible with OpenAI's Embeddings API schema. Defaults to OpenAI's default URL.

 OPENAI_DEFAULT_URL
model_kwargs dict[str, Any] | None

Additional keyword arguments for the OpenAI client. Defaults to None.

 None
default_hyperparameters dict[str, Any] | None

Default hyperparameters for invoking the model. Defaults to None.

 None
retry_config RetryConfig | None

The retry configuration for the embedding model. Defaults to None, in which case a default config with no retry and 30.0 seconds timeout will be used.

 None
truncation_config TruncationConfig | None

Configuration for text truncation behavior. Defaults to None, in which case no truncation is applied.

 None
vector_fuser BaseVectorFuser | VectorFuserType | None

The vector fuser to handle mixed content. Defaults to None, in which case handling the mixed modality content depends on the EM's capabilities.

 None