Lm invoker

Defines a base class for language model invokers used in Gen AI applications.

Authors

Henry Wicaksono (henry.wicaksono@gdplabs.id)

References

NONE

BaseLMInvoker(model_id, default_hyperparameters=None, valid_extensions_map=None, tools=None, response_schema=None, output_analytics=False, retry_config=None)

Bases: ABC

A base class for language model invokers used in Gen AI applications.

The BaseLMInvoker class provides a framework for invoking language models with prompts and hyperparameters. It handles both standard and streaming invocation.

Attributes:

Name	Type	Description
model_id	str	The model ID of the language model.
model_provider	str	The provider of the language model.
model_name	str	The name of the language model.
default_hyperparameters	dict[str, Any]	Default hyperparameters for invoking the language model.
tools	list[Tool]	Tools provided to the language model to enable tool calling.
response_schema	ResponseSchema | None	The schema of the response. If provided, the model will output a structured response as defined by the schema. Supports both Pydantic BaseModel and JSON schema dictionary.
output_analytics	bool	Whether to output the invocation analytics.
retry_config	RetryConfig	The retry configuration for the language model.

Initializes a new instance of the BaseLMInvoker class.

Parameters:

Name	Type	Description	Default
model_id	ModelId	The model ID of the language model.	required
default_hyperparameters	dict[str, Any] | None	Default hyperparameters for invoking the language model. Defaults to None, in which case an empty dictionary is used.	None
valid_extensions_map	dict[str, set[str]] | None	A dictionary mapping for validating the content type of the multimodal inputs. They keys are the mime types (e.g. "image") and the values are the set of valid file extensions for the corresponding mime type. Defaults to None, in which case an empty dictionary is used.	None
tools	list[Tool] | None	Tools provided to the language model to enable tool calling. Defaults to None, in which case an empty list is used.	None
response_schema	ResponseSchema | None	The schema of the response. If provided, the model will output a structured response as defined by the schema. Supports both Pydantic BaseModel and JSON schema dictionary. Defaults to None.	None
output_analytics	bool	Whether to output the invocation analytics. Defaults to False.	False
retry_config	RetryConfig | None	The retry configuration for the language model. Defaults to None, in which case a default config with no retry and 30.0 seconds timeout is used.	None

model_id: str property

The model ID of the language model.

Returns:

Name	Type	Description
str	str	The model ID of the language model.

model_name: str property

The name of the language model.

Returns:

Name	Type	Description
str	str	The name of the language model.

model_provider: str property

The provider of the language model.

Returns:

Name	Type	Description
str	str	The provider of the language model.

clear_response_schema()

Clears the response schema for the language model.

This method clears the response schema for the language model by calling the set_response_schema method with None.

clear_tools()

Clears the tools for the language model.

This method clears the tools for the language model by calling the set_tools method with an empty list.

invoke(prompt, hyperparameters=None, event_emitter=None) async

Invokes the language model with the provided prompt and hyperparameters.

This method validates the prompt and invokes the language model with the provided prompt and hyperparameters. It handles both standard and streaming invocation. Streaming mode is enabled if an event emitter is provided. The method includes retry logic with exponential backoff for transient failures.

Parameters:

Name	Type	Description	Default
prompt	MultimodalPrompt | str	The input prompt for the language model.	required
hyperparameters	dict[str, Any] | None	A dictionary of hyperparameters for the language model. Defaults to None, in which case the default hyperparameters are used.	None
event_emitter	EventEmitter | None	The event emitter for streaming tokens. If provided, streaming invocation is enabled. Defaults to None.	None

Returns:

Name	Type	Description
MultimodalOutput	MultimodalOutput	The generated response from the language model.

Raises:

Type	Description
CancelledError	If the invocation is cancelled.
ModelNotFoundError	If the model is not found.
ProviderAuthError	If the model authentication fails.
ProviderInternalError	If the model internal error occurs.
ProviderInvalidArgsError	If the model parameters are invalid.
ProviderOverloadedError	If the model is overloaded.
ProviderRateLimitError	If the model rate limit is exceeded.
TimeoutError	If the invocation times out.
ValueError	If the prompt is not in the correct format.

set_response_schema(response_schema)

Sets the response schema for the language model.

This method sets the response schema for the language model. Any existing response schema will be replaced.

Parameters:

Name	Type	Description	Default
response_schema	ResponseSchema | None	The response schema to be used.	required

set_tools(tools)

Sets the tools for the language model.

This method sets the tools for the language model. Any existing tools will be replaced.

Parameters:

Name	Type	Description	Default
tools	list[Tool]	The list of tools to be used.	required