Openai batch operations

Defines the batch operations for the OpenAILMInvoker class.

OpenAIBatchOperations(invoker)

Bases: BatchOperations

Handles batch operations for the OpenAILMInvoker class.

Examples:

Invoke the language model in batch mode:

results = await lm_invoker.batch.invoke(...)

Standalone batch operations: 1. Create a batch job:

batch_id = await lm_invoker.batch.create(...)

1. Get the status of a batch job:

status = await lm_invoker.batch.status(batch_id)

1. Retrieve the results of a batch job:

results = await lm_invoker.batch.retrieve(batch_id)

1. List the batch jobs:

batch_jobs = await lm_invoker.batch.list()

1. Cancel a batch job:

await lm_invoker.batch.cancel(batch_id)

Initializes the batch operations.

Parameters:

Name	Type	Description	Default
invoker	OpenAILMInvoker	The OpenAILMInvoker to use for the batch operations.	required

cancel(batch_id) async

Cancels a batch job.

Parameters:

Name	Type	Description	Default
batch_id	str	The ID of the batch job to cancel.	required

create(requests, hyperparameters=None) async

Creates a new batch job.

Parameters:

Name	Type	Description	Default
requests	dict[str, LMInput]	The dictionary of requests that maps request ID to the request. Each request must be a valid input for the language model. 1. If the request is a list of Message objects, it is used as is. 2. If the request is a list of MessageContent or a string, it is converted into a user message.	required
hyperparameters	dict[str, Any] | None	A dictionary of hyperparameters for the language model. Defaults to None, in which case the default hyperparameters are used.	None

Returns:

Name	Type	Description
str	str	The ID of the batch job.

list() async

Lists the batch jobs.

Returns:

Type	Description
list[dict[str, Any]]	list[dict[str, Any]]: The list of batch jobs.

retrieve(batch_id, **kwargs) async

Retrieves the results of a batch job.

Parameters:

Name	Type	Description	Default
batch_id	str	The ID of the batch job to get the results of.	required
**kwargs	Any	Additional keyword arguments.	{}

Returns:

Type	Description
dict[str, LMOutput]	dict[str, LMOutput]: The results of the batch job.

status(batch_id) async

Gets the status of a batch job.

Parameters:

Name	Type	Description	Default
batch_id	str	The ID of the batch job to get the status of.	required

Returns:

Name	Type	Description
BatchStatus	BatchStatus	The status of the batch job.