Skip to content

Multimodal Prompt Builder

Modules concerning the prompt builders used in multimodal Gen AI applications.

WARNING: This module is deprecated. Please use gllm_inference.prompt_builder instead. This module will be removed in version 0.5.0.

MultimodalPromptBuilder(system_template='', user_template='')

A prompt builder that is compatible with multimodal language models.

Attributes:

Name Type Description
system_template str

The system prompt template. May contain placeholders enclosed in curly braces {}.

user_template str

The user prompt template. May contain placeholders enclosed in curly braces {}.

prompt_key_set set[str]

A set of expected keys that must be present in the prompt templates.

WARNING: This module is deprecated. Please use gllm_inference.prompt_builder instead. This module will be removed in version 0.5.0.

Initializes a new instance of the MultimodalPromptBuilder class.

Parameters:

Name Type Description Default
system_template str

The system prompt template. May contain placeholders enclosed in curly braces {}. Defaults to an empty string.

''
user_template str

The user prompt template. May contain placeholders enclosed in curly braces {}. Defaults to an empty string.

''

Raises:

Type Description
ValueError

If both system_template and user_template are empty.

format_as_message_list(history=None, system_multimodal_contents=None, user_multimodal_contents=None, **kwargs)

Formats the prompt templates as a list of message tuples (role, formatted list of contents).

This method processes each prompt template, replacing the placeholders in the template content with the corresponding values from kwargs. If a required key is missing from kwargs, it raises a ValueError. It returns a list of tuples, where each tuple consists of a role and the corresponding formatted list of contents, which may contain multimodal inputs.

Parameters:

Name Type Description Default
history list[tuple[PromptRole, list[Any]]] | None

The optional chat history to be included in the prompt. Defaults to None.

None
system_multimodal_contents list[Any] | None

The optional multimodal contents for the system prompt. Defaults to None.

None
user_multimodal_contents list[Any] | None

The optional multimodal contents for the user prompt. Defaults to None.

None
**kwargs Any

A dictionary of placeholder values to be injected into the prompt templates.

{}

Returns:

Type Description
list[tuple[PromptRole, list[Any]]]

list[tuple[PromptRole, list[Any]]]: A list of tuples, each containing a role and the corresponding formatted list of contents, which may contain multimodal inputs.

Raises:

Type Description
ValueError

If a required key for the prompt template is missing from kwargs.