Repacker

Module for repacking chunks using various strategies.

Authors

Dimitrij Ray (dimitrij.ray@gdplabs.id)

References

NONE

RepackMethod

Bases: Enum

Enum representing the repack methods.

Repacker(method='forward', mode='chunk', delimiter='\n\n', size_func=lambda chunk: len(chunk.content), size_limit=None)

Bases: Component

A class for repacking chunks using various strategies.

Attributes:

Name	Type	Description
method	RepackMethod	The method used for repacking.
mode	RepackerMode	The mode of operation (chunk or context).
delimiter	str	The delimiter used in context mode.
size_func	Callable[[Chunk], int]	Function used to measure the size of chunks.
size_limit	int | None	The maximum allowed total size for the repacked chunks.

Initialize the Repacker instance.

Parameters:

Name	Type	Description	Default
method	str	The method used for repacking. Defaults to "forward".	'forward'
mode	str	The mode of operation (chunk or context). Defaults to "chunk".	'chunk'
delimiter	str	The delimiter used in context mode. Defaults to "\n\n".	'\n\n'
size_func	Callable[[Chunk], int]	Function used to measure the size of a single chunk. Defaults to the length of the chunk content.	lambda chunk: len(content)
size_limit	int | None	The maximum allowed total size for the repacked chunks. Defaults to None. Note: The size limit only accounts for the chunks, not including the delimiter in context mode.	None

repack(chunks) async

Repack the input chunks based on the chosen method and mode.

Parameters:

Name	Type	Description	Default
chunks	list[Chunk]	The input chunks to repack.	required

Returns:

Type	Description
list[Chunk] | str	list[Chunk] | str: The repacked chunks as a list (in chunk mode) or a string (in context mode).

RepackerMode

Bases: Enum

Enum representing the repacker modes.