Fulltext

Redis implementation of fulltext search and CRUD capability.

This module provides a Redis implementation of the FulltextCapability protocol using Redis's native data structures and search capabilities.

Authors

Kadek Denaya (kadek.d.r.diana@gdplabs.id)

References

NONE

DefaultBatchSize

Default batch sizes for Redis operations.

RedisFulltextCapability(index_name, client)

Redis implementation of FulltextCapability protocol.

Attributes:

Name	Type	Description
index_name	str	Name of the Redis index.
client	Redis	Redis client instance.

Initialize the Redis fulltext capability.

Schema will be automatically inferred from chunks when creating a new index, or auto-detected from an existing index when performing operations.

Parameters:

Name	Type	Description	Default
index_name	str	Name of the Redis index.	required
client	Redis	Redis client instance.	required

clear() async

Clear all records from the datastore.

create(data) async

Create new records in the datastore.

If the index does not exist and no filterable_fields were provided, the schema will be inferred from the chunks being created.

Examples:

Create a new chunk.

await fulltext_capability.create(Chunk(content="Test chunk", metadata={"category": "test"}))

Parameters:

Name	Type	Description	Default
data	Chunk | list[Chunk]	Data to create (single item or collection).	required

Raises:

Type	Description
ValueError	If data structure is invalid or chunk content is invalid.

delete(filters=None, options=None) async

Delete records from the datastore.

Processes deletions in batches to avoid loading all matching documents into memory. For delete operations, only document IDs are retrieved (not full content) to minimize memory usage.

Parameters:

Name	Type	Description	Default
filters	FilterClause | QueryFilter | None	Filters to select records to delete. Defaults to None.	None
options	QueryOptions | None	Query options for sorting and limiting deletions (for eviction-like operations). Defaults to None.	None

Raises:

Type	Description
Exception	If Redis operations fail.

retrieve(filters=None, options=None) async

Read records from the datastore with optional filtering.

Parameters:

Name	Type	Description	Default
filters	FilterClause | QueryFilter | None	Query filters to apply. Defaults to None.	None
options	QueryOptions | None	Query options for sorting and pagination. Defaults to None, in which case the default limit of 10 is used.	None

Returns:

Type	Description
list[Chunk]	list[Chunk]: List of matched chunks after applying filters and options.

retrieve_fuzzy(query, max_distance=2, filters=None, options=None) async

Find records that fuzzy match the query within distance threshold.

Parameters:

Name	Type	Description	Default
query	str	Text to fuzzy match against.	required
max_distance	int	Maximum edit distance for matches. Defaults to 2. Maximum value is 3 (limitation of Redis Vector Search).	2
filters	FilterClause | QueryFilter | None	Optional metadata filters to apply. Defaults to None.	None
options	QueryOptions | None	Query options, only limit is used here. Defaults to None.	None

Returns:

Type	Description
list[Chunk]	list[Chunk]: Matched chunks ordered by relevance/distance.

Raises:

Type	Description
ValueError	If max_distance is greater than 3.

Note

Maximum fuzzy distance is 3. This is a limitation of the Redis Search module.

update(update_values, filters=None) async

Update existing records in the datastore.

Processes updates in batches to avoid loading all matching documents into memory. 1. Get document IDs matching the filters. 2. In batch, get document data via document IDs. 3. In batch, update the document data.

Examples:

Update certain metadata of a chunk with specific filters.

from gllm_datastore.core.filters import filter as F

await fulltext_capability.update(
    update_values={"metadata": {"status": "published"}},
    filters=F.eq("metadata.status", "draft"),
)

Parameters:

Name	Type	Description	Default
update_values	dict[str, Any]	Mapping of fields to new values to apply.	required
filters	FilterClause | QueryFilter | None	Filters to select records to update. Defaults to None.	None

Raises:

Type	Description
Exception	If Redis operations fail.