Fulltext

Redis implementation of fulltext search and CRUD capability.

This module provides a Redis implementation of the FulltextCapability protocol using Redis's native data structures and search capabilities.

RedisFulltextCapability(index_name, client, encryption=None)

Redis implementation of FulltextCapability protocol.

Attributes:

Name	Type	Description
index_name	str	Name of the Redis index.
client	Redis	Redis client instance.

Initialize the Redis fulltext capability.

Schema will be automatically inferred from chunks when creating a new index, or auto-detected from an existing index when performing operations.

Parameters:

Name	Type	Description	Default
index_name	str	Name of the Redis index.	required
client	Redis	Redis client instance.	required
encryption	EncryptionCapability | None	Encryption capability for field-level encryption. Defaults to None.	None

clear() async

Clear all records from the datastore.

create(data) async

Create new records in the datastore.

This method will automatically encrypt the content and metadata of the chunks if encryption is enabled following the encryption configuration.

If the index does not exist and no filterable_fields were provided, the schema will be inferred from the chunks being created.

Examples:

Create a new chunk.

await fulltext_capability.create(Chunk(content="Test chunk", metadata={"category": "test"}))

Parameters:

Name	Type	Description	Default
data	Chunk | list[Chunk]	Data to create (single item or collection).	required

Raises:

Type	Description
ValueError	If data structure is invalid or chunk content is invalid.

delete(filters=None, options=None) async

Delete records from the datastore.

Processes deletions in batches to avoid loading all matching documents into memory. For delete operations, only document IDs are retrieved (not full content) to minimize memory usage.

Parameters:

Name	Type	Description	Default
filters	FilterClause | QueryFilter | None	Filters to select records to delete. Defaults to None.	None
options	QueryOptions | None	Query options for sorting and limiting deletions (for eviction-like operations). Defaults to None.	None

Raises:

Type	Description
Exception	If Redis operations fail.

retrieve(filters=None, options=None) async

Read records from the datastore with optional filtering.

Parameters:

Name	Type	Description	Default
filters	FilterClause | QueryFilter | None	Query filters to apply. Defaults to None.	None
options	QueryOptions | None	Query options for sorting and pagination. Defaults to None, in which case the default limit of 10 is used.	None

Returns:

Type	Description
list[Chunk]	list[Chunk]: List of matched chunks after applying filters and options.

retrieve_fuzzy(query, max_distance=2, filters=None, options=None) async

Find records that fuzzy match the query within distance threshold.

Parameters:

Name	Type	Description	Default
query	str	Text to fuzzy match against.	required
max_distance	int	Maximum edit distance for matches. Defaults to 2. Maximum value is 3 (limitation of Redis Vector Search).	2
filters	FilterClause | QueryFilter | None	Optional metadata filters to apply. Defaults to None.	None
options	QueryOptions | None	Query options, only limit is used here. Defaults to None.	None

Returns:

Type	Description
list[Chunk]	list[Chunk]: Matched chunks ordered by relevance/distance.

Raises:

Type	Description
ValueError	If max_distance is greater than 3.

Note

Maximum fuzzy distance is 3. This is a limitation of the Redis Search module.

update(update_values, filters=None) async

Update existing records in the datastore.

This method will automatically encrypt the content and metadata in update_values if encryption is enabled following the encryption configuration.

Warning

Filters cannot target encrypted fields. While update_values are encrypted before being written, the filters used to identify which documents to update are NOT encrypted. If you try to update documents based on an encrypted metadata field (e.g., filters=F.eq("metadata.secret", "val")), the filter will fail to match because the filter value is not encrypted but the stored data is. Always use non-encrypted fields (like 'id') in filters when working with encrypted data.

Processes updates in batches to avoid loading all matching documents into memory. 1. Get document IDs matching the filters. 2. In batch, get document data via document IDs. 3. In batch, update the document data.

Examples:

Update certain metadata of a chunk with specific filters: ```python from gllm_datastore.core.filters import filter as F

await fulltext_capability.update(
    update_values={"metadata": {"status": "published"}},
    filters=F.eq("id", "chunk_id"),
)
```

Update encrypted data (encryption must be enabled): ```python from gllm_datastore.core.filters import filter as F

# Correct: Use non-encrypted 'id' field in filter
await fulltext_capability.update(
    update_values={"metadata": {"secret_value": "new_secret"}},
    filters=F.eq("id", "chunk_id")
)

# Incorrect: Using encrypted field in filter will fail to match
# await fulltext_capability.update(
#     update_values={"metadata": {"status": "published"}},
#     filters=F.eq("metadata.secret_key", "value")  # Won't match!
# )
```

Parameters:

Name	Type	Description	Default
update_values	dict[str, Any]	Mapping of fields to new values to apply.	required
filters	FilterClause | QueryFilter | None	Filters to select records to update. Cannot use encrypted fields in filters. Defaults to None.	None

Raises:

Type	Description
Exception	If Redis operations fail.