Hybrid cache

Defines a base class for hybrid cache used in Gen AI applications.

Authors

Henry Wicaksono (henry.wicaksono@gdplabs.id)

References

NONE

BaseHybridCache(key_matcher=None)

Bases: BaseCache, ABC

A base class for hybrid cache used in Gen AI applications.

The BaseHybridCache class provides a framework for storing and retrieving cache data.

Attributes:

Name	Type	Description
key_matcher	BaseKeyMatcher	The key matcher that defines the cache key matching strategy.

Initialize a new instance of the BaseHybridCache class.

Parameters:

Name	Type	Description	Default
key_matcher	BaseKeyMatcher | None	The key matcher that defines the cache key matching strategy. Defaults to None, in which case the ExactKeyMatcher will be used.	None

cache(key_func=None, name='', ttl=None)

Decorator for caching function results.

This decorator caches the results of the decorated function using this cache storage. The cache key is generated using the provided key function or a default key generation based on the function name and arguments.

Synchronous and asynchronous functions are supported.

Parameters:

Name	Type	Description	Default
key_func	Callable[P, str] | None	Function to generate cache keys. Must accept the same parameters as the decorated function.	None
name	str	Name to use in the default key generation if key_func is None.	''
ttl	int | str | None	The time-to-live for the cached data. Can be an integer in seconds or a string (e.g. "1h", "1d", "1w", "1y"). If None, the cache data will not expire. Defaults to None. In this case, the cache will not expire.	None
matching_strategy	MatchingStrategy	The strategy to use for matching keys. This can be one of the values from the MatchingStrategy enum. Defaults to exact matching.	required
matching_config	dict[str, Any]	Configuration parameters for matching strategies. Defaults to None.	required

Example

def get_user_cache_key(user_id: int) -> str:
    return f"user:{user_id}"

@cache_store.cache(key_func=get_user_cache_key, ttl="1h")
async def get_user(user_id: int) -> User:
    return await db.get_user(user_id)

# will use/store cache with key "user:1", expiring after 1 hour
user1 = await get_user(1)

Returns:

Name	Type	Description
Callable	Callable[[Callable[P, T]], Callable[P, T]]	A decorator function.

clear() async

Clears all cache data from the storage.

This method clears all cache data from the storage by calling both the key_matcher.clear and _clear methods.

delete(key) async

Deletes cache data from the storage.

This method deletes the key by calling both the key_matcher.delete and _delete methods.

Parameters:

Name	Type	Description	Default
key	str | list[str]	The key(s) to delete the cache data.	required

retrieve(key) async

Retrieves cache data from the storage.

This method first retrieves the key using the strategy defined in the key_matcher. If a matching key is found, the method will retrieve the cache data from the storage using the _retrieve method. Otherwise, the method will return None.

Parameters:

Name	Type	Description	Default
key	str	The key to retrieve the cache data.	required

Returns:

Name	Type	Description
Any	Any	The retrieved cache data.

retrieve_all_keys() abstractmethod async

Retrieves all keys from the storage.

This method must be implemented by the subclasses to define the logic for retrieving all keys from the storage.

Returns:

Type	Description
set[str]	set[str]: A set of all keys in the storage.

Raises:

Type	Description
NotImplementedError	If the method is not implemented.

store(key, value, ttl=None) async

Stores cache data in the storage.

This method preprocesses the TTL (time-to-live) value to seconds if provided, and then calls both the key_matcher.store and _store methods to store the cache data in the storage.

Parameters:

Name	Type	Description	Default
key	str	The key to store the cache data.	required
value	Any	The cache data to store.	required
ttl	int | str | None	The time-to-live (TTL) for the cache data. Must either be an integer in seconds or a string (e.g. "1h", "1d", "1w", "1y"). If None, the cache data will not expire.	None