Cache
In-memory implementation of cache capability with advanced matching.
This module provides an in-memory implementation of the CacheCapability protocol using dictionary-based storage with configurable eviction policies.
References
NONE
InMemoryCacheCapability(em_invoker=None, store=None)
In-memory implementation of CacheCapability protocol.
This class provides key-value caching with advanced matching strategies and eviction policies using pure Python data structures.
Attributes:
| Name | Type | Description |
|---|---|---|
store |
dict[str, Chunk]
|
Dictionary storing cache entries. Entries will be Chunk objects. |
em_invoker |
BaseEMInvoker | None
|
Embedding model for semantic matching. |
Initialize the in-memory cache capability.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
em_invoker |
BaseEMInvoker | None
|
Embedding model for semantic matching. Defaults to None, in which case semantic matching is not available. |
None
|
store |
dict[str, Chunk] | None
|
Dictionary storing cache entries as Chunk. Defaults to None. |
None
|
create(key, value, metadata=None, ttl=None)
async
Insert a value into cache with optional TTL and metadata.
Usage Example
await cache_store.create("key", "value", ttl="1h")
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
key |
str
|
Cache key. |
required |
value |
str
|
Value to cache. |
required |
metadata |
dict | None
|
Optional metadata for advanced matching. Defaults to None. |
None
|
ttl |
int | str | None
|
The time-to-live for the cache. This can be an integer (in seconds) or a string (e.g., "1h", "30m"). Defaults to None. |
None
|
Raises:
| Type | Description |
|---|---|
TypeError
|
If key is not a string. |
ValueError
|
If key is empty or whitespace-only. |
delete(key)
async
Delete entries by exact key match.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
key |
str | list[str]
|
Single key or list of keys to delete. |
required |
delete_expired(now=None)
async
Delete expired entries and enforce size limits.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
now |
float | None
|
Current timestamp for expiration checks. Defaults to None, in which case the current timestamp is used. |
None
|
delete_lfu(num_entries)
async
Delete least frequently used entries.
The least frequently used are defined as the entries with the smallest access_count.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
num_entries |
int
|
Number of entries to delete. |
required |
delete_lru(num_entries)
async
Delete least recently used entries.
The least recently used are defined as the entries with the smallest last_used_at timestamp.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
num_entries |
int
|
Number of entries to delete. |
required |
retrieve_exact(key)
async
Retrieve exact key match from cache.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
key |
str
|
Cache key to retrieve. |
required |
Returns:
| Name | Type | Description |
|---|---|---|
Any |
str | None
|
Cached value if found, None otherwise. |
retrieve_fuzzy(key, max_distance=DEFAULT_FUZZY_MATCH_MAX_DISTANCE, filters=None)
async
Find fuzzy matches for key within distance threshold.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
key |
str
|
Base key for fuzzy matching. |
required |
max_distance |
int
|
Maximum edit distance for matches. Defaults to 2. |
DEFAULT_FUZZY_MATCH_MAX_DISTANCE
|
filters |
QueryFilter | None
|
Query filters to apply. Defaults to None. |
None
|
**kwargs |
Datastore-specific parameters. |
required |
Returns:
| Name | Type | Description |
|---|---|---|
Any |
str | None
|
Best matching value or None. |
retrieve_semantic(key, min_similarity=0.8, filters=None)
async
Find semantically similar matches using embeddings.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
key |
str
|
Base key for semantic matching. |
required |
min_similarity |
float
|
Minimum similarity threshold (0.0-1.0). Defaults to 0.8. |
0.8
|
filters |
QueryFilter | None
|
Query filters to apply. Defaults to None. |
None
|
**kwargs |
Datastore-specific parameters. |
required |
Returns:
| Type | Description |
|---|---|
str | None
|
Any | None: Best matching value or None. |
update(key, value, metadata=None)
async
Update an entry in the cache.
Usage Example
This will update the value of the cache entry for the key "key" with the new value "new_value", and create a new vector for the key if em_invoker is available.
await cache_store.create("key", "value")
await cache_store.update("key", "new_value")
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
key |
str
|
Cache key to update. |
required |
value |
Any
|
Value to update. |
required |
metadata |
dict | None
|
Optional metadata for advanced matching. Defaults to None. |
None
|
Raises:
| Type | Description |
|---|---|
TypeError
|
If key is not a string. |
ValueError
|
If key is empty or whitespace-only, or if key doesn't exist. |