Router

Modules concerning the routers used in Gen AI applications.

AurelioSemanticRouter(default_route, valid_routes, encoder, routes=None, index=None, auto_sync=SyncMode.LOCAL.value, **kwargs)

Bases: BaseRouter

A router that utilizes the Aurelio Labs library to route the input source to the appropriate path.

The AurelioSemanticRouter utilizes the Aurelio Labs library to route a given input source to an appropriate path based on the similarity with existing samples. If the determined route is not valid, it defaults to a predefined route.

Attributes:

Name	Type	Description
route_layer	RouteLayer	The Aurelio Labs route layer that handles the routing logic.
default_route	str	The default route to be used if the input source is not similar to any of the routes.
valid_routes	set[str]	A set of valid routes for the router.

Notes

For more information about the Aurelio Labs library, please refer to https://github.com/aurelio-labs/semantic-router

Initializes a new instance of the AurelioSemanticRouter class.

To define the routes, at least one of the routes or index parameters must be provided. When both parameters are provided, the routes parameter is ignored.

Parameters:

Name	Type	Description	Default
default_route	str	The default route to be used if the input source is not similar to any of the routes.	required
valid_routes	set[str]	A set of valid routes for the router.	required
encoder	DenseEncoder	An Aurelio Labs dense encoder to encode the input source and the samples. The encoded vectors are used to calculate the similarity between the input source and the samples.	required
routes	list[Route] | dict[str, list[EMContent]] | None	A list of Aurelio Labs Routes or a dictionary mapping route names to the list of samples. Each sample can be: 1. str: Plain text example 2. bytes: Binary data (converted to Attachment) 3. Attachment: Image, document, or other file attachment 4. URLAttachment: URL-based attachment 5. tuple: Mixed content example Ignored if index is provided. Defaults to None.	None
index	BaseAurelioIndex | None	A router index to retrieve the routes. If provided, it is prioritized over routes. Defaults to None.	None
auto_sync	str	The auto-sync mode for the router. Defaults to "local".	value
kwargs	Any	Additional keyword arguments to be passed to the Aurelio Labs Route Layer.	{}

Raises:

Type	Description
ValueError	If neither routes nor index is provided. If the parsed routes contains routes that are not in the set of valid routes. If the provided default route is not in the set of valid routes.

from_file(default_route, valid_routes, file_path) classmethod

Creates a new instance of the AurelioSemanticRouter class from a file.

This method creates a new instance of the AurelioSemanticRouter class from a file. It supports JSON and YAML file extensions.

Parameters:

Name	Type	Description	Default
default_route	str	The default route to be used if the input source is not similar to any of the routes.	required
valid_routes	set[str]	A set of valid routes for the router.	required
file_path	str	The path to the file containing the routes. The file extension must be either JSON or YAML.	required

Returns:

Name	Type	Description
AurelioSemanticRouter	AurelioSemanticRouter	A new instance of the AurelioSemanticRouter class.

Raises:

Type	Description
ValueError	If the file extension is not ".json" or ".yaml".

from_preset(modality, preset_name, preset_kwargs=None, **kwargs) classmethod

Initialize the Aurelio semantic based router component using preset model configurations.

Parameters:

Name	Type	Description	Default
modality	str	type of modality input.	required
preset_name	str	Name of the preset to use.	required
preset_kwargs	dict | None	placeholder for preset additional arguments.	None
**kwargs	Any	Additional arguments to pass for this class.	{}

Returns:

Name	Type	Description
AurelioSemanticRouter	AurelioSemanticRouter	Initialized aurelio semantic based router component using preset model.

LMBasedRouter(lm_request_processor, default_route, valid_routes, lm_output_key=DEFAULT_LM_OUTPUT_KEY)

Bases: BaseRouter, UsesLM

A router that utilizes a language model to determine the appropriate route for an input source.

This class routes a given input source to an appropriate path based on the output of a language model. If the determined route is not valid, it defaults to a predefined route.

The router delegates routing logic to a pluggable backend adapter (e.g., NativeLMBasedAdapter).

Attributes:

Name	Type	Description
backend	BaseAdapter	The backend implementation handling routing logic.
lm_request_processor	LMRequestProcessor	The request processor that handles requests to the language model.
default_route	str	The default route to be used if the language model's output is invalid.
valid_routes	set[str]	A set of valid routes for the router.
lm_output_key	str	The key in the language model's output that contains the route.

Notes

The lm_request_processor must be configured to: 1. Take a "source" key as input. The input source of the router should be passed as the value of this "source" key. 2. Return a JSON object which contains the selected route as a string. The key of the route is specified by the lm_output_key attribute. Furthermore, the selected route must be present in the valid_routes set.

Output example, assuming the lm_output_key is "route": { "route": "" }

Initializes a new instance of the LMBasedRouter class.

Parameters:

Name	Type	Description	Default
lm_request_processor	LMRequestProcessor	The request processor that handles requests to the language model.	required
default_route	str	The default route to be used if the language model's output is invalid.	required
valid_routes	set[str]	A set of valid routes for the router.	required
lm_output_key	str	The key in the language model's output that contains the route. Defaults to DEFAULT_LM_OUTPUT_KEY.	DEFAULT_LM_OUTPUT_KEY

Raises:

Type	Description
ValueError	If the provided default route is not in the set of valid routes.

from_preset(modality, preset_kwargs=None, **kwargs) classmethod

Initialize the LM based router component using preset model configurations.

Examples:

router = LMBasedRouter.from_preset(modality="image")
router = LMBasedRouter.from_preset(
    modality="image",
    preset_kwargs={"default_route": "custom_route", "valid_routes": {"custom_route", "other"}}
)

Parameters:

Name	Type	Description	Default
modality	str	Type of modality input (e.g., "image").	required
preset_kwargs	LMBasedRouterPresetKwargs | dict[str, Any] | None	Additional kwargs for the preset. Can be provided as a dict or as an LMBasedRouterPresetKwargs instance. When provided as a dict, unknown keys will raise a ValidationError. Defaults to None. The following keys are allowed: 1. lm_invoker_kwargs: Configuration for the language model invoker. 2. prompt_builder_kwargs: Configuration for the prompt builder. 3. lm_output_key: The key in the language model's output that contains the route. 4. default_route: Default route to use if LM cannot determine route. 5. valid_routes: Set of valid routes for the router.	None
**kwargs	Any	Additional arguments to pass for this class.	{}

Returns:

Name	Type	Description
LMBasedRouter	LMBasedRouter	Initialized LM-based router component using preset model.

native(lm_request_processor, default_route, valid_routes, lm_output_key=DEFAULT_LM_OUTPUT_KEY) classmethod

Create LM-based router with native backend.

This is a convenience classmethod that creates an LMBasedRouter with a NativeLMBasedAdapter.

Parameters:

Name	Type	Description	Default
lm_request_processor	LMRequestProcessor	The request processor that handles requests to the language model.	required
default_route	str	The default route to be used if the language model's output is invalid.	required
valid_routes	set[str]	A set of valid routes for the router.	required
lm_output_key	str	The key in the language model's output that contains the route. Defaults to DEFAULT_LM_OUTPUT_KEY.	DEFAULT_LM_OUTPUT_KEY

Returns:

Name	Type	Description
LMBasedRouter	LMBasedRouter	Initialized router with native backend.

RuleBasedRouter(ruleset_map, default_route, valid_routes)

Bases: BaseRouter

A rule-based router that directs the input text to an appropriate route based on a set of rules.

The RuleBasedRouter routes incoming input text to different paths by evaluating a set of rules encapsulated in RouterRuleset objects. Each ruleset consists of multiple RouterRule objects, and the router determines which route to take based on the input text and the rules defined in the ruleset.

If match_all is True in a RouterRuleset, all rules in that ruleset must match the input text for the associated route to be selected. If False, matching any rule in the ruleset will cause that route to be selected. If no match is found, the router defaults to the default_route.

Attributes:

Name	Type	Description
ruleset_map	dict[str, RouterRuleset]	A mapping of route names to their corresponding rulesets.
default_route	str	The default route to be taken if no ruleset matches the input text.
valid_routes	set[str]	A set of valid routes that the router can direct to.

Initializes a new instance of the RuleBasedRouter class.

Parameters:

Name	Type	Description	Default
ruleset_map	dict[str, RouterRuleset]	A mapping of route names to their corresponding rulesets.	required
default_route	str	The default route to be taken if no ruleset matches the input text.	required
valid_routes	set[str]	A set of valid routes for the router.	required

Raises:

Type	Description
ValueError	If the ruleset_map contains routes that are not in the set of valid routes. If the provided default route is not in the set of valid routes.

SemanticRouter(backend, default_route, valid_routes)

Bases: BaseRouter

A backend-agnostic router using semantic similarity.

This router delegates all routing logic to a pluggable backend implementation, supporting multiple semantic routing algorithms (Aurelio, Similarity-based, etc.).

The router provides a unified interface regardless of the underlying algorithm, allowing users to switch backends without changing their code.

Supported backends

1. BackendType.AURELIO: Aurelio Labs semantic router (via SemanticRouter.aurelio).
2. BackendType.NATIVE: Native GLLM semantic router (via SemanticRouter.native).

Attributes:

Name	Type	Description
backend	BaseAdapter	The backend implementation handling routing logic.
default_route	str	The default route to be used if no similar route is found.
valid_routes	set[str]	A set of valid routes for the router.

Initialize the SemanticRouter.

Parameters:

Name	Type	Description	Default
backend	BaseAdapter	The backend implementation for routing.	required
default_route	str	The default route to be used if no similar route is found.	required
valid_routes	set[str]	A set of valid routes for the router.	required

Raises:

Type	Description
ValueError	If backend is not a BaseAdapter instance.

aurelio(default_route, valid_routes, encoder, strategy=SemanticRouterStrategy.SIMILARITY, similarity_threshold=DEFAULT_SIMILARITY_THRESHOLD, route_examples=None, index=None, **kwargs) classmethod

Create semantic router with Aurelio backend similarity routing.

Uses Aurelio Labs semantic router for advanced semantic similarity routing. Supports custom encoders and vector store indexes.

Examples:

from gllm_inference.em_invoker import build_em_invoker
em_invoker = build_em_invoker("openai/text-embedding-3-small", credentials="sk-")
router = SemanticRouter.aurelio(
    default_route="general",
    valid_routes={"greeting", "help", "general"},
    encoder=em_invoker,
    route_examples={
        "greeting": ["hi", "hello", "hey"],
        "help": ["help me", "assist", "support"],
    }
)

Parameters:

Name	Type	Description	Default
default_route	str	Default route if no match found.	required
valid_routes	set[str]	Set of valid route names.	required
encoder	DenseEncoder | BaseEMInvoker	Dense encoder or EM invoker (required).	required
strategy	SemanticRouterStrategy	Routing strategy. Defaults to SIMILARITY.	SIMILARITY
similarity_threshold	float	Similarity threshold for routing. Defaults to DEFAULT_SIMILARITY_THRESHOLD.	DEFAULT_SIMILARITY_THRESHOLD
route_examples	dict[str, list[str | bytes]] | None	Route examples. Defaults to None.	None
index	BaseAurelioIndex | None	Custom vector store index. Defaults to None.	None
**kwargs	Any	Additional Aurelio-specific arguments (e.g., auto_sync).	{}

Returns:

Name	Type	Description
SemanticRouter	SemanticRouter	Initialized router instance.

Raise

ValueError: If strategy is not supported for Aurelio backend.

from_file(backend_type, strategy, file_path, default_route, valid_routes, **kwargs) classmethod

Create a new SemanticRouter instance from a file.

Parameters:

Name	Type	Description	Default
backend_type	BackendType	Type of backend (BackendType.AURELIO or BackendType.NATIVE).	required
strategy	SemanticRouterStrategy	Routing strategy (e.g., SIMILARITY).	required
file_path	str	The path to the file containing the routes. Must be JSON or YAML.	required
default_route	str	The default route to be used if no similar route is found.	required
valid_routes	set[str]	A set of valid routes for the router.	required
**kwargs	Any	Additional arguments for backend initialization (e.g., encoder, em_invoker).	{}

Returns:

Name	Type	Description
SemanticRouter	SemanticRouter	A new instance of the SemanticRouter class.

Raises:

Type	Description
ValueError

from_preset(backend, preset_name, modality=ModalityType.IMAGE, strategy=SemanticRouterStrategy.SIMILARITY, default_route=None, valid_routes=None, **kwargs) classmethod

Create router from preset configuration.

Uses the adapter's from_preset method to initialize the backend with preset route data, then wraps it in a SemanticRouter.

Examples:

router = SemanticRouter.from_preset(
    backend=BackendType.AURELIO,
    modality="image",
)

Args: backend (BackendType | str): Backend to use. Currently only BackendType.AURELIO (or the string "aurelio") is supported. preset_name (str): Name of the preset to use. modality (ModalityType | str, optional): Modality type for the preset. Defaults to ModalityType.IMAGE. strategy (SemanticRouterStrategy | None, optional): Routing strategy (e.g., SemanticRouterStrategy.SIMILARITY). Defaults to SemanticRouterStrategy.SIMILARITY. default_route (str | None, optional): Default route if no match found. If not provided, the first key of the preset routes is used. Defaults to None. valid_routes (set[str] | None, optional): Set of valid route names. If not provided, the keys of the preset routes are used. Defaults to None. **kwargs (Any): Backend and preset-specific arguments. May include encoder and routes.

Returns:

Name	Type	Description
SemanticRouter	SemanticRouter	Initialized router instance.

Raises:

Type	Description
ValueError	If the backend does not support preset initialization.

native(default_route, valid_routes, em_invoker, strategy=SemanticRouterStrategy.SIMILARITY, similarity_threshold=DEFAULT_SIMILARITY_THRESHOLD, route_examples=None, route_embeddings=None) classmethod

Create semantic router with native backend similarity routing.

Uses native gllm-inference cosine similarity for fast, simple routing. No training required. Best for small to medium-sized route sets.

Parameters:

Name	Type	Description	Default
default_route	str	Default route if no match found.	required
valid_routes	set[str]	Set of valid route names.	required
em_invoker	BaseEMInvoker	EM invoker for vectorization.	required
strategy	SemanticRouterStrategy	Routing strategy. Defaults to SemanticRouterStrategy.SIMILARITY.	SIMILARITY
similarity_threshold	float	Minimum similarity score (0-1). Defaults to DEFAULT_SIMILARITY_THRESHOLD.	DEFAULT_SIMILARITY_THRESHOLD
route_examples	dict[str, list[str]] | None	Route examples. Defaults to None.	None
route_embeddings	dict[str, list[list[float]]] | None	Route embeddings. Defaults to None.	None

Returns:

Name	Type	Description
SemanticRouter	SemanticRouter	Initialized router instance.

Examples:

from gllm_inference.em_invoker import build_em_invoker
em_invoker = build_em_invoker(...)
router = SemanticRouter.native(
    default_route="general",
    valid_routes={"greeting", "help", "general"},
    em_invoker=em_invoker,
    route_examples={
        "greeting": ["hi", "hello", "hey"],
        "help": ["help me", "assist", "support"],
    },
    similarity_threshold=0.7
)

Raise

ValueError: If strategy is not supported for Native backend.

SimilarityBasedRouter(em_invoker, route_examples, default_route, similarity_threshold=0.5)

Bases: BaseRouter

A router that utilizes an embedding model to determine the appropriate route for an input text.

This class routes a given input text to an appropriate path based on semantic similarity between the input text and predefined route examples. It calculates cosine similarity between the input text embedding and the embeddings of route examples, then selects the route with the highest similarity score above a threshold.

Attributes:

Name	Type	Description
em_invoker	BaseEMInvoker	The embedding model invoker to use for vectorization.
route_examples	dict[str, list[str]]	A mapping of route names to their example texts.
similarity_threshold	float	The minimum similarity score required for a route to be selected.
default_route	str	The default route to be used if no route meets the similarity threshold.
valid_routes	set[str]	A set of valid routes for the router.

Initializes a new instance of the SimilarityBasedRouter class.

Args: em_invoker (BaseEMInvoker): The embedding model invoker to use for vectorization. route_examples (dict[str, list[str]]): A mapping of route names to their example texts. The keys define the valid routes, and values are example texts for each route. default_route (str): The default route to be used if no route meets the similarity threshold. Must be one of the keys in route_examples. Examples: "general", "fallback", "customer_service" similarity_threshold (float, optional): The minimum similarity score required for a route to be selected. Must be between 0 and 1. This threshold is compared against normalized cosine similarity scores, which are derived from the standard cosine similarity (ranging from -1 to 1) and converted to a 0-1 range where 0 indicates maximum dissimilarity and 1 indicates maximum similarity. Examples of route_examples: { "tech_support": [ "I can't log in to my account", "The app keeps crashing", "Password reset not working", "Getting error code 500" ], "billing": [ "How much do I owe?", "I want to dispute a charge", "When is my payment due?", "Can I get a refund?" ], "general": [ "What are your business hours?", "How do I contact support?", "Tell me about your services", "Where are you located?" ] }

Raises:

Type	Description
ValueError	If route_examples is not a dictionary. If route_examples is falsy (None, empty dict, etc.). If the similarity threshold is not between 0 and 1. If the provided default route is not in the route_examples keys. If any route has an empty list of examples.

clear_cache()

Clears the cached route embeddings.

This method clears the cached embeddings, forcing them to be regenerated on the next routing operation.