Redundancy

Redundancy metric.

This metric is used to evaluate the redundancy of the model's output.

Authors

Surya Mahadi (made.r.s.mahadi@gdplabs.id)

References

None

RedundancyMetric(model=DefaultValues.MODEL, model_credentials=None, model_config=None, prompt_builder=None, response_schema=None)

Bases: LMBasedMetric

Redundancy metric.

Attributes:

Name	Type	Description
name	str	The name of the metric.
response_schema	ResponseSchema	The response schema to use for the metric.
prompt_builder	PromptBuilder	The prompt builder to use for the metric.
model	Union[str, ModelId, BaseLMInvoker]	The model to use for the metric.
model_config	dict[str, Any] | None	The model config to use for the metric. Defaults to an empty dictionary.
model_credentials	str	The model credentials to use for the metric.

Initialize the RedundancyMetric class.

Default expected input: - query (str): The query to evaluate the redundancy of the model's output. - generated_response (str): The generated response to evaluate the redundancy of the model's output.

Parameters:

Name	Type	Description	Default
model	Union[str, ModelId, BaseLMInvoker]	The model to use for the metric.	MODEL
model_credentials	str | None	The model credentials to use for the metric. Defaults to None.	None
model_config	dict[str, Any] | None	The model config to use for the metric. Defaults to an empty dictionary.	None
prompt_builder	PromptBuilder | None	The prompt builder to use for the metric. Defaults to default prompt builder.	None
response_schema	ResponseSchema | None	The response schema to use for the metric. Defaults to RedundancyResponseSchema.	None

RedundancyResponseSchema

Bases: BaseModel

Response schema for the redundancy metric.

Attributes:

Name	Type	Description
generated_response	list[str]	The generated response.
analysis	list[str]	The analysis of the generated response.
score	int	The score of the generated response.
explanation	str	The explanation of the generated response.