Simple Runner

Runner class for evaluating datasets.

Authors

Surya Mahadi (made.r.s.mahadi@gdplabs.id)

References

NONE

Runner(evaluator, batch_size=10)

Runner class for evaluating datasets.

Attributes:

Name	Type	Description
evaluator	BaseEvaluator	The evaluator to use.
batch_size	int	The batch size to use. Defaults to 10.

Initialize the Runner.

Parameters:

Name	Type	Description	Default
evaluator	BaseEvaluator	The evaluator to use.	required
batch_size	int	The batch size to use. Defaults to 10.	10

evaluate(dataset) async

Run the evaluator on the dataset.

The dataset is evaluated in batches of the given batch size.

Parameters:

Name	Type	Description	Default
dataset	BaseDataset	The dataset to run the evaluator on.	required

Returns:

Name	Type	Description
BaseDataset	BaseDataset	The dataset with the evaluator results.