Base

Frame extraction processor family.

Extracts one or more image frames from a video attachment at given timestamps. Concrete backends (FFmpeg, GStreamer) register under this family so callers only change backend=.

Shared constructor / process knobs live on FrameExtractionConfig and FrameExtractionProcessConfig. Backend modules inherit those bases and add engine-specific fields.

DeinterlaceMode

Bases: StrEnum

How frame extraction applies deinterlace (e.g. FFmpeg yadif).

Attributes:

Name	Type	Description
OFF		Never deinterlace.
FORCE		Always deinterlace (ignore field-order metadata).
AUTO		Deinterlace only when ffprobe reports an interlaced field order.

FrameExtractionConfig

Bases: BaseModel

Backend-agnostic stable configuration for frame extraction.

Yadif knobs are shared so FFmpeg can inline -vf yadif and GStreamer can forward the same values to DeinterlaceProcessor.

Attributes:

Name	Type	Description
output_format	str	Image encode format (JPEG/PNG). Defaults to JPEG.
deinterlace	DeinterlaceMode	Deinterlace policy (off / force / auto). Bool aliases True→force, False→off are accepted. Defaults to off.
yadif_mode	int	Yadif mode (0=send_frame, 1=send_field, 2=send_frame_nospatial, 3=send_field_nospatial). Defaults to 0.
yadif_parity	int	Field parity (-1=auto, 0=tff, 1=bff). Defaults to -1.
yadif_deint	int	Deinterlace all frames (0) or flagged-only (1). Defaults to 0.
default_timestamps	list[float] | None	Used when process is called without process_config. Defaults to None (must pass per-call).

validate_default_timestamps(value) classmethod

Validate optional constructor-level timestamps.

Parameters:

Name	Type	Description	Default
value	list[float] | None	Candidate timestamps.	required

Returns:

Type	Description
list[float] | None	list[float] | None: Validated timestamps or None.

validate_deinterlace_mode(value) classmethod

Coerce bool / string deinterlace values.

Parameters:

Name	Type	Description	Default
value	Any	Candidate mode.	required

Returns:

Name	Type	Description
DeinterlaceMode	DeinterlaceMode	Normalized mode.

validate_output_format(value) classmethod

Reject blank output format strings.

Parameters:

Name	Type	Description	Default
value	str	Candidate format.	required

Returns:

Name	Type	Description
str	str	Normalized upper-case format.

Raises:

Type	Description
ValueError	If blank.

yadif_filter()

Build the shared yadif=mode:parity:deint filter description.

Returns:

Name	Type	Description
str	str	Yadif filter string consumed by FFmpeg -vf and nested GStreamer deinterlace metadata.

FrameExtractionProcessConfig

Bases: ProcessorProcessConfig

Backend-agnostic per-invocation frame extraction config.

Attributes:

Name	Type	Description
timestamps	list[float]	Required non-empty timestamps in seconds.
output_format	str | None	Optional format override.
deinterlace	DeinterlaceMode | None	Optional deinterlace override.
yadif_mode	int | None	Optional yadif mode override.
yadif_parity	int | None	Optional field-parity override.
yadif_deint	int | None	Optional flagged-only override.

validate_optional_deinterlace_mode(value) classmethod

Coerce optional bool / string deinterlace overrides.

Parameters:

Name	Type	Description	Default
value	Any	Candidate mode or None.	required

Returns:

Type	Description
DeinterlaceMode | None	DeinterlaceMode | None: Normalized mode or None.

validate_optional_output_format(value) classmethod

Normalize optional format override.

Parameters:

Name	Type	Description	Default
value	str | None	Candidate format.	required

Returns:

Type	Description
str | None	str | None: Upper-case format or None.

validate_process_timestamps(value) classmethod

Validate per-call timestamps.

Parameters:

Name	Type	Description	Default
value	list[float]	Candidate timestamps.	required

Returns:

Type	Description
list[float]	list[float]: Validated timestamps.

FrameExtractionProcessor()

Bases: BackendSelectableProcessor[Attachment, list[Attachment]], ABC

Family base for extracting image frames at timestamps.

Why use this base class?

• Portability: Swap FFmpeg vs GStreamer without changing call sites.
• Batch I/O: Extract many keyframes in one process call.
• Separation: Keyframe planning stays in extractors; decode is here.

Usage

from gllm_multimodal.media_toolkit.processor.frame_extraction_processor import (
    FrameExtractionProcessConfig,
    FrameExtractionProcessor,
)

processor = FrameExtractionProcessor.build()
frames = await processor.process(
    video_attachment,
    process_config=FrameExtractionProcessConfig(timestamps=[1.5, 4.0]),
)

set_timestamps(timestamps) abstractmethod

Configure default timestamps used when process_config is omitted.

Parameters:

Name	Type	Description	Default
timestamps	list[float]	Non-empty list of non-negative seconds.	required

stream_frames(attachment, sample_fps, deinterlace=None) abstractmethod

Stream frames sampled at a fixed rate from a single decode pass.

Parameters:

Name	Type	Description	Default
attachment	Attachment	Source video.	required
sample_fps	float	Positive sampling rate in frames per second.	required
deinterlace	DeinterlaceMode | None	Deinterlace override. Defaults to None.	None

Returns:

Type	Description
AsyncIterator[tuple[float, ndarray]]	AsyncIterator[tuple[float, np.ndarray]]: Timestamps and BGR uint8 frames at source size.

coerce_deinterlace_mode(value)

Normalize bool / string inputs into DeinterlaceMode.

Bool aliases keep older call sites working: 1. True → FORCE 2. False → OFF

Parameters:

Name	Type	Description	Default
value	Any	Raw deinterlace config value.	required

Returns:

Name	Type	Description
DeinterlaceMode	DeinterlaceMode	Coerced mode.

Raises:

Type	Description
ValueError	If value cannot be coerced.

coerce_frame_extraction_config(config)

Normalize optional shared frame-extraction config.

Parameters:

Name	Type	Description	Default
config	FrameExtractionConfig | dict[str, object] | None	Raw config.	required

Returns:

Name	Type	Description
FrameExtractionConfig	FrameExtractionConfig	Shared configuration shape.

validate_timestamps(timestamps)

Validate a non-empty list of non-negative timestamps.

Parameters:

Name	Type	Description	Default
timestamps	list[float]	Candidate timestamps in seconds.	required

Returns:

Type	Description
list[float]	list[float]: The same timestamps when valid.

Raises:

Type	Description
ValueError	If empty or any timestamp is negative.