Skip to content

Overview

Modality converter modules for Gen AI applications.

This package provides the base classes and implementations for converting between different media modalities (image, audio, video) and text. The converters follow a common interface defined by BaseModalityConverter.

Submodules

  • image_to_text -- Convert images to text (OCR, captioning, mermaid).
  • audio_to_text -- Convert audio to text (speech-to-text, transcription).
  • video_to_text -- Convert video to text (video captioning, transcription).
  • schema -- Result schemas for converter operations.
  • modality_converter -- Abstract base class for all modality converters.

Usage

from gllm_multimodal.modality_converter.image_to_text import BaseImageToText
from gllm_multimodal.modality_converter.audio_to_text import BaseAudioToText
from gllm_multimodal.modality_converter.video_to_text import BaseVideoToText

Each converter implements the BaseModalityConverter.convert method, which takes a source (file path, URL, or bytes) and returns a TextResult or similar result object.

Authors

  • Yanfa Adi Putra (yanfa.a.putra@gdplabs.id)
  • Obryan Ramadhan (obryan.ramadhan@gdplabs.id)
  • Nico Samuelson Tjandra (nico.s.tjandra@gdplabs.id)