Cloud storage
Base module for cloud storage functionality.
This module defines the abstract base class for cloud storage providers. Each cloud provider implementation should extend this class and implement the required methods.
Reviewer
- Muhammad Afif Al Hawari (muhammad.a.a.hawari@gdplabs.id)
References
NONE
CloudStorageClient(config)
Bases: ABC
Abstract base class for cloud storage clients.
This class defines the interface that all cloud storage implementations must follow. It provides methods for uploading and downloading files from cloud storage services.
Initialize the cloud storage client with provider-specific configuration.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
config |
Dict[str, str]
|
Configuration parameters for the storage client. The exact keys will depend on the specific cloud provider implementation. |
required |
delete_file(remote_path)
abstractmethod
Delete a file from cloud storage.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
remote_path |
str
|
Path to the file in cloud storage. |
required |
Returns:
| Name | Type | Description |
|---|---|---|
bool |
bool
|
True if the file was deleted successfully, False otherwise. |
Raises:
| Type | Description |
|---|---|
FileNotFoundError
|
If the remote file does not exist. |
RuntimeError
|
If the deletion fails. |
download_file(remote_path, local_path)
abstractmethod
Download a file from cloud storage.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
remote_path |
str
|
Path to the file in cloud storage. |
required |
local_path |
Union[str, Path]
|
Path where the downloaded file should be saved. |
required |
Returns:
| Name | Type | Description |
|---|---|---|
Path |
Path
|
Path to the downloaded file. |
Raises:
| Type | Description |
|---|---|
FileNotFoundError
|
If the remote file does not exist. |
RuntimeError
|
If the download fails. |
get_storage_client(provider, config)
staticmethod
Factory method to get the appropriate storage client.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
provider |
str
|
The storage provider to use (e.g., "s3", "gcs", "azure"). |
required |
config |
Dict[str, str]
|
Configuration for the storage client. |
required |
Returns:
| Name | Type | Description |
|---|---|---|
CloudStorageClient |
CloudStorageClient
|
The storage client for the specified provider. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If the provider is not supported. |
upload_file(local_path, remote_path)
abstractmethod
Upload a file to cloud storage.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
local_path |
Union[str, Path]
|
Path to the local file to upload. |
required |
remote_path |
str
|
Path in the cloud storage where the file should be saved. |
required |
Returns:
| Name | Type | Description |
|---|---|---|
str |
str
|
URL or identifier of the uploaded file in the cloud storage. |
Raises:
| Type | Description |
|---|---|
FileNotFoundError
|
If the local file does not exist. |
RuntimeError
|
If the upload fails. |