scystream.sdk.file_handling package

Submodules

scystream.sdk.file_handling.s3_manager module

class scystream.sdk.file_handling.s3_manager.S3Config

Bases: BaseModel

Configuration for accessing an S3-compatible service.

This model holds the necessary parameters to authenticate and connect to an S3-compatible service.

Parameters:

• S3_ACCESS_KEY – access key for authentication.

• S3_SECRET_KEY – secret key for authentication.

• S3_HOST – The endpoint URL for the S3-compatible service.

• S3_PORT – The port used by the S3-compatible service.

S3_ACCESS_KEY: str

S3_HOST: str

S3_PORT: str

S3_SECRET_KEY: str

model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class scystream.sdk.file_handling.s3_manager.S3Operations

Bases: object

A class that encapsulates operations on an S3-compatible service.

This class includes methods to upload and download files to/from an S3 bucket. It also ensures the bucket exists before performing any operations.

__init__(config: S3Config | FileSettings)

Initializes the S3 client with the provided configuration.

Parameters:

config – An instance of the S3Config model containing the

necessary authentication and connection information.

static download(settings: FileSettings, local_path: str)

Convenience helper to download a file without manually instantiating S3Operations.

Parameters:

• settings – File-related configuration (bucket, path, file name, extension, and S3 creds).

• local_path – Local destination path for the downloaded file.

Raises:

botocore.client.ClientError – If the download fails.

download_file(bucket_name: str, s3_object_name: str, local_file_path: str)

Downloads a file from an S3 bucket to a local path.

This method downloads the specified file from the given S3 bucket and saves it to the local filesystem.

Parameters:

• bucket_name – The name of the S3 bucket from which to download the file.

• s3_object_name – The name of the file in the S3 bucket that will be downloaded.

• local_file_path – The local path where the downloaded file will be saved.

Raises:

ClientError – If there is an error with the S3 client, such as a failed download.

download_file_from_settings(settings: FileSettings, local_path: str)

Wrapper for download_file function, takes the settings class and downloads it to a local path.

Parameters:

• settings – The settings class which should be used for downloading

• local_path – The local path where the downloaded file will be saved.

static upload(settings: FileSettings, local_path: str)

Convenience helper to upload a file without manually instantiating S3Operations.

Parameters:

• settings – File-related configuration (bucket, path, file name, extension, and S3 creds).

• local_path – Path to the local file that should be uploaded.

Raises:

botocore.client.ClientError – If the upload fails.

upload_file(path_to_file: str, bucket_name: str, target_name: str)

Uploads a file from the local filesystem to an S3 bucket.

This method uploads a file to the specified S3 bucket, creating the bucket if it does not exist.

Parameters:

• path_to_file – The path to the local file that needs to be uploaded.

• bucket_name – The name of the S3 bucket where the file will be uploaded.

• target_name – The name the file will have once it is uploaded to the bucket.

Raises:

ClientError – If there is an error with the S3 client, such as a failed upload.

upload_file_from_settings(settings: FileSettings, local_path: str)

Wrapper for upload_file() that uses a FileSettings instance to construct the S3 object key.

The object key is built as {FILE_PATH}/{FILE_NAME}.{FILE_EXT}.

Parameters:

• settings – File-related configuration (bucket, path, file name, extension, and S3 creds).

• local_path – Path to the local file that should be uploaded.

Raises:

botocore.client.ClientError – If the upload fails.