s3

Configuration storage and management using AWS S3 without versioning.

Provides a unified interface for storing application configurations in S3 buckets with versioning disabled. This module requires S3 bucket versioning to be disabled and implements custom file-based versioning for configuration management.

Key Features:

• Requires S3 bucket versioning to be disabled

• Implements custom versioning using file naming conventions

• Stores multiple files per parameter with sequential version numbers

• Manual version tracking with integer sequences (1, 2, 3, …)

• Latest file always points to current version

Warning

This module only works with versioning-disabled S3 buckets. Versioned buckets are not supported.

aws_config.s3.read_text(s3path: S3Path, **kwargs) → str

Read text content from S3 object with proper error handling.

After_param s3path:

S3 path to read from

After_param kwargs:

Additional arguments passed to S3Path.read_text()

Returns:

Text content of the S3 object

Raises:

S3ObjectNotExist – If the S3 object does not exist

class aws_config.s3.S3Parameter(s3dir_config: S3Path, parameter_name: str)

S3 parameter storage for versioning-disabled buckets.

This class manages configuration storage in S3 buckets with versioning disabled. It provides a custom versioning system using file naming conventions to track configuration versions manually.

Why use S3 for configuration storage?

• Centralized configuration management across environments

• Built-in durability and availability guarantees

• Integration with AWS IAM for access control

• Audit trail through CloudTrail

• Cost-effective storage for configuration data

• Custom versioning without S3 versioning overhead

How it works:

Storage Structure:

• Multiple files per parameter using naming convention

• Latest file: {s3dir_config}/{parameter_name}/{parameter_name}-000000-latest.json

• Versioned files: {s3dir_config}/{parameter_name}/{parameter_name}-999999-1.json

• Version numbers are sequential integers (1, 2, 3, …)

• File naming ensures latest appears first in listings

Version Management:

• Uses integer sequence for versions (1, 2, 3, …)

• Latest file contains pointer to current version

• Historical versions preserved as separate files

• Manual cleanup of old versions when needed

File Structure Example: ` s3://bucket/config/myapp/myapp-000000-latest.json  (latest pointer) s3://bucket/config/myapp/myapp-999997-3.json       (version 3) s3://bucket/config/myapp/myapp-999998-2.json       (version 2) s3://bucket/config/myapp/myapp-999999-1.json       (version 1) `

After_param s3dir_config:

S3 directory where the parameter is stored (no filename)

After_param parameter_name:

Parameter name used as the base filename

Note

This class only works with versioning-disabled S3 buckets. The bucket must have versioning disabled before using this class.

property s3dir_param: S3Path

S3 directory containing all files for this parameter.

get_s3path(version: int | None = None) → S3Path

Get the S3 path for a specific version of the parameter.

For versioning-disabled buckets, this creates different filenames based on the version number. The filename encoding ensures that the latest version appears first in S3 object listings.

After_param version:

Version number (1, 2, 3, …), None for latest

Returns:

S3Path for the versioned file

File naming pattern:

• Latest: {parameter_name}-000000-latest.json

• Version 1: {parameter_name}-999999-1.json

• Version 2: {parameter_name}-999998-2.json

write(s3_client: S3Client, value: str, version: int | None = None, write_text_kwargs: dict[str, Any] | None = None) → S3Path

Write configuration data to S3 with custom versioning.

After_param s3_client:

S3Client for S3 operations

After_param value:

Configuration data as JSON string

After_param version:

Version number for metadata and filename

After_param write_text_kwargs:

Additional arguments for S3 write operation

Returns:

S3Path of the written object

read(s3_client: S3Client, version: int | None = None, read_text_kwargs: dict[str, Any] | None = None) → str

Read configuration data from S3.

Reads configuration data from the versioning-disabled S3 bucket using custom file naming conventions. Can read either the latest version or a specific version number.

After_param s3_client:

S3Client for S3 operations

After_param version:

Version number (1, 2, 3, …) or None for latest

After_param read_text_kwargs:

Additional arguments for S3 read operation

Returns:

Configuration data as JSON string

delete(s3_client: S3Client, version: int | None = None) → S3Path

Delete a specific version of the configuration file from S3.

After_param s3_client:

S3Client for S3 operations

After_param version:

Version number (1, 2, 3, …) or None for latest

Returns:

S3Path of the deleted object

delete_all(s3_client: S3Client)

Delete all configuration files for this parameter from S3.

Removes the entire parameter directory and all versioned files. This operation cannot be undone.

After_param s3_client:

S3Client for S3 operations

delete_last(s3_client: S3Client, keep_last_n: int = 10, purge_older_than_secs: int = 7776000)

Delete old configuration files based on retention policy.

Cleans up old versioned files while preserving recent ones. Files are deleted only if they exceed both the count limit and age limit.

After_param s3_client:

S3Client for S3 operations

After_param keep_last_n:

Minimum number of files to keep (default: 10)

After_param purge_older_than_secs:

Delete files older than this (default: 90 days)