S3 / S3-compatible (kind: s3)

Federate a bucket (AWS S3, MinIO, Ceph RGW, …) as a read-only catalog source. Objects stay private; every fetch is freshly signed and streamed. Zero-dep SigV4 - no AWS SDK.

What you need from the store

• A read-only access key pair scoped to the bucket (and prefix, if set). On AWS, an IAM user or role with a minimal policy:
  {
    "Version": "2012-10-17",
    "Statement": [{
      "Effect": "Allow",
      "Action": ["s3:GetObject", "s3:ListBucket"],
      "Resource": ["arn:aws:s3:::acme-brand", "arn:aws:s3:::acme-brand/brand/*"]
    }]
  }

On MinIO/Ceph: a read-only user/policy over the bucket. Do not grant write/delete - federation never writes.

• The bucket name, the region (SigV4 needs one even where MinIO ignores it), and for an S3-compatible store the endpoint URL.

Credential shape

One string, accessKeyId and secretAccessKey joined by a colon:

"<accessKeyId>:<secretAccessKey>"

lw providers credential acme-s3     # prompts for the "<key>:<secret>" string

instance.json / lw providers add

{
  "id": "acme-s3",
  "kind": "s3",
  "label": "Acme MinIO",
  "options": { "bucket": "acme-brand", "region": "us-east-1", "endpoint": "https://minio.internal:9000", "prefix": "brand" },
  "exposure": { "groups": ["design"], "tier": "reference" }
}

• options.bucket (required); region (default us-east-1); endpoint for S3-compatibles (absent → virtual-hosted AWS); prefix federates only keys under it (also maps prefix "folders" → sections for exposure scoping).

Verify

lw providers preview --kind s3 --options '{"bucket":"acme-brand","region":"us-east-1"}'
lw providers health acme-s3

403 AccessDenied on the ListBucket → the policy is missing s3:ListBucket on the bucket ARN (distinct from the /* object ARN). A SignatureDoesNotMatch → wrong secret/region.

Notes / limits

• Every request is freshly signed (expiringUrls semantics); presigned URLs are never minted into the feed. resolveBlob refuses an object outside the configured prefix.
• Supports the exit (materialize → cutover). Note this is the federation s3 kind - distinct from the instance's own blobs.driver: "s3" BlobStore, which is where materialized bytes land (its credential is env-only, LW_BLOBS_S3_CREDENTIAL).
• Does not accept published exports.

See also: catalog · permissions.