btay.io/wiki

AWS CLI: S3 & CloudFront

Everyday S3 and CloudFront commands — sync, invalidate, presign, cache headers.

Updated 1 mo ago

Lookup for the aws s3 / aws cloudfront commands worth not re-googling. Replace BUCKET and EXXXXXXXXXXXXX (distribution id) with your own.

S3 — files & buckets

TaskCommand
List bucketsaws s3 ls
List a prefixaws s3 ls s3://BUCKET/path/
Copy up / downaws s3 cp file s3://BUCKET/key · aws s3 cp s3://BUCKET/key .
Copy a treeaws s3 cp ./dir s3://BUCKET/dir --recursive
Move / renameaws s3 mv s3://BUCKET/a s3://BUCKET/b
Deleteaws s3 rm s3://BUCKET/key · add --recursive for a prefix
Make / remove bucketaws s3 mb s3://BUCKET · aws s3 rb s3://BUCKET --force
Total size of a prefixaws s3 ls s3://BUCKET/path/ --recursive --summarize --human-readable

S3 sync (the workhorse)

# Mirror a local dir to S3, deleting remote files that no longer exist locally
aws s3 sync ./dist s3://BUCKET --delete

# Preview without changing anything
aws s3 sync ./dist s3://BUCKET --delete --dryrun
FlagEffect
--deleteRemove dest files missing from source (respects --exclude)
--dryrunShow what would change; transfer nothing
--exclude "*" --include "*.js"Filter set — order matters, later rules win
--cache-control "..."Set Cache-Control on uploaded objects
--content-type "..."Override the guessed MIME type
--size-onlyCompare by size only (ignore mtime)
--exact-timestampsTreat any mtime difference as changed (downloads)

Static site deploy (two-pass cache)

Long-cache the fingerprinted assets, never-cache the HTML, then invalidate. --excluded files are not deleted, so the --delete on pass 1 leaves the HTML alone.

# 1) hashed assets — immutable, cache forever
aws s3 sync ./out s3://BUCKET --delete \
  --cache-control "public,max-age=31536000,immutable" \
  --exclude "*.html"

# 2) HTML — always revalidate
aws s3 sync ./out s3://BUCKET \
  --cache-control "public,max-age=0,must-revalidate" \
  --exclude "*" --include "*.html"

# 3) drop the CDN cache
aws cloudfront create-invalidation --distribution-id EXXXXXXXXXXXXX --paths "/*"

Presigned URLs

# Temporary download link (default 3600s, max 7 days)
aws s3 presign s3://BUCKET/key --expires-in 600

CloudFront

TaskCommand
Invalidate everythingaws cloudfront create-invalidation --distribution-id EXXX --paths "/*"
Invalidate specific paths... --paths "/index.html" "/assets/*"
Wait until it finishesaws cloudfront wait invalidation-completed --distribution-id EXXX --id IXXX
List distributions (readable)see snippet below
Inspect oneaws cloudfront get-distribution --id EXXX
# Map distribution id → domain → origin, as a table
aws cloudfront list-distributions \
  --query "DistributionList.Items[].{Id:Id,Domain:DomainName,Origin:Origins.Items[0].DomainName}" \
  --output table

Global flags worth remembering

FlagUse
--profile NAMEPick a named credentials profile
--region REGIONOverride the default region
--output json|table|textResponse format
--query '...'JMESPath filter to pull out fields
--no-cli-pagerDon't pipe output through a pager

Gotchas

  • aws s3 (high-level) auto-handles multipart and recursion; aws s3api is the low-level surface for things like bucket policies (put-bucket-policy --policy file://policy.json).
  • Invalidations are billed after the first 1,000 paths/month — prefer targeted paths over /* in high-frequency deploys.
  • --acl public-read only works if the bucket allows ACLs; modern buckets default to Bucket owner enforced (ACLs disabled) — use a bucket policy or OAC instead.
  • Setting --cache-control on sync only applies to objects it uploads; unchanged objects keep their old headers (re-upload or use s3api copy-object to change metadata).

Related pages

On this page