S3 — buckets, policies, static hosting, lifecycle

S3: object storage, buckets, and who can read them

Section 1 of 5 · ~3 min

Amazon S3 (Simple Storage Service) is the oldest AWS service and the one almost every other service leans on. It is object storage: you store a whole file — an object — under a key, inside a container called a bucket, and read or write it over HTTP. There are no disks to format and no size to pre-allocate; a bucket holds one object or a billion, each up to 5 TB, and AWS keeps them safe behind the scenes (S3 is designed for eleven nines — 99.999999999% — of durability).

Object storage is not a file system. The key logs/2026/app.log looks like a path, but the slashes are just naming — the namespace is flat, there are no real directories, and you cannot edit part of an object in place. To change one byte you upload the whole object again. That trade — no in-place edits, but limitless scale and any number of readers at once over the network — is what makes S3 right for backups, static assets, logs and data lakes, and wrong for a database’s live files.

Buckets and keys. A bucket name is globally unique across all of AWS, lives in one region, and holds objects addressed by key. The core operations are what you’d guess — put, get, list, delete — which the CLI wraps as aws s3 cp, aws s3 ls and aws s3 rb.

Who can read them: Block Public Access + policies. Every new bucket is private, and Block Public Access (BPA) is on — a master switch, at account and bucket level, that overrides any policy trying to make objects public. That default exists because misconfigured public buckets leaked data for years. A bucket policy is a JSON document (the same grammar as IAM) saying who may do which s3: actions on which objects. To serve a public site you must both turn BPA off and attach a policy granting s3:GetObject to everyone — two deliberate steps, never an accident.

Static website hosting. Enable website hosting, name an index.html, and the bucket serves your HTML, CSS and JS straight over an endpoint URL — no server, no EC2. In production you’d put CloudFront in front for HTTPS and caching.

Lifecycle rules let a bucket manage its own aging: a rule can transition objects to a cheaper storage class (Standard → Standard-IA → Glacier) after N days, or expire — delete — them entirely, so ninety-day-old logs move to cold storage or vanish without a cron job.

Real world: A bucket is a self-storage warehouse. Each object is a labelled box; the key is the label you find it by. Block Public Access is the locked front gate that stays shut whatever any single unit’s key says — you have to deliberately unlock the gate and hand out a unit key before anyone off the street can walk in. Lifecycle rules are the manager who quietly moves boxes you haven’t touched in months to the cheaper back room, then to the shredder.

Netflix stores its entire encoded-video catalog in S3 as the source of truth, streaming petabytes from it every day — the same put, get and list calls you’ll run by hand today, just at planetary scale.

Today you create a bucket, upload and list objects, write a policy, host a static page, add a lifecycle rule, then empty and delete the bucket so it costs nothing.

Hands-On Lab

Section 2 of 5 · ~3 min

Budget about 25 minutes. Drive this from the AWS CLI v2 as your Day 47 IAM user (not root), in the us-east-1 region — stay consistent so the website endpoint matches. Type every command and read the output before moving on. Bucket names, timestamps and endpoints below are examples — yours will differ.

What this costs: ₹0 if you delete the bucket at the end. On older accounts the S3 free tier gave 5 GB of Standard storage plus 20,000 GET and 2,000 PUT requests a month for the first 12 months; accounts created after mid-July 2025 instead get a credit-based free tier that covers small experiments like this the same way. Either way a few tiny objects cost effectively nothing. Storage and requests bill by the GB and per thousand, so the only way this charges you is leaving gigabytes of objects — or a forgotten bucket — behind. Steps 11–12 empty and delete the bucket, so finish the lab; an emptied, deleted bucket costs nothing.

# 1. Confirm you're your Day 47 IAM user (NOT root) and pin the region.
aws sts get-caller-identity
export AWS_DEFAULT_REGION=us-east-1
# Output (your Account and Arn will differ):
# {
#     "UserId": "AIDA...EXAMPLE",
#     "Account": "123456789012",
#     "Arn": "arn:aws:iam::123456789012:user/devops-you"
# }

# 2. Bucket names are globally unique — a timestamp makes yours available.
BUCKET="m90-s3-lab-$(date +%s)"
aws s3 mb "s3://$BUCKET"
echo "$BUCKET"
# Output (your bucket name will differ):
# make_bucket: m90-s3-lab-1752230400
# m90-s3-lab-1752230400

# 3. Make two files and copy (upload) them into the bucket.
echo "<h1>Hello from S3</h1>" > index.html
echo "hello object storage" > notes.txt
aws s3 cp index.html "s3://$BUCKET/"
aws s3 cp notes.txt  "s3://$BUCKET/"
# Output:
# upload: ./index.html to s3://m90-s3-lab-1752230400/index.html
# upload: ./notes.txt to s3://m90-s3-lab-1752230400/notes.txt

# 4. List what's in the bucket — the two objects you just uploaded.
aws s3 ls "s3://$BUCKET/"
# Output (size and timestamp are yours):
# 2026-07-11 10:14:02         23 index.html
# 2026-07-11 10:14:03         21 notes.txt

# 5. Every new bucket is private — Block Public Access is ON by default.
aws s3api get-public-access-block --bucket "$BUCKET" \
  --query 'PublicAccessBlockConfiguration'
# Output — all four switches true means nothing is public:
# {
#     "BlockPublicAcls": true,
#     "IgnorePublicAcls": true,
#     "BlockPublicPolicy": true,
#     "RestrictPublicBuckets": true
# }

# 6. To serve a public site you must FIRST turn Block Public Access off — deliberately.
aws s3api put-public-access-block --bucket "$BUCKET" \
  --public-access-block-configuration \
  "BlockPublicAcls=false,IgnorePublicAcls=false,BlockPublicPolicy=false,RestrictPublicBuckets=false"
# Output: (no output on success — the gate is now unlocked)

# 7. Grant everyone read-only GetObject — the policy half of "make it public".
aws s3api put-bucket-policy --bucket "$BUCKET" --policy "{
  \"Version\": \"2012-10-17\",
  \"Statement\": [{
    \"Sid\": \"PublicReadGetObject\",
    \"Effect\": \"Allow\",
    \"Principal\": \"*\",
    \"Action\": \"s3:GetObject\",
    \"Resource\": \"arn:aws:s3:::$BUCKET/*\"
  }]
}"
# Output: (no output on success)

# 8. Turn on static website hosting with index.html as the home page.
aws s3 website "s3://$BUCKET/" --index-document index.html
# Output: (no output on success)

# 9. Fetch the site over its regional website endpoint — no server, no EC2.
curl "http://$BUCKET.s3-website-us-east-1.amazonaws.com"
# Output:
# <h1>Hello from S3</h1>

# 10. Add a lifecycle rule: expire (delete) every object 90 days after upload.
aws s3api put-bucket-lifecycle-configuration --bucket "$BUCKET" \
  --lifecycle-configuration '{
    "Rules": [{
      "ID": "expire-90d",
      "Status": "Enabled",
      "Filter": {"Prefix": ""},
      "Expiration": {"Days": 90}
    }]
  }'
# Output: (no output on success — confirm with: aws s3api get-bucket-lifecycle-configuration --bucket "$BUCKET")

# 11. TEARDOWN — empty the bucket, then delete it. A bucket must be empty before rb.
aws s3 rm "s3://$BUCKET/" --recursive
aws s3 rb "s3://$BUCKET"
rm -f index.html notes.txt
# Output:
# delete: s3://m90-s3-lab-1752230400/index.html
# delete: s3://m90-s3-lab-1752230400/notes.txt
# remove_bucket: m90-s3-lab-1752230400

# 12. Confirm the bucket is gone — it should NOT appear in your bucket list.
aws s3 ls | grep "$BUCKET" || echo "bucket deleted — nothing left to bill"
# Output:
# bucket deleted — nothing left to bill

Read the last outputs back: remove_bucket in step 11 and the “nothing left to bill” line in step 12 are the ones that matter — an emptied, deleted bucket is the difference between a ₹0 lab and objects quietly billing you by the GB.

Common Errors & Fixes

Section 3 of 5 · ~3 min

These three catch almost everyone on their first bucket. Read the error text slowly — parsing it is the actual skill.

Common error: Reusing a bucket name someone, somewhere, already took:

An error occurred (BucketAlreadyExists) when calling the CreateBucket operation: The requested bucket name is not available. The bucket namespace is shared by all AWS accounts.

Why: S3 bucket names are globally unique across every AWS account on Earth, not just yours — so a plain name like test or backups was claimed years ago. The namespace is shared, so the collision isn’t with your account, it’s with the whole planet.

Fix: Make the name unique — append a timestamp or your account ID (m90-s3-lab-$(date +%s), as in step 2). If you see the sibling error BucketAlreadyOwnedByYou, you already created that exact bucket — just reuse it.

How you’d spot it in prod: A Terraform apply or deploy failing on CreateBucket with BucketAlreadyExists almost always means a non-unique name in the config — add a suffix (environment, region, account ID) rather than retrying the same name.

Common error: Attaching a public bucket policy while Block Public Access is still on:

An error occurred (AccessDenied) when calling the PutBucketPolicy operation: User: arn:aws:iam::123456789012:user/devops-you is not authorized to perform: s3:PutBucketPolicy on resource: "arn:aws:s3:::m90-s3-lab-1752230400" because public policies are blocked by the BlockPublicPolicy block public access setting.

Why: BlockPublicPolicy is doing exactly its job — it refuses any policy that would grant public access, before the policy is ever stored. This is the guardrail that stopped years of accidental data leaks, so the account isn’t broken; the switch is protecting you.

Fix: If you genuinely intend a public site, turn the relevant Block Public Access switches off first (step 6), then put the policy (step 7). If you don’t, leave BPA on — the error means you were about to make a bucket public you didn’t mean to.

How you’d spot it in prod: PutBucketPolicy denied with “blocked by the BlockPublicPolicy” is a policy being rejected on purpose, not a permissions bug. Confirm the bucket is meant to be public before you touch BPA — most buckets should never be.

Common error: Trying to delete a bucket that still has objects in it:

An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: The bucket you tried to delete is not empty.

Why: S3 won’t delete a bucket that still holds objects (or old versions, if versioning is on), so you can’t orphan data or lose track of what was inside. The bucket must be fully emptied first.

Fix: Empty it, then delete: aws s3 rm "s3://$BUCKET/" --recursive before aws s3 rb "s3://$BUCKET" — exactly what step 11 does. On a versioned bucket that’s not enough: aws s3 rb --force (and rm --recursive) only drop current versions, leaving every non-current version and delete marker behind. You must delete each version and delete marker first — list them with aws s3api list-object-versions and remove them with aws s3api delete-objects — and only then will rb succeed.

How you’d spot it in prod: A cleanup or terraform destroy failing with BucketNotEmpty means objects (often versioned ones) remain — empty the contents first rather than forcing the delete, so you know what you’re throwing away.

S3 Interview Questions

Section 4 of 5 · ~1 min

Cover the answers below and say your own version out loud first — name what an object, a bucket, Block Public Access and a lifecycle rule each are before you reveal each answer. Recalling before revealing is what makes these stick when an interviewer asks them cold. The four questions and answers render right after this note.

Go Deeper

Section 5 of 5 · ~1 min

Optional extras if you have ~30 more minutes today:

• 5 min — Skim the S3 storage classes page and note the price-versus-retrieval trade between Standard, Standard-IA, Glacier Instant and Deep Archive — the tiers your lifecycle rule moves objects between.
• 10 min — Turn on bucket versioning (aws s3api put-bucket-versioning), overwrite notes.txt, and list versions — see how S3 keeps every old copy, and why an emptied versioned bucket needs its versions deleted before rb works.
• 15 min — Read how CloudFront fronts a private S3 bucket with origin access control for HTTPS and caching, and generate a presigned URL (aws s3 presign) that grants time-limited access to one object without ever making the bucket public.