Load balancers & auto scaling

Load balancers and auto scaling: spread the load, then grow the fleet

Section 1 of 5 · ~3 min

One server is both a single point of failure and a hard ceiling: when it dies your site is down, and when traffic doubles it falls over. Two AWS building blocks fix that together — a load balancer spreads requests across many servers, and an Auto Scaling Group changes how many servers there are. Yesterday’s single EC2 box becomes a fleet that survives a failure and grows with demand.

The load balancer is a managed front door that receives every request and hands it to a healthy backend. On AWS it’s the Elastic Load Balancer, and you pick between two types:

• ALB (Application Load Balancer) — works at layer 7 (HTTP/HTTPS). It can route on hostname, URL path, or headers (/api/* to one group, everything else to another). It’s the default for web apps.
• NLB (Network Load Balancer) — works at layer 4 (TCP/UDP). Extremely fast, gives you static IPs, and handles millions of connections. Reach for it for non-HTTP protocols or extreme throughput.

Both forward to a target group: the list of backends (instances, IPs, or Lambdas) plus a health check. The load balancer repeatedly requests a path you choose — say GET / — and only routes traffic to targets that answer 200. The moment a target starts failing, it’s pulled out of rotation automatically and put back when it recovers. That’s what turns a fleet into something that survives one box dying.

Auto Scaling Groups: a fleet that resizes itself

A launch template is the recipe for one instance — AMI, instance type, key pair, security group, and a user-data script. An Auto Scaling Group (ASG) uses that template to keep a desired number of instances running across multiple AZs, between a min and a max. If an instance dies or fails its health check, the ASG launches a replacement without you touching anything.

The ASG earns its name through a scaling policy. The common one is target tracking: “keep average CPU at 50%.” When traffic rises and CPU climbs, the ASG adds instances; when it falls, it removes them — and it registers each new instance into the target group automatically, so the load balancer always knows the current fleet. You set the goal, not the instance count.

Real world: Think of a supermarket. The load balancer is the greeter directing each shopper to whichever till is open — and skipping any till whose light is off (a failed health check). The Auto Scaling Group is the manager who opens more tills when a queue builds at rush hour and closes them when it goes quiet. Neither counts shoppers by hand; both just react.

Netflix made this pattern famous: its open-source deploy tool Asgard was built entirely around Auto Scaling Groups as the unit of deployment, launching and retiring instances behind load balancers thousands of times a day. Today you build a miniature of the same thing — an ALB in front of an ASG of two tiny instances — then tear it all down.

Hands-On Lab

Section 2 of 5 · ~6 min

Budget about 30 minutes, driven from the AWS CLI v2 as your Day 47 IAM user (not root), in us-east-1. Work top to bottom and read each output — and don’t wander off before the teardown, because one resource here bills by the hour.

What this costs: Not ₹0 — an Application Load Balancer is not free-tier. It bills roughly ₹1.7/hour (about ₹1,300/month) plus a small capacity charge, so this lab costs a few rupees for the ~20 minutes it runs, then drops back to ₹0 the moment you delete it. The two t2.micro targets the Auto Scaling Group launches are free-tier eligible. So: finish the whole lab in one sitting, and run the teardown (steps 11–12) — an ALB left running overnight is a real, silent charge. Account IDs, ARNs, IPs and DNS names below are examples — yours will differ.

# 1. Confirm you're the Day 47 IAM user (not root), set the region, and grab your
#    default VPC plus two subnets in different AZs (an ALB spans >=2 AZs).
export AWS_DEFAULT_REGION=us-east-1
aws sts get-caller-identity --query 'Arn' --output text
VPC=$(aws ec2 describe-vpcs --filters Name=is-default,Values=true \
  --query 'Vpcs[0].VpcId' --output text)
read SUBNET1 SUBNET2 _ < <(aws ec2 describe-subnets \
  --filters Name=vpc-id,Values=$VPC --query 'Subnets[].SubnetId' --output text)
echo "$VPC $SUBNET1 $SUBNET2"
# Output (yours will differ):
# arn:aws:iam::123456789012:user/devops-you
# vpc-0a1b2c3d4e subnet-0aaa111 subnet-0bbb222

# 2. Get the latest Ubuntu 24.04 AMI ID from the public SSM parameter (no hard-coding).
AMI=$(aws ssm get-parameters \
  --names /aws/service/canonical/ubuntu/server/24.04/stable/current/amd64/hvm/ebs-gp3/ami-id \
  --query 'Parameters[0].Value' --output text)
echo "$AMI"
# Output (changes as Canonical publishes new builds):
# ami-0abcd1234ef567890

# 3. One security group for the ALB and the instances. Allow HTTP:80 from YOUR IP
#    (a real public site would allow 0.0.0.0/0 — we scope it for a throwaway demo),
#    then allow the group to reach itself so the ALB can hit its targets on :80.
MY_IP=$(curl -s https://checkip.amazonaws.com)
SG=$(aws ec2 create-security-group --group-name m90-alb-sg \
  --description "ALB + web demo" --vpc-id "$VPC" --query 'GroupId' --output text)
aws ec2 authorize-security-group-ingress --group-id "$SG" \
  --protocol tcp --port 80 --cidr "${MY_IP}/32"
aws ec2 authorize-security-group-ingress --group-id "$SG" \
  --protocol tcp --port 80 --source-group "$SG"
echo "$SG"
# Output (this GroupId is yours):
# sg-0abc123def4567890

# 4. A launch template is the recipe for one instance: AMI, size, SG, and a
#    user-data script that installs a tiny web server showing the hostname.
USERDATA=$(base64 -w0 <<'EOF'
#!/bin/bash
apt-get update -y && apt-get install -y nginx
echo "Hello from $(hostname -f)" > /var/www/html/index.html
EOF
)
aws ec2 create-launch-template --launch-template-name m90-web \
  --launch-template-data "{\"ImageId\":\"$AMI\",\"InstanceType\":\"t2.micro\",\"SecurityGroupIds\":[\"$SG\"],\"UserData\":\"$USERDATA\"}" \
  --query 'LaunchTemplate.LaunchTemplateId' --output text
# Output (yours will differ):
# lt-0abc123def4567890

# 5. Create a target group: HTTP:80 in your VPC, health-checking GET / .
TG=$(aws elbv2 create-target-group --name m90-tg \
  --protocol HTTP --port 80 --vpc-id "$VPC" --health-check-path / \
  --query 'TargetGroups[0].TargetGroupArn' --output text)
echo "$TG"
# Output (this ARN is yours):
# arn:aws:elasticloadbalancing:us-east-1:123456789012:targetgroup/m90-tg/abc123

# 6. Create the ALB across your TWO subnets (an ALB needs >=2 AZs), add an HTTP:80
#    listener forwarding to the target group, then read the ALB's DNS name.
ALB=$(aws elbv2 create-load-balancer --name m90-alb \
  --subnets "$SUBNET1" "$SUBNET2" --security-groups "$SG" \
  --query 'LoadBalancers[0].LoadBalancerArn' --output text)
LISTENER=$(aws elbv2 create-listener --load-balancer-arn "$ALB" \
  --protocol HTTP --port 80 \
  --default-actions Type=forward,TargetGroupArn="$TG" \
  --query 'Listeners[0].ListenerArn' --output text)
ALB_DNS=$(aws elbv2 describe-load-balancers --load-balancer-arns "$ALB" \
  --query 'LoadBalancers[0].DNSName' --output text)
echo "$ALB_DNS"
# Output (the ALB DNS name — yours will differ):
# m90-alb-1234567890.us-east-1.elb.amazonaws.com

# 7. Create the Auto Scaling Group from the launch template: keep 2 instances
#    (max 4) across both subnets, and register them into the target group.
aws autoscaling create-auto-scaling-group \
  --auto-scaling-group-name m90-asg \
  --launch-template LaunchTemplateName=m90-web \
  --min-size 2 --max-size 4 --desired-capacity 2 \
  --vpc-zone-identifier "$SUBNET1,$SUBNET2" --target-group-arns "$TG"
# Output: (no output on success — the ASG launches 2 t2.micro targets)

# 8. Add a target-tracking scaling policy: keep average CPU at 50%, so the ASG
#    adds instances when busy and removes them when idle — no manual tuning.
aws autoscaling put-scaling-policy \
  --auto-scaling-group-name m90-asg \
  --policy-name cpu50 --policy-type TargetTrackingScaling \
  --target-tracking-configuration '{"PredefinedMetricSpecification":{"PredefinedMetricType":"ASGAverageCPUUtilization"},"TargetValue":50.0}' \
  --query 'PolicyARN' --output text
# Output (yours will differ):
# arn:aws:autoscaling:us-east-1:123456789012:scalingPolicy:.../policyName/cpu50

# 9. Give the instances ~2-3 min to boot and install nginx, then check target
#    health — both must read "healthy" before the ALB will route to them.
aws elbv2 describe-target-health --target-group-arn "$TG" \
  --query 'TargetHealthDescriptions[].TargetHealth.State' --output text
# Output (after they pass the health check — retry if you still see "initial"):
# healthy	healthy

# 10. Hit the ALB a few times — it spreads requests across both targets, so the
#     hostname in the response alternates between the two instances.
for i in 1 2 3 4; do curl -s "http://$ALB_DNS/"; done
# Output (two different hostnames, load-balanced — yours will differ):
# Hello from ip-172-31-1-10.ec2.internal
# Hello from ip-172-31-5-22.ec2.internal
# Hello from ip-172-31-1-10.ec2.internal
# Hello from ip-172-31-5-22.ec2.internal

# 11. Teardown part 1 — grab the ASG's instance IDs, force-delete the ASG (which
#     terminates them), then WAIT for those instances to fully terminate — until
#     they do, they still hold the shared SG and step 12 can't delete it.
IDS=$(aws autoscaling describe-auto-scaling-groups \
  --auto-scaling-group-names m90-asg \
  --query 'AutoScalingGroups[0].Instances[].InstanceId' --output text)
aws autoscaling delete-auto-scaling-group \
  --auto-scaling-group-name m90-asg --force-delete
aws ec2 wait instance-terminated --instance-ids $IDS   # ~1-3 min
aws ec2 delete-launch-template --launch-template-name m90-web \
  --query 'LaunchTemplate.LaunchTemplateName' --output text
# Output:
# m90-web

# 12. Teardown part 2 — delete the listener, the ALB (the billable part!), the
#     target group, then the security group. Order matters: listener -> ALB -> TG.
aws elbv2 delete-listener --listener-arn "$LISTENER"
aws elbv2 delete-load-balancer --load-balancer-arn "$ALB"
# The ALB deletes asynchronously: its ENIs take a minute or two to detach, and
# until they do the target group is still "in use" and the SG can't be deleted
# (the instances from step 11 also held it). So retry each delete until it
# succeeds instead of guessing a single sleep.
until aws elbv2 delete-target-group --target-group-arn "$TG" 2>/dev/null; do
  echo "target group still in use (ALB draining) — retrying in 20s..."; sleep 20
done
until aws ec2 delete-security-group --group-id "$SG" 2>/dev/null; do
  echo "SG still in use (ALB ENIs draining) — retrying in 20s..."; sleep 20
done
echo "cleaned up — the ALB is gone, billing stops"
# Output:
# cleaned up — the ALB is gone, billing stops

Read the last outputs back: the two hostnames in step 10 prove the ALB is actually spreading load, and the clean teardown in steps 11–12 is what returns the bill to ₹0. The ALB is the one resource this week that bills by the hour whether or not you use it — deleting it promptly is the whole discipline.

Common Errors & Fixes

Section 3 of 5 · ~3 min

These three catch almost everyone standing up their first load balancer. Read the error text slowly — parsing it is the actual skill.

Common error: Creating an ALB with subnets that don’t span two Availability Zones:

An error occurred (ValidationError) when calling the CreateLoadBalancer operation: At least two subnets in two different Availability Zones must be specified

Why: An ALB is a highly available service by design — AWS places a node in each AZ you give it, so it requires at least two subnets in two different Availability Zones. Passing one subnet, or two subnets that happen to sit in the same AZ, fails this check.

Fix: Pass two subnet IDs from different AZs (step 1 grabs two from your default VPC, which has one subnet per AZ). Confirm their zones with aws ec2 describe-subnets --query 'Subnets[].AvailabilityZone' if you’re unsure.

How you’d spot it in prod: A Terraform apply or CLI call failing with “two different Availability Zones” means the subnet list is wrong or too narrow — usually a module wired to a single-AZ subnet group. Widen it; don’t work around it, because a single-AZ load balancer defeats its own purpose.

Common error: Targets never turn healthy, so the ALB answers 503 Service Unavailable:

"State": "unhealthy",
"Reason": "Target.Timeout",
"Description": "Request timed out"

Why: The load balancer can’t get a good response from the target on the health-check port. The two usual causes: the security group doesn’t allow the ALB to reach the instance on port 80, or the app isn’t listening yet / returns a non-200 on the health-check path. A brand-new instance also shows initial while user-data is still installing nginx.

Fix: Make sure the target group’s security group allows inbound on the health-check port from the ALB (this lab’s self-referencing SG rule does exactly that), confirm the app answers 200 on the path, and give a fresh instance a couple of minutes to finish booting before you judge it.

How you’d spot it in prod: A 503 from an ALB with all targets unhealthy is almost always a health-check or security-group mismatch, not an application outage — check target health and the health-check path first, before you touch the app.

Common error: Deleting the target group or security group before the ALB that still references it:

An error occurred (ResourceInUse) when calling the DeleteTargetGroup operation: Target group '...' is currently in use by a listener or a rule

Why: AWS refuses to delete a resource something still points at. The listener forwards to the target group, and the ALB uses the security group, so you can’t remove the child while the parent references it — the same protection you saw deleting an IAM user with an attached policy. It also surfaces as DependencyViolation when you delete the SG too soon.

Fix: Delete in dependency order — listener, then load balancer, then target group, then security group (step 12 does exactly this). The ALB deletes asynchronously and its ENIs take a minute or two to detach, so the target group and SG deletes may still return ResourceInUse/DependencyViolation for the first attempt or two — step 12 simply retries each until it succeeds rather than guessing a fixed sleep.

How you’d spot it in prod: ResourceInUse or DependencyViolation during a teardown or terraform destroy is an ordering problem, not a stuck resource — remove the referencing parent first, then the child, rather than reaching for a force-delete.

Load Balancing & Auto Scaling Interview Questions

Section 4 of 5 · ~1 min

Cover the answers below and say your own version out loud first — name what an ALB does that an NLB doesn’t, and what an ASG gives you over a launch template alone, before you reveal each answer. Recalling before revealing is what makes these stick when an interviewer asks them cold. The four questions and answers render right after this note.

Go Deeper

Section 5 of 5 · ~1 min

Optional extras if you have ~30 more minutes today:

• 10 min — Turn this into a real public ALB: change the SG to allow port 80 from 0.0.0.0/0, add a second target group, and create a path-based listener rule so /api/* routes to a different fleet. Read the ALB “listener rules” doc to see how layer-7 routing works.
• 10 min — Read the ALB vs NLB vs Gateway Load Balancer comparison and note exactly when an NLB wins: static IPs, TLS passthrough, UDP, or raw throughput an ALB can’t match. Knowing the boundary is the interview answer.
• 10 min — Add load and watch the ASG scale: SSH to a target, run a CPU burner like yes > /dev/null, and watch aws autoscaling describe-auto-scaling-groups add instances as the target-tracking policy fires — then read about cooldowns and instance warm-up.