Project 1, Day 2: write & optimize the Dockerfiles

A multi-stage Dockerfile for linkstash

Section 1 of 5 · ~2 min

Yesterday you scaffolded linkstash — app/main.py, app/db.py, requirements.txt and the tests. Today you containerize it the way production teams do: a multi-stage build that compiles dependencies in a throwaway builder stage and copies only the finished result into a lean, non-root runtime stage.

Why two stages? linkstash reaches Postgres through psycopg2, a C extension. To build it, pip needs a compiler (build-essential) and the Postgres client headers (libpq-dev) — together roughly 250 MB of toolchain. None of that is needed to run the app: at run time psycopg2 only links the shared library libpq5, a few megabytes. A single-stage image bakes the whole compiler toolchain into the shipped image forever. Multi-stage leaves it on the cutting-room floor.

The key design decision: install into a virtualenv in the builder, then copy just that venv. The builder installs build-essential + libpq-dev, creates /opt/venv, and pip installs your requirements into it. The final stage starts fresh from python:3.12-slim, installs only the runtime libpq5, then COPY --from=builder /opt/venv /opt/venv. Put /opt/venv/bin on PATH and the app runs against exactly the packages you built — with zero compilers, headers, or pip caches in the shipped layers.

Two hardening habits ship today too. Run as non-root: create an unprivileged app user and end with USER app, so a container breakout doesn’t hand an attacker root inside the image. And EXPOSE 8000 documents the port while CMD launches uvicorn on 0.0.0.0:8000 — binding all interfaces so traffic from outside the container actually reaches the app.

Real world: Think of building a house. Scaffolding, cement mixers and power tools crowd the site while the walls go up — but you don’t hand them to the new owner. You strike the scaffolding and deliver a clean, empty house. The builder stage is your scaffolding; the runtime image is the house you ship — everything used to construct it, left behind.

Google takes this idea to its limit with its distroless base images, which strip out even the shell and package manager, leaving only your app and its language runtime: less to download, a smaller attack surface, and nothing for an intruder who lands inside to pivot with. You are not going distroless today, but python:3.12-slim plus a copied venv is the same instinct — ship only what runs. Build it below, then prove the win: the fat builder stage next to the slim image you actually run.

Hands-On Lab

Section 2 of 5 · ~4 min

Budget about 30 minutes. Open your WSL2 Ubuntu 24.04 terminal with Docker 27+ installed, in the linkstash repo you scaffolded on Day 1. You’ll write the multi-stage Dockerfile and a .dockerignore, build the slim image, expose the fat builder stage next to it to see the size win, then run the container and curl the health check. Image IDs, digests, sizes and build timings are version-variant — yours will differ from the samples.

# 1. In the linkstash repo from Day 1, confirm the app and requirements are present.
cd ~/linkstash && ls
cat requirements.txt
# Output:
# app  requirements-dev.txt  requirements.txt  tests
# fastapi==0.115.0
# uvicorn[standard]==0.30.6
# sqlalchemy==2.0.35
# psycopg2==2.9.9

# 2. Write the multi-stage Dockerfile: builder compiles deps into a venv; runtime ships only the venv.
cat > Dockerfile <<'EOF'
# syntax=docker/dockerfile:1

# ---------- builder: compile deps into a venv ----------
FROM python:3.12-slim AS builder
# psycopg2 is built from source, so the builder needs a C toolchain + libpq headers.
RUN apt-get update && apt-get install -y --no-install-recommends \
        build-essential libpq-dev \
    && rm -rf /var/lib/apt/lists/*
ENV PATH="/opt/venv/bin:$PATH"
RUN python -m venv /opt/venv
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# ---------- runtime: slim, non-root, no compilers ----------
FROM python:3.12-slim
# Only the shared library psycopg2 links at run time — not the -dev headers or gcc.
RUN apt-get update && apt-get install -y --no-install-recommends libpq5 \
    && rm -rf /var/lib/apt/lists/*
RUN useradd --create-home --uid 1000 app
ENV PATH="/opt/venv/bin:$PATH" \
    PYTHONUNBUFFERED=1
WORKDIR /app
COPY --from=builder /opt/venv /opt/venv
COPY app ./app
USER app
EXPOSE 8000
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]
EOF
head -5 Dockerfile
# Output:
# # syntax=docker/dockerfile:1
#
# # ---------- builder: compile deps into a venv ----------
# FROM python:3.12-slim AS builder
# # psycopg2 is built from source, so the builder needs a C toolchain + libpq headers.

# 3. Keep the build context lean: never ship git, the local venv, caches, tests or docs.
cat > .dockerignore <<'EOF'
.git
.venv
__pycache__/
*.pyc
.pytest_cache/
tests/
*.md
.env
EOF
cat .dockerignore
# Output:
# .git
# .venv
# __pycache__/
# *.pyc
# .pytest_cache/
# tests/
# *.md
# .env

# 4. Build the slim runtime image. Watch the builder stage run, then the runtime stage assemble.
docker build -t linkstash:dev .
# Output (BuildKit; digests, image IDs and timings are yours-will-differ):
# [+] Building 41.6s (18/18) FINISHED                              docker:default
#  => [internal] load build definition from Dockerfile                       0.0s
#  => [internal] load metadata for docker.io/library/python:3.12-slim        0.9s
#  => [internal] load .dockerignore                                          0.0s
#  => [internal] load build context                                          0.0s
#  => => transferring context: 6.14kB                                        0.0s
#  => [builder 1/6] FROM docker.io/library/python:3.12-slim@sha256:<digest>  2.7s
#  => [builder 2/6] RUN apt-get update && apt-get install -y ... build-essential libpq-dev   22.4s
#  => [builder 3/6] RUN python -m venv /opt/venv                             2.1s
#  => [builder 4/6] WORKDIR /app                                             0.0s
#  => [builder 5/6] COPY requirements.txt .                                  0.0s
#  => [builder 6/6] RUN pip install --no-cache-dir -r requirements.txt       7.8s
#  => CACHED [stage-1 1/6] FROM docker.io/library/python:3.12-slim@sha256:<digest>  0.0s
#  => [stage-1 2/6] RUN apt-get update && apt-get install -y ... libpq5       4.3s
#  => [stage-1 3/6] RUN useradd --create-home --uid 1000 app                 0.3s
#  => [stage-1 4/6] WORKDIR /app                                             0.0s
#  => [stage-1 5/6] COPY --from=builder /opt/venv /opt/venv                  0.4s
#  => [stage-1 6/6] COPY app ./app                                           0.0s
#  => exporting to image                                                     0.6s
#  => => naming to docker.io/library/linkstash:dev                           0.0s

# 5. Build ONLY the builder stage into its own tag so you can see the fat intermediate.
docker build --target builder -t linkstash:builder .
# Output (every step is reused from the build above — seconds, all CACHED):
# [+] Building 0.8s (12/12) FINISHED                              docker:default
#  => CACHED [builder 1/6] FROM docker.io/library/python:3.12-slim@sha256:<digest>  0.0s
#  => CACHED [builder 6/6] RUN pip install --no-cache-dir -r requirements.txt       0.0s
#  => => naming to docker.io/library/linkstash:builder                        0.0s

# 6. Compare the two side by side — this is the multi-stage win, in one column. (SIZE is yours-will-differ.)
docker images linkstash
# Output:
# REPOSITORY   TAG       IMAGE ID       CREATED         SIZE
# linkstash    builder   9a3f1c8be2d1   1 minute ago    486MB
# linkstash    dev       1c8e4d9a2b73   1 minute ago    214MB

# 7. Run the slim image, publishing container port 8000 as host 8000.
#    Pass DATABASE_URL for config only — /healthz doesn't touch Postgres (that arrives Day 3).
docker run -d -p 8000:8000 --name linkstash \
  -e DATABASE_URL=postgresql://postgres:postgres@db:5432/linkstash \
  linkstash:dev
# Output (the container ID is yours-will-differ):
# c4e1f9a2b7d83e60c5a8d2b4f1097e3c2d5a8b4e0f3c9d17a2b6e5c8d4f0a19b

# 8. Hit the health check through the published port.
curl -s http://localhost:8000/healthz
# Output:
# {"status":"ok"}

# 9. Prove the container runs as the non-root app user, not root.
docker exec linkstash whoami
# Output:
# app

# 10. Stop and remove the container so port 8000 is free for tomorrow's compose lab.
docker rm -f linkstash
# Output:
# linkstash

Read the last steps back: one Dockerfile, two stages, and the image you ship (linkstash:dev, ~214 MB) is less than half the size of the builder that produced it (~486 MB) — the ~270 MB gap is the compiler toolchain you deliberately left behind. It runs as app, not root, and answers /healthz before a database even exists.

Common Errors & Fixes

Section 3 of 5 · ~3 min

These three are what almost everyone hits the first time they write a multi-stage Python image. Read the error text slowly — parsing it is the actual skill.

Common error: The builder tries to compile psycopg2 without the Postgres headers — the pip install step fails:

Error: pg_config executable not found.

Please add the directory containing pg_config to the PATH
or specify the full path in the setup.cfg by setting pg_config.
error: subprocess-exited-with-error

Why: Building psycopg2 from source needs pg_config, which ships in libpq-dev. If the builder installs only build-essential (or nothing), pip has a compiler but no Postgres client headers and can’t build the extension.

Fix: Install libpq-dev alongside build-essential in the builder stage before pip install. Alternatively switch the requirement to psycopg2-binary, which ships precompiled wheels and needs no toolchain — but the -binary package is discouraged for production, which is exactly why the multi-stage source build is worth learning.

How you’d spot it in prod: A build that dies at the pip install step with pg_config executable not found is a missing build-time dependency, not an app bug — the fix is in the Dockerfile’s builder stage, never in the application code.

Common error: The final stage copies the venv but never installs libpq5, so the container crashes on boot. docker logs linkstash shows:

ImportError: libpq.so.5: cannot open shared object file: No such file or directory

Why: psycopg2 is a compiled extension that dynamically links libpq5 at import time. The builder had that library via libpq-dev, but the slim final stage copied only /opt/venv — the .so it depends on isn’t there. When uvicorn imports app.main → app.db → create_engine loads the driver, the missing library aborts the import and the container exits.

Fix: Add apt-get install -y --no-install-recommends libpq5 to the final stage (the runtime library, not the libpq-dev headers), then rebuild. The import resolves and uvicorn boots.

How you’d spot it in prod: An image that built green but crashloops immediately, with logs pointing at a .so and “cannot open shared object file”, is a missing runtime shared library the multi-stage copy left behind — check what the final stage installs, not the app.

Common error: The venv is copied but PATH is only set in the builder, so the final stage can’t find uvicorn. docker run fails:

docker: Error response from daemon: failed to create task for container:
exec: "uvicorn": executable file not found in $PATH: unknown.

Why: Each build stage has its own environment. An ENV PATH=... in the builder does not carry into the final stage. Without /opt/venv/bin on PATH there, the shell has no idea where the copied uvicorn shim lives, even though the file is present under /opt/venv/bin.

Fix: Repeat ENV PATH="/opt/venv/bin:$PATH" in the final stage (as the Dockerfile above does), or invoke the absolute path in CMD — CMD ["/opt/venv/bin/uvicorn", "app.main:app", ...].

How you’d spot it in prod: A container that exits instantly with “executable file not found in $PATH” for a binary you know you installed is almost always a PATH that didn’t survive a stage boundary — confirm the final stage sets it, not just the builder.

Docker Image Optimization Interview Questions

Section 4 of 5 · ~1 min

Multi-stage builds, non-root images and the build-versus-run split below are standard mid-round questions once “can you write a Dockerfile?” becomes “can you ship a lean, safe one?” — cover each answer, say your own version out loud first, then compare, because recalling before revealing is what makes it stick. The four questions and answers render right after this note.

Go Deeper

Section 5 of 5 · ~1 min

Optional extras if you have ~30 more minutes today:

• 5 min — Run docker history linkstash:dev and read each layer against your Dockerfile; note the single COPY --from=builder /opt/venv layer and that no compiler or -dev package appears anywhere in the shipped image.
• 10 min — Skim Docker’s multi-stage builds guide and note --target (you used it above to expose the builder) and how COPY --from references a named stage.
• 15 min — Read the multi-stage and image-slimming section of the Docker for DevOps guide to see how today’s builder/runtime split fits the build-and-ship workflow you carry into compose tomorrow and CI on Day 4.