Project 1, Day 4: build the CI pipeline

Continuous integration: test every push, publish only tags

Section 1 of 5 · ~2 min

Days 1–3 you built linkstash — a FastAPI URL shortener, its Dockerfile, and a compose stack that runs it beside PostgreSQL 16. It all works on your laptop. Today you make it work on a machine that is not your laptop: a fresh GitHub Actions runner that lints, tests, and — on a release — builds and publishes the image, on every push, with no human in the loop. That is continuous integration — the safety net that catches a broken commit before anyone pulls it.

The whole pipeline is one file, .github/workflows/ci.yml, and the key design decision is to split it into two jobs with a gate between them:

• test runs on every push and every pull request. It checks out the code, installs Python 3.12, runs ruff for lint and pytest for the tests. Cheap, fast, runs constantly.
• build-and-push runs only when you push a version tag (v*), and only if test already passed (needs: test). It logs in to GHCR and publishes the image.

Why the split? You want the feedback — lint and tests — on every single commit, but you do not want a new image in the registry for every work-in-progress push. Publishing is a release action, so you gate it behind a tag. The needs: test line makes that gate real: a red test can never produce a published image.

Testing against a real Postgres, not a mock

linkstash talks to Postgres, so a test that fakes the database proves little. GitHub Actions solves this with service containers: you declare postgres:16 under services: and the runner starts it in a sidecar container, waits for its healthcheck, then exposes it on localhost:5432. Your tests connect with the same DATABASE_URL shape as compose — only the host changes (db on the compose network becomes localhost on the runner). Same engine, same version, real SQL.

Real world: think of a car plant’s test track. Every car off the line does a lap before it ships — not a simulation, an actual drive on real tarmac. The test job is that lap; the service-container Postgres is real tarmac, not a driving-game screen. Only cars that finish the lap (needs: test) roll onto the transporter (build-and-push).

GitHub’s own Actions service containers feature is what makes this a few lines rather than a docker run script — the runner manages the container’s lifecycle and health for you, using the official postgres image from Docker Hub. Declare it, point DATABASE_URL at localhost, and your pytest run has a live database.

Below you write the tests, then the workflow, push it, and watch the test job go green against a live Postgres — then cut a tag and watch the second job publish the image.

Hands-On Lab

Section 2 of 5 · ~5 min

Budget about 30 minutes. You’re in the linkstash repo from Days 1–3 (FastAPI app in app/, Dockerfile, compose.yaml, requirements.txt, requirements-dev.txt with ruff/pytest/httpx), pushed to github.com/pushkar/linkstash, with gh authenticated and your .venv active. You’ll add the tests and the CI workflow, then watch both jobs run. Run IDs, SHAs, digests and timestamps are unique per run — yours will differ.

# 1. Confirm the Day-3 stack is here and the remote is set.
cd ~/linkstash && ls && git remote -v
# Output:
# app  compose.yaml  Dockerfile  requirements-dev.txt  requirements.txt
# origin  https://github.com/pushkar/linkstash.git (fetch)
# origin  https://github.com/pushkar/linkstash.git (push)

Save this as tests/test_app.py. It drives the app through FastAPI’s TestClient (backed by httpx) — health check, shorten, and the 307 redirect round-trip:

# tests/test_app.py
import pytest
from fastapi.testclient import TestClient

from app.main import app


@pytest.fixture(scope="session")
def client():
    # Enter TestClient as a context manager so FastAPI startup/lifespan runs —
    # that is what calls init_db() and creates the links table before the DB tests.
    with TestClient(app) as c:
        yield c


def test_healthz_ok(client):
    r = client.get("/healthz")
    assert r.status_code == 200


def test_shorten_returns_code(client):
    r = client.post("/shorten", json={"url": "https://example.com/a/very/long/path"})
    assert r.status_code == 200
    assert isinstance(r.json()["code"], str)


def test_redirect_roundtrip(client):
    code = client.post("/shorten", json={"url": "https://opscanopy.com"}).json()["code"]
    r = client.get(f"/{code}", follow_redirects=False)
    assert r.status_code == 307
    assert r.headers["location"] == "https://opscanopy.com"

# 2. Lint locally first — ruff needs no database, so it's your fastest gate.
mkdir -p tests && touch tests/__init__.py
ruff check .
# Output:
# All checks passed!
# (The DB-backed pytest run needs Postgres — that's exactly what the CI
#  service container gives it, so you'll watch pytest go green in CI next.)

Save this as .github/workflows/ci.yml. Note the services: block with its healthcheck, the DATABASE_URL pointing at localhost, and how build-and-push is gated by both needs: test and the tag if::

# .github/workflows/ci.yml
name: ci
on:
  push:
    branches: [main]
    tags: ["v*"]
  pull_request:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest
    services:
      db:
        image: postgres:16
        env:
          POSTGRES_PASSWORD: postgres
          POSTGRES_DB: linkstash
        ports:
          - 5432:5432
        options: >-
          --health-cmd "pg_isready -U postgres"
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5
    env:
      DATABASE_URL: postgresql://postgres:postgres@localhost:5432/linkstash
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - name: Install deps
        run: |
          python -m pip install --upgrade pip
          pip install -r requirements.txt -r requirements-dev.txt
      - name: Lint
        run: ruff check .
      - name: Test
        run: pytest -q

  build-and-push:
    needs: test
    if: startsWith(github.ref, 'refs/tags/')
    runs-on: ubuntu-latest
    permissions:
      contents: read
      packages: write            # only this job publishes to GHCR
    steps:
      - uses: actions/checkout@v4
      - name: Log in to GHCR
        uses: docker/login-action@v3
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}
      - name: Build and push
        uses: docker/build-push-action@v6
        with:
          context: .
          push: true
          tags: ghcr.io/${{ github.repository }}:${{ github.sha }}

# 3. Commit the tests and the workflow, then push to trigger the first run.
git add tests/ .github/workflows/ci.yml
git commit -m "ci: lint + test on Postgres service, tag-gated GHCR push"
git push
# Output (your object hashes and commit SHA will differ):
# [main 4c1f7ae] ci: lint + test on Postgres service, tag-gated GHCR push
#  3 files changed, 61 insertions(+)
# To https://github.com/pushkar/linkstash.git
#    b2d9e10..4c1f7ae  main -> main

# 4. Watch the run: the test job runs; build-and-push is skipped (this is not a tag).
gh run watch $(gh run list --limit 1 --json databaseId --jq '.[0].databaseId')
# Output (run/job IDs and timing are yours-will-differ):
# ✓ ci · 18402117733
# Triggered via push about 20 seconds ago
#
# JOBS
# ✓ test in 41s (ID 51230884417)
#   ✓ Set up job
#   ✓ Initialize containers          # GitHub starts postgres:16 and waits for healthy
#   ✓ Run actions/checkout@v4
#   ✓ Run actions/setup-python@v5
#   ✓ Install deps
#   ✓ Lint
#   ✓ Test
#   ✓ Complete job
#
# ✓ Run ci (18402117733) completed with 'success'

# 5. Confirm the second job was SKIPPED, not run — the tag gate held.
gh run view $(gh run list --limit 1 --json databaseId --jq '.[0].databaseId')
# Output (trimmed):
# ✓ test           in 41s
# - build-and-push  Skipped               # if: startsWith(github.ref,'refs/tags/') was false

# 6. Now exercise the release path — a throwaway pre-release tag (v1.0.0 is tomorrow's real cut).
git tag v0.1.0
git push origin v0.1.0
# Output:
# To https://github.com/pushkar/linkstash.git
#  * [new tag]         v0.1.0 -> v0.1.0

# 7. Watch the tag run: test passes, then build-and-push actually publishes.
gh run watch $(gh run list --limit 1 --json databaseId --jq '.[0].databaseId')
# Output (yours-will-differ):
# ✓ ci · 18402301994
# Triggered via push about 15 seconds ago
#
# JOBS
# ✓ test in 40s (ID 51231044820)
# ✓ build-and-push in 33s (ID 51231052611)
#   ✓ Log in to GHCR
#   ✓ Build and push
#   ✓ Complete job
#
# ✓ Run ci (18402301994) completed with 'success'

# 8. Confirm the image landed in GHCR, tagged with the exact commit SHA the tag pointed at.
gh api /user/packages/container/linkstash/versions --jq '.[0].metadata.container.tags'
# Output (the SHA matches the commit v0.1.0 points to — yours will differ):
# ["4c1f7ae9b3d2c1e0f8a7b6c5d4e3f2a1b0c9d8e7"]

# 9. Clean up the throwaway tag locally and on the remote — you cut the real v1.0.0 tomorrow.
git tag -d v0.1.0
git push origin :refs/tags/v0.1.0
# Output:
# Deleted tag 'v0.1.0' (was 4c1f7ae)
# To https://github.com/pushkar/linkstash.git
#  - [deleted]         v0.1.0

Read the flow back: a push ran ruff and pytest against a real Postgres the runner started for you, and the publish job stayed skipped because it wasn’t a tag. A version tag ran the same tests and, once they were green, built the image and pushed it to GHCR under the commit SHA. That is the exact shape of a real service-to-registry pipeline — you finish and version it tomorrow.

Common Errors & Fixes

Section 3 of 5 · ~3 min

These three are what almost everyone hits the first time a service-backed CI job runs. Read the text slowly — parsing it is the actual skill.

Common error: Leaving DATABASE_URL pointing at the compose host db inside CI — the Test step fails while collecting or running tests:

could not translate host name "db" to address: Name or service not known

Why: On the compose network the database is reachable as db (Day 3), but a GitHub Actions service container is published to the runner’s localhost, not under the service key as a hostname your app can resolve. If your test config inherits the compose DATABASE_URL, it tries to resolve db, which doesn’t exist on the runner, and DNS resolution fails before any SQL runs.

Fix: Set the job’s env: DATABASE_URL to postgresql://postgres:postgres@localhost:5432/linkstash — same credentials and database, host localhost. Because you mapped ports: - 5432:5432, the service is right there. Keep db only in compose.yaml.

How you’d spot it in prod: A test suite that passes under docker compose but fails only in CI with a host-resolution error is almost always a hard-coded service hostname leaking into the CI environment. Compare the DATABASE_URL the job sets against the one compose sets.

Common error: Declaring the Postgres service without a healthcheck, so steps start before it accepts connections — an early, flaky failure:

connection to server at "localhost" (127.0.0.1), port 5432 failed: Connection refused
  Is the server accepting connections on that host and port?

Why: A service container takes a second or two to initialise. Without an options: --health-cmd, GitHub can’t tell when Postgres is ready — it only knows the container started — so your pytest step can race ahead and connect before the database is listening. It often passes on a slow runner and fails on a fast one, which is what makes it look flaky.

Fix: Add the healthcheck options (--health-cmd "pg_isready -U postgres" plus interval/timeout/retries). GitHub then waits for the service to report healthy in the “Initialize containers” step before running any of your steps, so the database is guaranteed up.

How you’d spot it in prod: Intermittent “Connection refused” on job startup — green on retry, red at random — is the classic signature of a dependency that isn’t gated on a readiness check. Look for a missing healthcheck before you blame the network.

Common error: Expecting an image after a normal git push and finding none, because build-and-push never ran:

gh api /user/packages/container/linkstash/versions
#   HTTP 404: Package not found
# In the run, the job shows:  build-and-push  Skipped

Why: The job carries if: startsWith(github.ref, 'refs/tags/'), so it only runs for a tag ref. A push to main is refs/heads/main, which fails that condition, and the job is skipped — correctly. Nothing is broken; the gate is doing its job. People miss the greyed-out “Skipped” job and assume the build failed silently.

Fix: To publish, push a tag: git tag v1.0.0 && git push origin v1.0.0. That gives a refs/tags/ ref, the condition passes, and — once needs: test is green — the image builds and pushes.

How you’d spot it in prod: A “release” that never produces a new image, with a skipped publish job, means the trigger condition wasn’t met — check whether a tag was actually pushed (and matches the v* pattern) before suspecting the registry or credentials.

CI Pipeline Interview Questions

Section 4 of 5 · ~1 min

These four are what a screening round asks once “can you write a workflow?” becomes “can you run a real test-and-ship pipeline?” — cover each answer, say your own version out loud first, then compare, because recalling before revealing is what makes it stick. The four questions and answers render right after this note.

Go Deeper

Section 5 of 5 · ~1 min

Optional extras if you have ~30 more minutes today:

• 5 min — In the Actions tab, open the tag run and expand the test job’s “Initialize containers” step — watch GitHub pull postgres:16 and wait for the healthcheck to pass before any of your steps start.
• 10 min — Skim GitHub’s service containers docs and read the options: --health-cmd section — the healthcheck is the one line that makes the runner wait for Postgres.
• 15 min — Read the CI/CD section of the Docker for DevOps guide to see how today’s test-then-tag-then-push pipeline fits the wider build-and-ship workflow you carry into deployment.