CI/CD concepts — pipelines, environments, artifacts (+ Jenkins legacy aside)

Continuous integration, continuous delivery, and the path to prod

Section 1 of 5 · ~2 min

Yesterday you debugged a single container by hand. Today you zoom out to the question every team eventually faces: how does code get from a laptop to production reliably, many times a day, without someone SSHing in at midnight? The answer is CI/CD — a pipeline that automates the path to prod.

Continuous integration (CI) means every push is automatically built and tested against the shared main branch. Instead of merging a month of work in one terrifying integration week, everyone integrates small changes constantly and a machine catches breakage within minutes.

Continuous delivery (CD) takes the tested build and keeps it always ready to release — packaged, promoted through environments, one click or merge away from shipping. Continuous deployment goes one step further: no human click, every green build lands in prod on its own.

A pipeline is the ordered sequence a change runs through, and its stages are almost always the same three: build → test → deploy. Build turns source into a runnable artifact. Test runs checks against that artifact. Deploy ships it. Each stage only runs if the previous one passed, so the first red stage stops the line and broken code never reaches the next step.

Real world: A pipeline is an airport security line, not a free-for-all. Each passenger clears document check, then the scanner, then the gate, in order — and if the scanner flags a bag, that passenger stops right there and never reaches the plane. Nobody re-checks documents at the gate because that stage already passed. The same person moves forward through each checkpoint; you don’t clone a fresh passenger at every desk.

A named example makes the payoff concrete. Amazon reported back in 2011 that it was deploying to production every 11.6 seconds on average across its fleet — thousands of deploys a day. That pace is impossible by hand; it only exists because a pipeline builds, tests and ships each tiny change automatically, so releasing is routine and reversible rather than a scary all-hands event.

Environments and artifacts

An artifact is the single built thing the pipeline produces once — the image, jar or bundle — and then carries forward. The discipline is build once, promote the same bytes: you test that exact artifact, then move that same artifact to staging, then to production. Rebuilding per environment risks shipping something you never tested.

Environments are the staged copies of your app the artifact passes through. The classic ladder is staging → production: staging mirrors prod so you can exercise the real artifact safely, and only after it passes there is it promoted to prod, where real users hit it. Promoting the identical artifact is what makes “it worked in staging” actually mean something.

You’ll still meet Jenkins running these pipelines at older shops — it pioneered CI/CD — but GitHub Actions is the modern default, and it’s where the next few days live.

Hands-On Lab

Section 2 of 5 · ~4 min

Budget about 25 minutes. Open your WSL2 Ubuntu 24.04 terminal (Docker 27+ from earlier this week, plus the gh CLI logged in with gh auth login). You’ll first run a three-stage pipeline by hand so you feel what CI/CD automates, then encode the exact same stages as a GitHub Actions workflow and watch GitHub run them. Run IDs, SHAs, URLs and timings are unique to each run — yours will differ from the samples.

# 1. New project: a one-line app and a test that checks it.
mkdir -p ~/cicd-lab && cd ~/cicd-lab
echo 'def add(a, b): return a + b' > app.py
echo 'from app import add; assert add(2, 3) == 5; print("test ok")' > test_app.py
ls
# Output:
# app.py  test_app.py

# 2. Stage 1 — BUILD: package the source into a versioned artifact.
mkdir -p dist && tar -czf dist/app-1.0.tgz app.py
ls dist
# Output:
# app-1.0.tgz

# 3. Stage 2 — TEST: run the checks against what you just built.
python3 test_app.py
# Output:
# test ok

# 4. Stage 3 — DEPLOY (to staging): the SAME artifact moves forward, never rebuilt.
mkdir -p staging && cp dist/app-1.0.tgz staging/ && ls staging
# Output:
# app-1.0.tgz

# 5. Chain the three stages into one script. `set -e` stops at the first red stage.
cat > pipeline.sh <<'EOF'
#!/usr/bin/env bash
set -e
echo "== build ==" && tar -czf dist/app-1.0.tgz app.py
echo "== test =="  && python3 test_app.py
echo "== deploy ==" && cp dist/app-1.0.tgz staging/
echo "pipeline green"
EOF
bash pipeline.sh
# Output:
# == build ==
# == test ==
# test ok
# == deploy ==
# pipeline green

# 6. Prove a red stage STOPS the line: break the test, re-run, watch deploy never happen.
echo 'from app import add; assert add(2, 3) == 6, "math is wrong"' > test_app.py
bash pipeline.sh; echo "exit: $?"
# Output (note "== deploy ==" is never printed — the pipeline halted at test):
# == build ==
# == test ==
# Traceback (most recent call last):
#   File "test_app.py", line 1, in <module>
#     from app import add; assert add(2, 3) == 6, "math is wrong"
#                                 ^^^^^^^^^^^^^^
# AssertionError: math is wrong
# exit: 1

# 7. Fix the test, then make room for the automated version of these exact stages.
echo 'from app import add; assert add(2, 3) == 5; print("test ok")' > test_app.py
mkdir -p .github/workflows

Here is that same build → test → (upload) pipeline as a GitHub Actions workflow. Save it as .github/workflows/ci.yml (current syntax — pinned action majors, runs-on: ubuntu-latest):

# .github/workflows/ci.yml — the same stages, run on GitHub on every push.
name: CI
on: [push]
jobs:
  build-test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: '3.12'
      - name: Build artifact
        run: mkdir -p dist && tar -czf dist/app-1.0.tgz app.py
      - name: Test
        run: python3 test_app.py
      - name: Upload artifact
        uses: actions/upload-artifact@v4
        with:
          name: app-build
          path: dist/app-1.0.tgz

# 8. Commit everything, including the workflow file.
git init -q -b main
git add .
git -c user.name=pushkar -c user.email=pushkar@example.com commit -qm "Add CI pipeline"
git log --oneline
# Output (the short SHA is yours-will-differ):
# a1b2c3d (HEAD -> main) Add CI pipeline

# 9. Create the repo on GitHub and push — the push triggers the workflow automatically.
gh repo create cicd-lab --private --source=. --remote=origin --push
# Output (the repo URL is yours-will-differ):
# ✓ Created repository pushkar/cicd-lab on GitHub
# ✓ Added remote https://github.com/pushkar/cicd-lab.git
# ✓ Pushed commits to https://github.com/pushkar/cicd-lab.git

# 10. Watch the run GitHub just started — the same build → test, now on their runners.
gh run watch
# Output (run ID and timings are yours-will-differ):
# ✓ CI · a1b2c3d
# Triggered via push about 8 seconds ago
#
# JOBS
# ✓ build-test in 19s
#   ✓ Set up job
#   ✓ actions/checkout@v4
#   ✓ actions/setup-python@v5
#   ✓ Build artifact
#   ✓ Test
#   ✓ Upload artifact
#
# ✓ Run CI (a1b2c3d) completed with 'success'

# 11. Download the artifact the pipeline produced — the exact bytes a deploy would promote.
gh run download --name app-build
ls
# Output (the artifact is now local, alongside your source):
# app-1.0.tgz  app.py  dist  pipeline.sh  staging  test_app.py

Read the last steps back to yourself: you ran build → test → deploy by hand, broke the test and watched the line halt before deploy, then pushed the identical three stages as a workflow that GitHub ran on its own and handed you the artifact. That is CI/CD — the path to prod, automated, fail-fast, and promoting one artifact forward.

Common Errors & Fixes

Section 3 of 5 · ~3 min

These three hit almost everyone shipping their first workflow. Read the error text slowly — parsing it is the actual skill.

Common error: Pushing a workflow whose YAML is misindented — GitHub refuses to run it and surfaces this on the Actions tab:

Invalid workflow file: .github/workflows/ci.yml#L8
The workflow is not valid. .github/workflows/ci.yml (Line: 8, Col: 9): Unexpected value 'run'

Why: YAML is whitespace-significant, so a steps: entry indented one space off, or a run: that isn’t nested under its step, changes the structure GitHub parses. It isn’t a typo in the command — the file’s shape is wrong, so the parser can’t tell where a step begins and ends.

Fix: Line the keys up consistently (two spaces per level, spaces not tabs), and keep each step’s keys (name, uses, run) at the same indent under the - bullet. Paste the file into a YAML linter or run gh workflow view — the reported line and column point straight at the break.

How you’d spot it in prod: A workflow that vanishes from the Actions tab or shows a red “Invalid workflow file” annotation after an edit is almost always indentation, not logic. Check the diff’s whitespace before you debug the commands themselves.

Common error: A test step failing turns the whole job red with a message that says nothing about the test:

Run python3 test_app.py
Traceback (most recent call last):
AssertionError: math is wrong
Error: Process completed with exit code 1.

Why: GitHub Actions judges each run step by its exit code. Anything non-zero — a failed assertion, a compiler error, a curl that 404s — makes the step fail, and a failed step fails the job and stops the ones after it. “Process completed with exit code 1” is the generic wrapper; the real reason is the lines just above it.

Fix: Read upward from the exit-code line to the actual error (here the assertion), reproduce it locally with the same command, fix it, and push again. Don’t chase the exit code — it’s only telling you the step returned non-zero.

How you’d spot it in prod: A pipeline that was green yesterday and is red today on “exit code 1” is doing its job — it caught a regression. Open the failing step, read the lines above the wrapper, and treat the stopped pipeline as the safety net that kept the bug out of prod.

Common error: Pinning an old major of an artifact action — the job fails before it even uploads:

Error: This request has been automatically failed because it uses a deprecated version of `actions/upload-artifact: v3`.
Learn more: https://github.blog/changelog/2024-04-16-deprecation-notice-v3-of-the-artifact-actions/

Why: GitHub retired v3 of the upload/download artifact actions, and deprecated majors are hard-failed rather than silently run, to force the migration. The workflow logic is fine — the action version is simply no longer accepted.

Fix: Bump to the current major: uses: actions/upload-artifact@v4 (and actions/download-artifact@v4). Re-pin any other actions to a supported major while you’re in the file — actions/checkout@v4, actions/setup-python@v5.

How you’d spot it in prod: A pipeline that ran for months and suddenly fails every run on a “deprecated version” line means an action major reached end of life, not that your code broke. Grep your workflows for the named action and bump the tag.

CI/CD Interview Questions

Section 4 of 5 · ~1 min

These four are the highest-yield CI/CD screening questions — the CI-versus-CD distinction, what a pipeline is, why you promote one artifact, and what automating the path to prod actually buys you. Cover each answer, say your own version out loud first, then compare — recalling before revealing is what makes it stick for interview day. The four questions and answers render right after this note.

Go Deeper

Section 5 of 5 · ~1 min

Optional extras if you have ~30 more minutes today:

• 5 min — Re-run bash pipeline.sh, then break the build stage instead of the test (rename app.py), and confirm the pipeline halts even earlier — proof that fail-fast stops at the first red stage, wherever it is.
• 10 min — On your cicd-lab repo, open the Actions tab in the browser and click into the run you watched: read each step’s expanded log and find where the artifact was uploaded, so the CLI output and the web UI map onto the same pipeline.
• 15 min — Skim the build-and-ship sections of the Docker for DevOps guide and the DevOps projects guide to see how a real image becomes the artifact a pipeline promotes — the workflow you’ll flesh out over the next three days.