Harden a Linux Server

Steps

1. Provision a disposable Linux host

   You need a real, disposable Linux box with SSH access. The fastest path is a
   Docker container running sshd; the more realistic path is a full VM via
   Vagrant. Pick one.

   Option A — Docker (fastest)

   # Build a minimal Ubuntu image with SSH server and a starting root password
   cat > Dockerfile <<'EOF'
   FROM ubuntu:22.04
   RUN apt-get update && apt-get install -y openssh-server sudo && \
       mkdir /var/run/sshd && \
       echo 'root:changeme123' | chpasswd && \
       sed -i 's/#PermitRootLogin prohibit-password/PermitRootLogin yes/' /etc/ssh/sshd_config && \
       sed -i 's/#PasswordAuthentication yes/PasswordAuthentication yes/' /etc/ssh/sshd_config
   EXPOSE 22
   CMD ["/usr/sbin/sshd", "-D"]
   EOF
   
   docker build -t hardening-lab .
   docker run -d --name hardening-lab -p 2222:22 hardening-lab
   
   # Confirm you can reach it (password: changeme123)
   ssh -p 2222 root@localhost
   

   Option B — Vagrant (closer to a real VM)

   vagrant init ubuntu/jammy64
   vagrant up
   vagrant ssh   # confirm you land inside the box
   

   Whichever you pick, you should end this step logged in as root (or via
   vagrant ssh with passwordless sudo), with a known way back in if
   something goes wrong: for Docker, docker exec -it hardening-lab bash
   always gets you a root shell even if SSH breaks; for Vagrant,
   vagrant ssh similarly bypasses your hardening changes.

   Checkpoint

   Run whoami and cat /etc/os-release inside the box and confirm you're
   root on a fresh Ubuntu/Debian host. Keep the "escape hatch" command
   (docker exec or vagrant ssh) written down somewhere — you'll want it if
   a later step goes wrong.

   Deliverable for this step: a running, disposable Linux host you can SSH
   into, plus a documented escape hatch that does not depend on SSH.

2. Create a non-root sudo user

   Running everything as root means any compromised process — a bug in your
   app, a malicious dependency, a hijacked script — instantly has full control
   of the machine. A non-root user with sudo gives you the same administrative
   power on demand, with an audit trail and a smaller blast radius for
   anything that runs without sudo.

   # As root:
   adduser --disabled-password --gecos "" opslab
   usermod -aG sudo opslab      # Debian/Ubuntu; use 'wheel' group on RHEL/Fedora
   
   # Give the new user a password for now (you'll disable password SSH later)
   passwd opslab
   
   # Confirm the sudo group grants what you expect
   cat /etc/sudoers.d/* /etc/sudoers 2>/dev/null | grep -i sudo
   

   Switch to it and confirm sudo works before you touch SSH config:

   su - opslab
   sudo whoami        # should print: root
   

   Why this order matters

   You create and verify the non-root user before locking down root SSH
   access in Step 3. If you disable root login first and your new user isn't
   working yet, you can lock yourself out entirely (on a real server — on this
   disposable box you'd just use your escape hatch and start over, but build
   the habit now).

   Checkpoint

   From a fresh terminal (not the root session), run:

   ssh -p 2222 opslab@localhost   # or `vagrant ssh` equivalent
   sudo whoami                    # must print "root" without error
   

   Deliverable for this step: a non-root user that can authenticate and
   successfully run sudo whoami, confirmed from a separate login, not just
   the session where you created it.

3. Key-only SSH, no root login

   Passwords are guessable and phishable; SSH key pairs are not. And even a
   strong password on root is one credential away from full compromise —
   removing root's ability to log in over SSH entirely means an attacker must
   first compromise a named, auditable user and escalate via sudo.

   Generate and install a key

   On your host machine (not inside the box):

   ssh-keygen -t ed25519 -f ~/.ssh/hardening_lab -C "hardening-lab"
   ssh-copy-id -p 2222 -i ~/.ssh/hardening_lab.pub opslab@localhost
   # or manually: append the .pub contents to
   # /home/opslab/.ssh/authorized_keys inside the box (mode 600, owned by opslab)
   

   Confirm key-based login works before disabling passwords:

   ssh -p 2222 -i ~/.ssh/hardening_lab opslab@localhost 'whoami'
   

   Lock down sshd_config

   Inside the box, edit /etc/ssh/sshd_config:

   PermitRootLogin no
   PasswordAuthentication no
   ChallengeResponseAuthentication no
   PubkeyAuthentication yes
   AllowUsers opslab
   

   Apply and restart:

   sudo sshd -t                       # validate syntax before restarting
   sudo systemctl restart sshd
   

   Checkpoint — prove both halves

   # 1) Key login for opslab still works
   ssh -p 2222 -i ~/.ssh/hardening_lab opslab@localhost 'whoami'
   
   # 2) Password login is refused (should fail / be rejected, not prompt success)
   ssh -p 2222 -o PubkeyAuthentication=no opslab@localhost
   
   # 3) Root login is refused entirely, even with the right password
   ssh -p 2222 root@localhost
   

   If step 2 or 3 above still lets you in, sshd_config didn't reload —
   re-check sudo sshd -t output and systemctl status sshd.

   Deliverable for this step: opslab logs in only with the key,
   PasswordAuthentication and PermitRootLogin are both confirmed off by
   an actual failed login attempt, not just by reading the config file.

4. Lock down the firewall with ufw

   Even with SSH hardened, every other service on the box is still reachable
   from anywhere unless a firewall says otherwise. ufw (Uncomplicated
   Firewall) gives you a default-deny policy in a few commands.

   sudo apt-get update && sudo apt-get install -y ufw
   
   # Default-deny everything inbound, allow all outbound
   sudo ufw default deny incoming
   sudo ufw default allow outgoing
   
   # Allow only what you actually need. Rate-limit SSH against brute force.
   sudo ufw limit 22/tcp comment 'SSH, rate-limited'
   # Add more only as needed, e.g.:
   # sudo ufw allow 80/tcp comment 'HTTP'
   # sudo ufw allow 443/tcp comment 'HTTPS'
   
   sudo ufw enable
   

   Order matters inside a container. sudo ufw enable before you've
   confirmed your SSH rule is exactly right can drop your own connection. In
   Docker specifically, ufw interacts awkwardly with iptables-based port
   mapping — if you're on Docker, treat this step as learning the ufw
   workflow and verify with ufw status; for a fully enforced firewall in
   front of Docker port mappings, a Vagrant/VM target is more realistic.

   Why limit instead of allow for SSH

   ufw limit 22/tcp denies an IP that makes more than 6 connection attempts
   within 30 seconds — a cheap first line of defense against brute force,
   complementary to (not a replacement for) fail2ban in Step 6.

   Checkpoint

   sudo ufw status verbose
   

   Confirm the output shows:

   • Default: deny (incoming), allow (outgoing)
   • Only the ports you explicitly allowed are listed as ALLOW/LIMIT
   • Nothing else is open

   Then, from your host machine, try to reach a port you did not open
   (e.g. nc -zv localhost 2222 should work, but a port you never allowed,
   like 9999, should time out or be refused).

   Deliverable for this step: ufw status verbose output showing a
   default-deny inbound policy with an explicit, minimal allowlist — and a
   failed connection attempt to a port you did not open, proving the deny
   actually holds.

5. Audit and disable unnecessary services

   Every running service is a potential vulnerability, even behind a firewall —
   a local privilege-escalation bug in a daemon you never use is still a risk,
   and a misconfigured firewall rule later can accidentally expose something
   you forgot was running. The rule: if you don't need it, don't run it.

   Inventory what's listening and what's enabled

   # What's actually listening on a network socket right now
   sudo ss -tulpn
   
   # What services are enabled to start at boot
   systemctl list-unit-files --type=service --state=enabled
   

   On a minimal Ubuntu server image, you'll typically see things like
   cron, rsyslog, and sshd — keep those. Watch for anything you didn't
   knowingly install: an unused mail transfer agent (exim4, postfix), a
   print service (cups), or a database/service bound to 0.0.0.0 that you
   never intended to expose beyond localhost.

   Disable what you don't need

   # Example: disable and stop a service you identified as unnecessary
   sudo systemctl disable --now <service-name>
   
   # Confirm it's gone from both "enabled" and "listening"
   systemctl is-enabled <service-name>     # should print "disabled" or error
   sudo ss -tulpn | grep <service-name>    # should print nothing
   

   For anything bound to 0.0.0.0 that you do need but only for local use
   (e.g. a database you only query from the same host), rebind it to
   127.0.0.1 in its config instead of relying solely on the firewall — this
   is defense in depth: even if a firewall rule is later loosened by mistake,
   the service still won't accept remote connections.

   Checkpoint

   sudo ss -tulpn
   

   Every line in the output should be something you can name and justify. If
   you can't say why a listening socket exists, that's your next thing to
   investigate and likely disable.

   Deliverable for this step: a ss -tulpn output where every listening
   socket is accounted for, plus the list of services you disabled and why.

6. Install fail2ban and verify a ban (submission)

   ufw limit slows brute force down; fail2ban actively bans an offending IP
   after repeated failures, watching your logs and updating the firewall for you.
   This is your last layer for this lab, and the one you'll prove end-to-end.

   Install and enable the SSH jail

   sudo apt-get install -y fail2ban
   
   sudo tee /etc/fail2ban/jail.local > /dev/null <<'EOF'
   [sshd]
   enabled  = true
   port     = 22
   filter   = sshd
   logpath  = /var/log/auth.log
   maxretry = 3
   findtime = 5m
   bantime  = 15m
   EOF
   
   sudo systemctl enable --now fail2ban
   sudo systemctl status fail2ban --no-pager
   

   Trigger a real ban

   From your host machine, deliberately fail SSH login more than
   maxretry times against your own disposable box:

   for i in 1 2 3 4; do
     ssh -p 2222 -o PubkeyAuthentication=no -o PreferredAuthentications=password \
         -o ConnectTimeout=3 opslab@localhost 'echo should-not-reach' || true
   done
   

   Verify the ban actually took effect

   sudo fail2ban-client status sshd
   

   You should see your host's IP (likely 127.0.0.1 or the Docker gateway
   address) listed under "Banned IP list". Confirm a subsequent connection
   attempt is refused outright (connection refused/timeout, not even a
   password prompt) until bantime elapses, or unban it manually to keep
   working:

   sudo fail2ban-client set sshd unbanip <the-ip>
   

   ---

   Submission criteria

   Submit when all of the following hold, each backed by the command output
   that proves it:

   1. Non-root sudo user — sudo whoami succeeds as opslab from a fresh
      login (Step 2).
   2. Key-only SSH, no root — password login and root login both fail;
      key login for opslab succeeds (Step 3).
   3. Firewall default-deny — sudo ufw status verbose shows deny (incoming) with an explicit, minimal allowlist (Step 4).
   4. Minimal service surface — sudo ss -tulpn output with every listening
      socket accounted for, and the list of services you disabled (Step 5).
   5. fail2ban working — sudo fail2ban-client status sshd showing at
      least one IP that got banned by your own deliberate failed-login test
      (this step).

   Write a short note on what you'd add next for a production box (e.g.
   automatic security updates via unattended-upgrades, centralized log
   shipping, an intrusion detection agent) — you don't need to implement it,
   just name the next layer.