The first VPS I ever rented, I had the app running within the hour. Root login over SSH, password auth still on, no firewall, straight from apt install to a live process listening on the internet. I was proud of that hour for about four days, which is how long it took me to actually look at /var/log/auth.log.
It was full. Thousands of failed login attempts, all against root, all from IPs I’d never heard of, arriving every few seconds around the clock. Nobody had gotten in. But nobody had to try very hard either, and I had done nothing to make it harder for them. That log file is the reason I now spend the first twenty minutes on any new box doing the same handful of things before a single line of application code goes anywhere near it.
Nobody is targeting you specifically, and that’s the problem
The failed logins weren’t a person. They were a bot sweeping public IPv4 ranges, hitting port 22 on every machine that answers, trying root with a short list of common passwords. It’s not personal, it’s just cheap to run at that scale, and a brand new VPS with default settings is exactly what it’s looking for.
That’s the part that surprised me. I assumed you needed to be interesting to get attacked. You don’t. You just need a public IP and an open port.
Locking down SSH first
The single biggest change is disabling password authentication entirely and requiring a key. A key can’t be brute-forced in any practical sense; a password, given enough attempts, can.
Generate a key pair if you don’t already have one for this box, and copy the public key over while password login still works:
ssh-keygen -t ed25519 -C "deploy@myapp"
ssh-copy-id -i ~/.ssh/myapp_ed25519.pub deploy@your-server-iped25519 over rsa mostly because it’s faster and the keys are shorter, not because RSA is broken. Either is fine if you’re stuck with older tooling that doesn’t support ed25519.
Once you’ve confirmed the key logs you in, drop a hardening file instead of editing the shipped sshd_config directly:
# /etc/ssh/sshd_config.d/99-hardening.conf
PermitRootLogin no
PasswordAuthentication no
KbdInteractiveAuthentication no
AllowUsers deploy
MaxAuthTries 3
LoginGraceTime 20
X11Forwarding no
AllowTcpForwarding no
ClientAliveInterval 300Every line in that file is worth knowing, not just copying. PermitRootLogin no is the one that would have stopped my own story cold, root simply can’t authenticate over SSH anymore, key or password, so a bot hammering root all day is wasting its time before it even gets to guess anything. PasswordAuthentication no is the other half of that, it turns off password login entirely, for every user, so a leaked or guessed password is useless without also having the private key.
AllowUsers deploy is an explicit allowlist, if a username isn’t on it, SSH refuses the connection before it even gets to asking for a key or a password. MaxAuthTries 3 cuts a session off after three failed auth attempts instead of the default six, so a script can’t sit there guessing forever on one connection. LoginGraceTime 20 gives you twenty seconds to actually authenticate once connected, down from the two-minute default, closing off the trick of opening a connection and just leaving it idle. X11Forwarding no and AllowTcpForwarding no both close off things this box almost certainly doesn’t need, forwarding a graphical display or tunneling other traffic through the SSH connection, either of which is one more thing a stolen key could be used for if you leave it on. ClientAliveInterval 300 disconnects a session that’s gone quiet for five minutes, so a forgotten terminal on your laptop isn’t a live authenticated session sitting open indefinitely.
I also had ChallengeResponseAuthentication no in there for years because every tutorial I’d ever read used that name. It’s a deprecated alias, KbdInteractiveAuthentication is the current directive and does the same thing. Worth fixing if you’re copying an old config.
Restart the service and test in a second terminal before closing your first session:
sudo systemctl restart ssh
# in a NEW terminal, don't close the old one yet
ssh deploy@your-server-ipThat last line matters more than it looks like it should.
The cloud-init gotcha that undid mine the first time
I set all of the above once, restarted sshd, and password auth was still working. Spent a confused twenty minutes convinced I’d typo’d something before I found the actual cause.
Ubuntu’s cloud images ship cloud-init, and cloud-init writes its own SSH config to /etc/ssh/sshd_config.d/50-cloud-init.conf, usually with PasswordAuthentication yes baked in. sshd_config.d/*.conf files are read in filename order, and the first matching directive for a given setting wins. My hardening file was sitting in 10-hardening.conf, which loses to 50-cloud-init.conf alphabetically.
The fix is just naming it something that sorts after 50-, which is why the file above is 99-hardening.conf. If you’re hardening a fresh cloud image and your changes don’t seem to take, check for a 50-cloud-init.conf in that same directory before you assume you made a mistake.
One account, not root, for everything
Using the root account for everyday work, not just emergencies, means every command you run, and every mistake or leaked credential, has full, unrestricted power over the box instead of access that has to go through sudo and gets logged. The account you do log in as shouldn’t be root either, day to day.
adduser deploy
usermod -aG sudo deployNow deploy can sudo for the specific things that need root, restarting a service, editing a config file owned by root, without every single command on the box running as root by default. If that account’s key ever leaks, whoever has it still has to go through sudo for anything destructive, and that’s logged.
You can go further and scope sudo itself down to just the commands an account actually needs, instead of full sudo access. A line in /etc/sudoers.d/ like this restricts an operator to a fixed set of commands rather than everything:
# /etc/sudoers.d/deploy-restricted
deploy ALL=(root) NOPASSWD: /usr/bin/systemctl restart myapp, /usr/bin/systemctl status myappI don’t do this for every box. For a single-app VPS where I’m the only operator, full sudo on one account is a reasonable tradeoff between security and not fighting my own permissions at 2am. Where it’s actually paid off is on shared boxes with more than one person touching them, where I want a monitoring account that can restart a service and read logs and genuinely cannot do anything else.
Firewall: default deny, then open only what you use
UFW is a simpler front end for Linux’s iptables that blocks every incoming connection by default and only opens the ports you explicitly allow, which on this box means just SSH, HTTP, and HTTPS, set up like this:
sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw allow 22/tcp
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
sudo ufw enableEverything inbound is refused unless there’s an explicit allow for it. That’s the part that actually matters, not the specific ports. If your app talks to a managed Postgres instance somewhere else, port 5432 has no business being open on this box at all, and with default deny incoming it just isn’t, without you having to remember to block it.
ufw status verbose shows you what’s actually active, which is worth checking after any change instead of trusting that the command you ran did what you think:
sudo ufw status verboseOne more UFW feature that’s easy to miss: it can rate-limit a port directly, denying an IP that makes six or more connection attempts to that port within thirty seconds.
sudo ufw limit 22/tcpThat’s a coarser tool than fail2ban, it doesn’t look at whether the attempts actually failed, just how many connections came in. But it’s one command and it’s already there, so there’s no real reason to skip it even if you’re also running fail2ban.
fail2ban as the net under the net
fail2ban matters even with key-only SSH because a locked-down login doesn’t stop the attempts themselves, attackers still hammer the port nonstop and those doomed logins pile up in your logs and eat the server’s attention regardless of whether they can ever succeed. fail2ban’s job is to stop it from getting that far, banning an IP after it accumulates too many failures in a window, so the noise stops and the box spends less time answering junk.
The defaults in fail2ban’s shipped jail.conf are bantime 10m, findtime 10m, maxretry 5. Don’t edit jail.conf directly, it gets overwritten on package updates. Put overrides in jail.local instead:
# /etc/fail2ban/jail.local
[DEFAULT]
bantime = 24h
findtime = 10m
maxretry = 3
[sshd]
enabled = trueThree strikes in ten minutes gets you a day-long ban, which is a lot more aggressive than the default and has been fine for me on a single-app box where I know my own login pattern. Enable and check it actually started, fail2ban has occasionally needed a manual systemctl enable --now for me on a fresh install rather than starting cleanly the first time:
sudo systemctl enable --now fail2ban
sudo fail2ban-client ping
sudo fail2ban-client status sshdfail2ban-client ping should answer “Server replied: pong”. If it doesn’t, the service isn’t actually running yet, worth checking before you assume it’s protecting anything.
flowchart LR
Internet([Internet])
UFW[UFW firewall]
F2B[fail2ban]
SSHD[sshd, key-only]
Internet --> UFW
UFW -->|port 22 allowed| F2B
F2B -->|IP not banned| SSHD
UFW -->|everything else| Deny[Dropped]
F2B -->|too many failures| Ban[IP banned]
None of these layers replace each other. UFW keeps closed ports closed regardless of what’s listening behind them. fail2ban reacts to behavior on the ports that are open. sshd itself refuses passwords no matter what gets past the first two. If one layer has a gap, the other two are still there.
The stuff people skip
I used to stop at the three things above and call it done, and for a personal project that’s genuinely a defensible place to stop. Two more things I’ve added since that are cheap enough not to skip:
Unattended security upgrades, so patches for known vulnerabilities land without you having to remember to apt upgrade:
sudo apt install unattended-upgrades
sudo dpkg-reconfigure unattended-upgradesBy default this only applies security updates automatically, not feature upgrades, and it downloads kernel updates without auto-applying them so you don’t get an unplanned reboot. You still want to reboot into new kernels yourself on some cadence, it just won’t do it for you at 3am.
And a minimal audit trail, so if something does look wrong later you have more than “the box seemed fine yesterday” to go on. auditd watching /etc/passwd, /etc/shadow, and sudoers for changes covers the events I’d actually want to know about:
sudo apt install auditd
sudo auditctl -w /etc/passwd -p wa -k identity
sudo auditctl -w /etc/sudoers -p wa -k privilege_escalationWhere this stops helping
Everything above deals with opportunistic scanning, the automated sweeps that hit every IP on the internet whether or not there’s anything interesting behind it. That’s most of what a new VPS actually faces, and this handles it well.
It does not make you resistant to someone specifically targeting your application, a vulnerability in the app itself, or a leaked credential from somewhere other than SSH. That’s a different, bigger problem than locking down a box, and I’d rather say that plainly than let the length of this post imply otherwise.
The auth.log on my current boxes still fills up with rejected attempts. It’s just that “rejected” is now the whole story, instead of a countdown.
References
- sshd_config(5) — the canonical list of every SSH server directive, including current names for deprecated options
- UFW community documentation — default policy, port rules, and status commands
- fail2ban jail.conf — the shipped default jail definitions and their default
bantime/findtime/maxretry