How to Secure a Linux VPS: SSH Keys, UFW and Fail2ban in the First Hour

To secure a new Linux VPS, do six things in the first hour: install updates, create a non-root sudo user, log in with SSH keys, disable password and root logins, turn on a firewall that only allows the ports you use, and add fail2ban plus automatic security updates. That removes the attacks that hit every public server within minutes of it going online: automated password guessing against SSH and exploits for packages that are months out of date.
The commands below are for Ubuntu 22.04 and 24.04 and also work on Debian 12. Replace 203.0.113.10 with your server's IP and deploy with the username you want.
Golden rule: keep your current SSH session open until you have confirmed, from a second terminal, that the new login works. If something goes wrong, the open session is your way back in.
Step 1: Update the system
A fresh image can be weeks behind on security patches. Log in as root (or the default user your provider gave you) and update everything:
apt update && apt full-upgrade -y
[ -f /var/run/reboot-required ] && rebootThe second line reboots only if a kernel or core library update needs it. If you are still choosing a distribution, our comparison of Ubuntu, Debian, AlmaLinux and Rocky for a VPS covers support periods and package freshness.
Step 2: Create a non-root user with sudo
Daily work as root means one typo or one compromised script has unlimited power. Create a normal user and give it sudo rights:
adduser deploy
usermod -aG sudo deployadduser asks for a password. Choose a strong one: you will still need it for sudo, even after SSH passwords are switched off.
Step 3: Set up SSH key login
An SSH key pair replaces the password with a private key that never leaves your computer. Generate it on your laptop, not on the server:
ssh-keygen -t ed25519 -C "deploy laptop"Accept the default path and set a passphrase. Then copy the public key to the server:
ssh-copy-id [email protected]On Windows 10 or 11, ssh-keygen works in PowerShell, but ssh-copy-id is missing. Use this instead:
type $env:USERPROFILE\.ssh\id_ed25519.pub | ssh [email protected] "mkdir -p ~/.ssh && chmod 700 ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys"Now open a new terminal and test:
ssh [email protected]
sudo whoamiIf you get in without typing the account password (only the key passphrase) and sudo whoami prints root, the key works. Do not continue until it does.
Step 4: Disable password and root login over SSH
On Ubuntu 22.04 and 24.04, the main /etc/ssh/sshd_config includes every file in /etc/ssh/sshd_config.d/ at the top, and for most settings sshd uses the first value it reads. Some cloud images ship a file such as 50-cloud-init.conf containing PasswordAuthentication yes. A drop-in file whose name sorts earlier wins over it:
sudo nano /etc/ssh/sshd_config.d/00-hardening.confPermitRootLogin no
PasswordAuthentication no
KbdInteractiveAuthentication no
PubkeyAuthentication yes
MaxAuthTries 3
LoginGraceTime 30
X11Forwarding no
AllowUsers deployCheck the syntax, then confirm the values sshd will actually use:
sudo sshd -t
sudo sshd -T | grep -Ei 'permitrootlogin|passwordauthentication|kbdinteractive|allowusers'If sshd -t complains that /run/sshd is missing, run sudo mkdir -p /run/sshd and test again. When the output is clean, apply the change:
sudo systemctl restart sshUbuntu 24.04 note: SSH is socket-activated by default, so ssh.socket owns the listening port. Normal settings like the ones above apply after systemctl restart ssh. If you also change Port or ListenAddress, run sudo systemctl daemon-reload and sudo systemctl restart ssh.socket, or the server keeps listening on the old port. On 22.04 a plain restart of ssh covers everything.
From a second terminal, confirm that root and password logins are refused:
ssh [email protected]
ssh -o PubkeyAuthentication=no [email protected]Both should end with Permission denied (publickey). Changing the SSH port is optional. It reduces log noise from bots but is not a security control by itself; keys and the firewall are.
Step 5: Turn on the UFW firewall
UFW is a simple front end for the kernel firewall and is installed on Ubuntu by default. Allow SSH before enabling it, or you will lock yourself out:
sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw allow OpenSSH
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
sudo ufw enable
sudo ufw status verboseOnly open what you serve. A database should never be reachable from the internet; if an app on another server needs it, allow that one IP: sudo ufw allow from 198.51.100.20 to any port 5432 proto tcp. If you moved SSH to a custom port, allow it (for example sudo ufw allow 2222/tcp) before removing the OpenSSH rule. UFW also filters IPv6 by default, which matters because your VPS has a public IPv6 range as well.
One trap: Docker writes its own firewall rules, and ports published with -p 8080:80 bypass UFW. Bind containers to 127.0.0.1 and put a reverse proxy in front.
Step 6: Install fail2ban
Keys already stop password guessing, but fail2ban cuts the noise and protects any other service with a login, such as a mail server or a web admin panel. It watches logs and bans IPs that fail too often.
sudo apt install -y fail2ban python3-systemdNever edit jail.conf, since updates overwrite it. Create /etc/fail2ban/jail.local:
[DEFAULT]
bantime = 1h
bantime.increment = true
findtime = 10m
maxretry = 5
ignoreip = 127.0.0.1/8 ::1
[sshd]
enabled = true
port = ssh
backend = systemd
maxretry = 3Add your office's fixed IP to ignoreip so you never ban yourself, and set port = 2222 if you changed the SSH port. Then:
sudo systemctl enable --now fail2ban
sudo systemctl restart fail2ban
sudo fail2ban-client status sshdTo release an IP you banned by mistake: sudo fail2ban-client set sshd unbanip 198.51.100.7.
Step 7: Automatic security updates
Most compromises use known vulnerabilities that already have a patch. Ubuntu's unattended-upgrades installs security updates daily:
sudo apt install -y unattended-upgrades
sudo dpkg-reconfigure -plow unattended-upgrades
sudo unattended-upgrade --dry-run --debugAnswer "Yes" in the dialog. Kernel updates still need a reboot. You can let the system do it at a quiet hour by setting Unattended-Upgrade::Automatic-Reboot "true"; and Unattended-Upgrade::Automatic-Reboot-Time "04:00"; in /etc/apt/apt.conf.d/50unattended-upgrades, or reboot manually each week.
Step 8: Check what is listening
Every listening port is a door. List them:
sudo ss -tulpnLook at the local address column. 0.0.0.0:22 or [::]:22 means reachable from anywhere (subject to UFW). 127.0.0.1:3306 means local only, which is what you want for MySQL, PostgreSQL and Redis. If you see a service you do not recognise, find it with systemctl status and stop or remove it. If the output is new to you, our list of essential Linux commands for VPS beginners explains this and the other everyday tools.
Step 9: Backups and what the firewall cannot do
Hardening reduces risk; it does not remove it. Keep at least one copy of your data off the server and test a restore. On an IM HOST Linux VPS, daily backup is available as an optional paid add-on, and our 3-2-1 VPS backup and restore guide shows how to combine it with an offsite copy.
Also be clear on limits: UFW and fail2ban work on packets that already reached your server. A large flood saturates the link before your firewall sees it, which is why network-level filtering matters. Our explainer on how DDoS attacks work and which layer stops which attack covers that side. Once the base is secure, continue with the web stack in our first VPS setup guide for a fast website.
Your first-hour checklist
System fully updated and rebooted if required
Non-root sudo user created
SSH key login tested from a second terminal
Root login and password authentication disabled, confirmed with
sshd -TUFW enabled with only required ports
fail2ban running with an sshd jail
Unattended security upgrades on
No unexpected services in
ss -tulpnBackup in place and one restore tested
Frequently asked questions
Do I still need fail2ban if I use SSH keys?
It is not strictly required for SSH once passwords are off, but it keeps logs readable, slows scanners and protects other services that still use passwords, such as mail or web admin logins.
Should I change the default SSH port 22?
It is optional. A custom port reduces automated login attempts in your logs but does not stop a targeted attacker. Keys, disabled passwords and a firewall do the real work. On Ubuntu 24.04, remember to reload the socket after changing the port.
What if I lock myself out of SSH?
Use the provider's console access to log in locally and undo the change, usually by fixing the sshd drop-in file or running ufw allow OpenSSH. If you have no console access, contact support. This is why the second-session test matters.
Is UFW enough as a firewall?
For a single server, UFW with default deny and a short allow list is enough for most workloads. It does not replace network-level DDoS filtering or a web application firewall for login and form abuse.
Every IM HOST Linux VPS comes with full root access, KVM isolation and DDoS protection, so the steps above are all you need to start from a clean, locked-down base. If anything in this checklist does not behave as described, our engineers are available 24/7 to look at it with you.
More from our blog
Discover more practical guides and product insights from the IM Host team.
View all articles