How to Detect and Respond to a Server Breach

Responding to a compromised Linux server means working through five phases in order: detect the incident, contain it so it cannot spread or worsen, eradicate the actual cause, recover services safely, and document lessons learned. This structure follows the widely used SANS PICERL model and NIST Special Publication 800-61, and skipping a phase, especially containment before eradication, is one of the most common mistakes small teams make under pressure.

This guide assumes you already have a security baseline in place. If you have not yet reviewed your server's fundamentals, start with the VPS security checklist before an incident happens, not during one.

How do you detect a server breach?

You detect a server breach by watching for anomalies in authentication logs, running processes, network connections, and system files, then confirming whether an anomaly is actually malicious before reacting. According to NIST SP 800-61, detection and analysis is a distinct lifecycle phase in which the response team analyzes symptoms to decide whether an event genuinely constitutes an incident, rather than assuming every alert is one.

Practical signals worth checking on a Linux server include:

• Authentication anomalies: successful SSH logins from unfamiliar countries or IP ranges, logins at unusual hours, or a spike in failed login attempts immediately followed by a success.
• Unexpected processes or privilege changes: suspicious processes running with root privileges that you did not start, or a non-root user account that has somehow gained sudo access. CISA's guidance on uncovering malicious activity specifically calls out reviewing authentication activity, unexpected privilege changes, and processes running with root privileges as core indicators.
• Unusual outbound connections: a server that normally only accepts inbound web traffic suddenly making outbound connections to unfamiliar IPs, which often indicates a command-and-control channel or data exfiltration.
• Resource spikes: sudden sustained CPU or bandwidth usage with no corresponding legitimate workload, frequently a sign of cryptomining or the server being used to attack other systems.
• File and cron changes: modified system binaries, new SUID binaries, or unfamiliar entries in crontab and systemd timers, which are common persistence mechanisms.

None of these signals is proof on its own. The analysis step is confirming, with logs and command output, that what you are seeing cannot be explained by legitimate activity. Ongoing log review makes this phase dramatically faster; see our guide to server monitoring and log analysis for how to build that habit before you need it during an active incident.

How do you contain an active breach?

Containment means stopping the incident from spreading or getting worse while preserving evidence, typically by isolating the affected server's network access rather than shutting it down outright. NIST SP 800-61 distinguishes short-term containment, immediate actions like isolating a host, from long-term containment, sustainable measures such as network segmentation that let you keep operating safely while eradication is planned.

On a single small business VPS, practical containment steps include:

1. Restrict the server's firewall (or your cloud provider's security group) to allow only your own management IP, cutting off the attacker's access without destroying the running system.
2. Remove the server from any load balancer or DNS rotation so it stops receiving live production traffic.
3. Capture volatile evidence before it is lost: a memory snapshot if your infrastructure supports it, current process list, active network connections (ss -tulpn), and logged-in users.
4. Take a disk snapshot or image of the server in its current state. NIST guidance calls for acquiring forensic images before remediation changes the disk, since eradication steps like deleting files will destroy the evidence you need to determine root cause.
5. Rotate credentials that could have been exposed: SSH keys, application secrets, database passwords, and any API tokens stored on or accessible from the server.

Resist the urge to immediately delete suspicious files or kill processes before capturing evidence. Acting too fast at this stage is a common way small teams lose the ability to determine how the attacker got in, which then makes it far harder to be confident the same path is closed after cleanup.

How do you eradicate the cause of a breach?

Eradication means removing the actual root cause of the compromise, whether that is malware, a backdoor account, a vulnerable package, or a leaked credential, not just cleaning up its visible symptoms. This phase only starts after containment, because you cannot reliably clean a system that is still actively being accessed by an attacker.

For a server compromised at root level, the most reliable eradication path is rebuilding from a known-clean base image or snapshot rather than attempting to clean the live system in place. It is extremely difficult to be fully confident that every backdoor, modified binary, cron job, and SSH key an attacker planted has been found and removed on a system where the attacker had root access, since root access lets an attacker modify essentially anything, including the tools you would normally use to detect them.

If you do clean in place rather than rebuild, at minimum you should:

• Identify and remove any unauthorized user accounts, SSH keys in ~/.ssh/authorized_keys, and sudo grants. Check /etc/passwd for any account other than root with a UID of 0: it is a common way for an attacker to plant a second root-equivalent account that does not stand out in a casual user list.
• Review and clean cron jobs, systemd timers, and startup scripts for anything unfamiliar.
• Patch or remove the specific vulnerability that allowed entry, whether that was an outdated package, exposed service, or weak credential.
• Reinstall or verify system binaries against known-good checksums where a rootkit is suspected.

Whichever path you choose, do not restore normal access until you can point to the specific cause and confirm it is closed. Restoring service before eradication is complete risks a near-immediate repeat compromise through the same path.

How do you recover a server safely after a breach?

Recovery means restoring the server to normal operation deliberately, with verified clean backups and rotated credentials, and with monitoring in place to catch any sign the attacker retained access. NIST groups containment, eradication, and recovery as one connected lifecycle phase because rushing back into production before eradication and verification are complete tends to produce a repeat incident.

Before declaring the incident over:

• Restore from backups taken before the compromise window, not from the compromised system itself, and verify the backup content is clean.
• Rotate every credential again after rebuild, including ones rotated during containment, since a rebuilt system is a natural point to confirm nothing old survived.
• Reapply your full security baseline on the rebuilt system: firewall rules, SSH hardening, automatic updates, and monitoring, rather than assuming the old configuration was sufficient given that it was compromised once already.
• Set up or verify fail2ban (setup guide) and other brute-force protections on the rebuilt server before it goes back into production, particularly if credential stuffing or brute forcing was part of how the original breach occurred.
• Monitor more closely than usual for the first days and weeks after recovery, watching for any sign the same actor is probing again.

What happens in the lessons learned phase?

The lessons learned phase is a structured, written review conducted after recovery that documents what happened, how it was detected, how effective the response was, and what should change to prevent a repeat. Both the SANS PICERL model and the NIST incident response lifecycle treat this as a required final phase rather than an optional retrospective, because its output directly feeds back into the preparation phase for the next incident.

A useful lessons learned review answers, in plain language: How was the incident first noticed, and could it have been noticed sooner? What was the actual root cause? How long did containment and eradication take, and what slowed them down? What specific configuration or process change closes this exact gap? And which parts of the response plan worked well enough to keep as-is? Write this down while details are fresh, even for a small incident, since the same review is what turns an incident into a durable improvement rather than a one-time fire drill.

How do you prepare so the next incident is easier?

Preparation means having monitoring, hardened configuration, and a basic response plan in place before an incident happens, since NIST SP 800-61 identifies preparation as the first lifecycle phase specifically because everything else depends on it. A server with no log retention, no monitoring, and no documented baseline configuration makes every later phase, detection especially, dramatically slower and less reliable.

Concretely, preparation for a small business VPS means: centralizing and retaining logs long enough to investigate an incident that is discovered days after it started, hardening SSH access as covered in our SSH hardening guide, maintaining tested and recent backups stored separately from the production server, keeping an up-to-date inventory of what should normally be running so anomalies stand out, and knowing in advance who is responsible for making the call to isolate a server if something looks wrong. None of this needs to be elaborate for a small business, but it needs to exist before you need it.

Two companion guides go deeper on the recovery side: how to protect a small VPS from ransomware, and the VPS backup strategy guide, which explains how to keep a copy an attacker cannot delete. To audit for a hidden extra root account, see users, sudo and least privilege.

Evidence sources: auditd and journalctl.