SSH Hardening for Business Linux Servers

SSH is one of the most important controls on a Linux server. It is also one of the easiest to treat too casually.

For many small teams, SSH access has grown over time: old keys remain trusted, root login is still possible, sudo access is broad, and nobody is completely sure which users still need access. The server may still work every day, but the access model has become harder to explain and harder to defend.

SSH hardening should make access safer and clearer without creating unnecessary operational risk.

The goal is not to apply every possible setting from a benchmark. The goal is to know who can log in, how they authenticate, what they can do, how access is removed, and how the team avoids locking itself out during changes.

For readers who only need the short version, a sensible SSH baseline is:

• named owners for the server and SSH access;
• no unknown or stale authorised keys;
• no direct root login where practical;
• password authentication disabled where practical;
• restricted network reachability;
• sudo access reviewed separately from login access;
• a tested rollback path before changing configuration.

Start with ownership and recovery

Before changing SSH configuration, confirm the basics.

Ask:

• What business service depends on this server?
• Who owns the server?
• Who approves access changes?
• Who currently has SSH access?
• Is console access available if SSH breaks?
• Is there a recent backup, snapshot or rebuild path?
• Is there a safe change window?
• Who can roll back the change if needed?

This matters because SSH hardening can break access. A technically correct setting can still cause an outage if it is applied without a recovery path.

For important systems, keep one active session open while testing a new SSH configuration from a second session. Do not close the original session until the new login path has been confirmed.

Know who can log in

Start by identifying current access.

Review:

• local user accounts;
• authorised SSH keys;
• privileged groups;
• sudo access;
• service accounts;
• deployment users;
• monitoring or backup users;
• vendor or contractor access;
• any shared accounts.

The basic question is simple:

Can the team explain every account and key that can access the server?

If not, access should be cleaned up before more advanced hardening settings are considered.

A useful review table looks like this:

Account	Purpose	Access owner	Still required?
deploy	Application deployment	Engineering lead	Yes
ubuntu	Default cloud image user	Platform lead	Review
contractor-old	Former project access	Unknown	Remove
backup-agent	Backup process	Operations	Yes

Prefer keys or centralised access over passwords

Password-based SSH access is often a weak default for internet-facing servers.

Where practical, disable SSH password authentication and require stronger access methods such as:

• SSH keys managed through an access process;
• centralised identity;
• short-lived certificates;
• bastion or zero-trust access;
• MFA-protected access paths.

For many lean teams, the first step is simpler: remove password login, confirm keys are current, and make sure access is owned.

If password login must remain temporarily, document why, restrict where it can be used from, and set a remediation date.

Remove or restrict root login

Direct root login makes access harder to attribute and increases the impact of a stolen credential.

A better baseline is:

• no direct root login over SSH;
• named user accounts for administrators;
• sudo for privilege escalation;
• logging of privileged activity;
• a documented break-glass process for emergencies.

In sshd_config, this usually means reviewing PermitRootLogin and setting it to a safer value such as no, where operationally appropriate.

Do not make this change blindly. Confirm that named administrative users can log in and use sudo before disabling root SSH access.

Use named users, not shared accounts

Shared SSH accounts create poor evidence. They make it harder to know who performed an action, who still needs access, and what should happen when a person leaves.

Where practical, use named users or a central access system rather than shared accounts.

If a shared account is unavoidable for a specific operational reason, document:

• why it exists;
• who is allowed to use it;
• how the credential or key is protected;
• how usage is logged;
• when the exception will be reviewed.

The more sensitive the server, the less acceptable shared administrative access becomes.

Keep authorised keys under control

Authorised keys files often become long-lived dumping grounds.

Review:

• keys for users who have left the business;
• keys belonging to old laptops;
• keys added for temporary troubleshooting;
• vendor keys;
• duplicate or unknown keys;
• keys without comments or ownership metadata;
• keys on default cloud image users.

A good SSH key process should answer:

• Who owns this key?
• Why does it exist?
• When was it added?
• When should it be removed?
• What systems can it access?

Where possible, manage SSH keys through configuration management, central identity or another repeatable process. Manual key edits are easy to forget and hard to audit.

Reduce where SSH is reachable from

SSH does not need to be open to the whole internet in most business environments.

Consider restricting SSH access through:

• cloud security groups;
• host firewalls;
• VPN;
• bastion hosts;
• zero-trust access brokers;
• office or managed IP ranges;
• just-in-time access workflows.

Changing the SSH port can reduce background noise, but it should not be treated as the main control. Strong authentication, restricted reachability and clear ownership matter more.

For cloud-hosted Linux servers, review both the host firewall and the cloud network controls. A hardened sshd_config does not compensate for a forgotten security group that exposes SSH broadly.

Review useful sshd_config settings

SSH configuration should be reviewed in context. Some settings are broadly useful; others depend on how the server is used.

Common areas to review include:

Setting area	Why it matters
PasswordAuthentication	Controls whether password login is allowed
PermitRootLogin	Controls direct root access
PubkeyAuthentication	Confirms public key authentication support
AllowUsers or AllowGroups	Restricts which users or groups can log in
MaxAuthTries	Reduces repeated authentication attempts
LoginGraceTime	Limits how long unauthenticated sessions remain open
X11Forwarding	Often unnecessary on servers
AllowTcpForwarding	Should be enabled only where needed
ClientAliveInterval and ClientAliveCountMax	Helps manage idle sessions
PermitEmptyPasswords	Should not allow empty password login

Treat this as a review list, not a universal copy-and-paste configuration. Some environments legitimately need port forwarding, bastion workflows or automation-specific access. The key is to make those needs explicit.

Protect sudo as part of SSH hardening

SSH hardening is incomplete if every user who can log in can immediately become root.

Review:

• /etc/sudoers;
• files under /etc/sudoers.d/;
• members of privileged groups;
• passwordless sudo rules;
• deployment user privileges;
• service account privileges;
• whether sudo usage is logged.

A good access model separates login access from administrative power.

Some users may need to log in only to view logs or restart a service. Others may need full administrative rights. Treating every operational user as a full administrator increases risk and weakens evidence.

Monitor authentication and access events

A hardened SSH service should leave useful evidence.

Review whether the team can see:

• successful SSH logins;
• failed login attempts;
• sudo usage;
• login attempts for disabled or invalid users;
• access from unusual locations;
• changes to authorised keys;
• changes to SSH configuration.

On many Linux systems, this evidence appears in authentication logs, system journals and sudo logs. For important servers, consider centralising logs so they are still available if the host is compromised or lost.

The question to ask is:

If an old key was used to access this server last night, would anyone know?

Avoid locking yourself out

SSH changes should be made with a rollback plan.

Before applying changes:

• confirm console or out-of-band access;
• confirm at least one valid admin login path;
• keep an existing session open;
• test the new configuration with sshd -t where available;
• reload rather than restart where appropriate;
• test a new login before closing the old session;
• document the previous configuration;
• have a backup or snapshot for important systems.

A common failure mode is disabling root login or password authentication before confirming that named users and keys actually work. That is avoidable with a simple test plan.

Evidence customers and leadership may ask for

SSH hardening can support customer security reviews, insurance questions and internal assurance.

Useful evidence might include:

Claim	Useful evidence	Stronger evidence
Password SSH is disabled	sshd_config setting or managed baseline	Change record plus periodic configuration check
Root SSH login is disabled	PermitRootLogin setting	Evidence of named admin users and sudo logging
Access is limited to approved users	User and key review notes	Dated access review with removals recorded
SSH is not broadly exposed	Firewall and cloud security group rules	Exposure review covering host and cloud controls
Privileged actions are auditable	Sudo configuration and logs	Centralised logging and review process

Do not share sensitive keys, raw logs containing customer data, or unnecessary internal details. Evidence should prove the control without exposing the environment.

A practical SSH hardening checklist

For a business Linux server, a sensible SSH baseline includes:

• clear server owner;
• documented current SSH users;
• no unknown or stale authorised keys;
• no direct root login where practical;
• password authentication disabled where practical;
• named users instead of shared admin accounts;
• sudo access reviewed and limited;
• SSH exposure restricted by firewall or cloud controls;
• authentication and sudo logs retained;
• emergency access and rollback path confirmed;
• access review repeated on a schedule.

This is not exotic hardening. It is the operational baseline that makes a Linux server easier to trust and easier to manage.

What good SSH hardening leaves behind

Good SSH hardening should leave the team with a clearer access model, safer authentication, fewer stale keys, reduced exposure and better evidence.

The best outcome is not a server that is difficult to operate. The best outcome is a server where access is boring, explainable and recoverable.

That means the team knows who can log in, why they can log in, what they can do, how their access is removed, and what happens if something goes wrong.