How to Fix Permission Denied (publickey) Error on Linux SSH

The Secure Shell protocol or SSH serves as the backbone of modern server administration. It allows engineers to manage infrastructure across the globe with encrypted tunnels that ensure confidentiality and integrity. However, even the most seasoned DevOps professionals occasionally encounter the dreaded Permission Denied publickey error. This message acts as a gatekeeper, slamming shut the door to your remote server and often causing significant frustration during critical deployment windows or emergency maintenance tasks. Understanding why this happens requires a deep dive into the handshake process, the file system permissions governing your home directory, and the configuration nuances of both the client and the server.

To fix the error, you must first understand how SSH establishes trust. When you initiate a connection, your client presents a public key to the remote server. The server checks this key against its list of authorized identities, typically located in a file named authorized_keys within the user home directory. If the server cannot verify that the presented key matches the one on record, or if it finds the configuration settings for the authentication process unacceptable, it terminates the connection. The Permission Denied publickey error specifically indicates that the server has rejected all offered authentication methods, or that it is failing to reach the authentication stage entirely due to restrictive security policies.

The most common culprit behind this error is incorrect file system permissions on the remote server. SSH is famously strict about security. If your home directory, the .ssh directory, or the authorized_keys file itself has permissions that are too open, the SSH daemon will refuse to read them. This is a deliberate defense mechanism designed to prevent unauthorized users from tampering with your identity files.

On a standard Linux system, your home directory should be writable only by you. The .ssh folder must be restricted to a mode of 700, which grants read, write, and execute permissions exclusively to the owner. Furthermore, the authorized_keys file needs to be set to a mode of 600. If these files are group writable or world readable, the server daemon assumes the configuration is insecure and effectively ignores your keys. When you see the publickey error, the very first step should always be to verify these permissions. You can inspect them by logging in through a console connection or a recovery interface provided by your hosting provider, such as a VNC console or an out of band management system like iLO or IPMI.

Before you start changing settings, you must see exactly what is happening during the connection attempt. SSH provides an incredibly powerful diagnostic tool known as verbose mode. By adding the v flag to your command, you can peel back the layers of the authentication process. Using ssh v user@host will output every step of the handshake, from key exchange to the specific point where the server closes the connection.

When you run this command, watch closely for the list of identities being offered. Does the client actually load your private key from your local machine? If the output shows that it is trying every key in your agent but finding no match, the issue might reside on your local machine rather than the server. Conversely, if the server explicitly states that it is skipping public key authentication due to restrictive settings, you know the problem is server side. Verbose mode turns a blind guess into a structured forensic investigation.

Sometimes the issue is not the files themselves, but the rules governing how the server handles them. The SSH daemon configuration file, located at /etc/ssh/sshd_config, contains directives that dictate authentication behavior. A common point of failure occurs when the PubkeyAuthentication directive is disabled. If this is set to no, the server will ignore all public key attempts regardless of how perfect your permissions are.

Another potential hurdle involves the AuthorizedKeysFile directive. If you have moved your keys to a non standard location or if the path is misconfigured, the server will look for them in the wrong place. Always ensure that the path points to the correct location relative to the user home directory. After making any changes to this file, you must remember to restart the service using systemctl restart sshd. However, be careful not to close your existing session before verifying that the new settings work, as you might lock yourself out entirely if a syntax error exists in the configuration.

Your local machine acts as the first line of defense in the authentication chain. If you have multiple keys for different servers, it is easy for the SSH agent to lose track of which key to offer first. When you connect, the client cycles through keys until the server accepts one or exhausts the list. If your local agent is not holding the correct private key, or if you have too many keys loaded, the server might reach its maximum authentication attempts limit before it gets to the right one.

You can manage your local keys using the ssh add command. If you suspect your local environment is the source of the trouble, clear your current agent using ssh add D and then manually add your specific key using ssh add path/to/your/private/key. This forces the agent to prioritize your key. This is particularly relevant when working with large infrastructure setups where you might have dozens of keys for different cloud environments. Keeping your local keychain tidy is a vital habit for any DevOps engineer.

While the Permission Denied error usually points to authentication issues, it can sometimes be confused with the host key verification failure. If you have reinstalled your server or changed its IP address, your local known_hosts file will contain a stale record. The server will present a new identity that does not match the old signature, and your client will rightly view this as a potential security breach.

While this typically results in a warning about potential man in the middle attacks, it can sometimes manifest as a login failure if you have configured your client to strictly reject unknown hosts. To resolve this, you can remove the offending line from your local ~/.ssh/known_hosts file. Use the command ssh keygen f “/home/youruser/.ssh/known_hosts” R “remote_server_ip” to safely strip the old signature. Once the record is gone, the next attempt will prompt you to accept the new fingerprint, restoring the trusted connection.

In complex enterprise environments, the security policy known as SELinux can block access to the .ssh directory even if the file permissions appear perfectly correct. SELinux enforces mandatory access control and relies on security contexts to define what processes can do with specific files. If your .ssh directory has the incorrect context, the SSH daemon will be denied access to read the files, triggering the publickey error.

You can inspect the security context of your files using the ls Z command. If you find that the contexts are improperly set, you can restore them using the restorecon command. Specifically, running restorecon r v ~/.ssh will recursively reset the security contexts for your SSH configuration directory to the default system values. This is an advanced troubleshooting step that is frequently overlooked but critical in highly locked down Linux distributions like RHEL, CentOS, or AlmaLinux.

Some servers are configured to require multi factor authentication, or they might demand a specific order of authentication. If your server expects a password and a key, or if it requires a challenge response, standard key based authentication might fail if the configuration is not explicitly set to allow public key as a fallback. You can experiment with different options from the command line using the o flag.

For example, you can force the client to use only the identity file you specify by running ssh i /path/to/key o IdentitiesOnly=yes user@host. This bypasses the SSH agent entirely and tells the client to use only the identity provided. If this succeeds, it confirms that your keys and permissions are correct, but that your local SSH agent configuration was causing the conflict. This is a very powerful way to isolate variables when troubleshooting.

In modern DevOps, we rarely configure servers manually. We use tools like Ansible, Terraform, or Chef to manage thousands of nodes. A single typo in an automation script can propagate an incorrect SSH configuration across your entire fleet. If you have deployed a new version of your infrastructure and suddenly find yourself unable to log in, the culprit is likely the automation code.

Check your provisioning scripts for lines that modify user home directories or SSH configuration files. It is common to accidentally apply permissions that are too loose when attempting to automate access for service accounts. If you are using Ansible, check your group_vars and role files for any tasks that manipulate authorized_keys files. Automated tools are powerful, but they can lock you out with incredible efficiency if the logic is flawed.

The most effective way to recover from a Permission Denied error is to have a path that does not rely on SSH at all. Never rely exclusively on remote SSH access for critical production infrastructure. Always ensure that you have access to a console or a recovery terminal provided by your cloud provider or your hardware vendor.

These tools allow you to bypass the SSH daemon entirely. Once you are logged into the physical or virtual console, you are interacting with the system as a local user. This allows you to check system logs in /var/log/auth.log or /var/log/secure, which will provide the specific reason the SSH daemon rejected your connection. The logs often contain explicit messages stating exactly which file had incorrect permissions or why a specific key was rejected. Without log access, you are effectively flying blind.

When multiple engineers share access to a single server, the authorized_keys file can become a mess of conflicting entries. If a colleague leaves the organization or if their key expires, managing their access becomes a maintenance burden. Using a centralized SSH key management system or even a simple script to synchronize keys from a central repository is highly recommended.

However, be aware that these systems can also fail. If a synchronization job fails, it might wipe your keys or corrupt the authorized_keys file. Always maintain a backup of your own administrative key in a secure, offline location. Having a backdoor, such as a root login allowed only from a specific jump host or a secondary identity with hardcoded credentials in the local repository, can save your business from significant downtime.

Never modify SSH configurations on a live server without verifying the syntax. The sshd command has a useful t flag that tests your configuration file for syntax errors before you apply them. Running sshd t will report any issues within your /etc/ssh/sshd_config file, ensuring that you do not leave the service in a state where it refuses to start.

Furthermore, always keep a secondary terminal session open while making changes. If you apply a bad configuration, you can immediately revert it using your existing connection. If you close your terminal, you lose that final safety net. This simple discipline of keeping a persistent administrative session active is one of the most effective strategies for preventing self inflicted outages.

While it might be tempting to loosen permissions to fix the error, always remember that you are reducing the security of your server. Never chmod 777 your .ssh directory. The standard 700 for the directory and 600 for the file exist for a reason. If your server is exposed to the public internet, a misconfigured SSH directory is an open invitation for malicious actors to attempt unauthorized entry.

Consider disabling password authentication entirely once you have verified your public key access. This drastically reduces the attack surface against brute force attempts. Also, consider using alternative authentication mechanisms such as FIDO/U2F security keys or certificates. These methods provide a much higher level of security and eliminate the risk of private key theft from your local machine.

Sometimes the SSH service itself enters a state where it is no longer responding to authentication requests correctly. A systemd service might be running, but the underlying process might be hung. Restarting the service is a common troubleshooting step, but if the problem persists, check the status of the service using systemctl status sshd.

If you see error messages related to file locking or shared memory, you might be facing a deeper operating system issue. Occasionally, updating the operating system or the SSH package itself can resolve underlying bugs. Always keep your server software updated, as vulnerabilities in the SSH implementation are patched frequently. An outdated SSH daemon is not just a risk, it is a liability.

The Permission Denied publickey error is a rite of passage for every system administrator. It is a frustrating obstacle, but it is also a fundamental lesson in the importance of security, permissions, and configuration discipline. By mastering the diagnostic tools like verbose mode, understanding the nuances of file system permissions, and maintaining robust out of band recovery paths, you can turn this error from a catastrophic failure into a manageable routine event.

Always approach the problem methodically. Start with the simplest explanations, such as file permissions and local agent state, before moving to more complex issues like security contexts or service configurations. Document your fixes and refine your automation scripts to ensure that you do not encounter the same issue twice. Reliability is built on the ruins of past failures, and with a systematic approach, you can ensure that your infrastructure remains secure, accessible, and resilient in the face of even the most stubborn authentication errors.