AWS EC2 Linux user key loss recovery


In this blog, I will show how to recover the AWS EC2 Linux user key. We will discuss Preface in which how key got lost and then we will look into solutions.


A colleague mistakenly deleted the files in the ec2-user user directory of an AWS EC2 Linux instance .ssh/authorized_keys, which made it impossible to log in using SSH, so there is this record of reconfiguring the SSH connection key. This method requires the corresponding operation permissions in the AWS EC2 console and is suitable for the case where the private key of the local connection is lost or the key saved by the remote server is lost.


First, you need to log in to the AWS EC2 console page, follow the steps below to generate a new key pair and attach the root device volume of the instance to a temporary instance.

  • Network & Security Create a new key pair on the EC2 console page. If the server loses the authorized_keys key file instead of the local key file, it is not necessary to generate a new key pair or generate a new key pair for use.
ec2 linux user key loss recovery
  • Find the corresponding EC2 instance Storage(memory) tab Root device name(root device name), the name of a recording apparatus (e.g., /dev/sda1or /dev/xvda) and Block devices find the corresponding case Volume ID(Volume ID).
  • Stop the original instance in the EC2 console ( Stop instance).
  • Create a new temporary instance, and select the key pair created in the first step as the key pair for the connection (if the server-side key is lost, the original key pair can be used here). If there are other existing instances available in the same area, you can choose not to create a new instance. In the case of using different key pairs, the operation in step 9 will be different, and there is no need to copy the secret in the instance. The key public key, but the corresponding public key is generated according to the private key of the key. For the generation method, please refer to the following.
  • Detach the root device volume of the original instance from the original instance ( Detach Volume), availableand attach it to the temporary instance after waiting for its status to change ( Attach Volume), and write down the device name set when attaching (for example /dev/xvdf).

After performing the above steps, you need to connect to the temporary instance via SSH for subsequent operations:

  1. lsblk Determine whether the volume has been partitioned through the command, and the partitioned volume will have the part type of partition. For example, the vxdadevice in the following example has partitions vxda1, but the vxdgdevice is not partitioned. If the volume has been partitioned, you will need to mount the corresponding partition (such as a partition /dev/xvdf1) when mounting in the following steps , otherwise you will need to mount the original device (such as a device /dev/vxdg). $ lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT xvda 202:0 0 8G 0 disk └─xvda1 202:1 0 8G 0 part / xvdf 202:80 0 101G 0 disk └─xvdf1 202:81 0 101G 0 part xvdg 202:96 0 30G 0 disk Note: Different Linux distributions device name naming rules vary, for example, it may be /dev/sdf/dev/vdg/dev/xvdkand so on.
  2. Create a temporary directory for mounting the volume, or you can directly use the current directory (used in the example, you /mnt/tempvolcan replace it with other directories). $ sudo mkdir /mnt/tempvol
  3. Use mountcommands to mount the volume to a temporary directory. The commands used in different releases may be different. # Amazon Linux、Ubuntu、Debian $ sudo mount /dev/xvdf1 /mnt/tempvol # Amazon Linux 2、CentOS、SUSE Linux 12、RHEL 7. $ sudo mount -o nouuid /dev/xvdf1 /mnt/tempvol If the mountfile system is damaged error is prompted when the command is executed, the command can be used fsck to repair it.$ sudo fsck /dev/xvdf1
  4. Update the authorized_keysfile in the mounted volume. The location of the file will be different in different releases. If the directory is in Amazon Linux /home/ec2-user/.ssh/authorized_keys, other distributions may use different locations or different user names, such as those used in Ubuntu ubuntu. The target location in the mounted volume needs to add the mounted directory before the corresponding file directory, for example /mnt/tempvol/home/ec2-user/.ssh/authorized_keys.
    • If you use the key public key in the temporary instance, copy it to the corresponding location in the mounted volume. $ cp .ssh/authorized_keys /mnt/tempvol/home/ec2-user/.ssh/authorized_keys If there is insufficient authority, please sudocopy and then modify the user and group to which the key public key in the mounted volume belongs. # $ ls /mnt/tempvol/home/ec2-user/.ssh total 4 -rw——- 1 < user> <group> 381 Dec 23 2019 authorized_keys # Copy key public key $ sudo cp .ssh/authorized_keys /mnt/tempvol/home/ec2-user/.ssh/authorized_keys # Modify user and user group $ sudo chown <user> :<group> /mnt/tempvol/home/ec2-user/.ssh/authorized_keys
    • If you are using a newly generated unbound key, you need to use a ssh-keygencommand to obtain the corresponding key public key. $ ssh-keygen -y -f key-pair.pem
  5. Use the unmountcommand to unmount the mounted volume. $ sudo unmount /mnt/tempvol

After unmounting the mounted volume, you can disconnect the SSH connection and return to the AWS EC2 console for subsequent operations.

  1. Detach the original volume from the temporary instance, and availablereattach it to the original instance after its state changes back . When attaching, the device name needs to be filled in as the original root device name recorded in step 2 (for example /dev/xvda).
  2. Use the sshconnection test, if you no longer need to use the temporary instance after success, you can terminate it to avoid additional costs.

SSH connection appears REMOTE HOST IDENTIFICATION HAS CHANGED!and the connection fails

After resetting the server-side user authorized_keysand reconnecting to the server, the connection may fail and the following message will appear:

$ ssh ***@***.***.***.***
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that a host key has just been changed.
The fingerprint for the ECDSA key sent by the remote host is
Please contact your system administrator.
Add correct host key in /home/***/.ssh/known_hosts to get rid of this message.
Offending ECDSA key in /home/***/.ssh/known_hosts:3
ECDSA host key for ***.***.***.*** has changed and you have requested strict checking.
Host key verification failed.

The reason for this error is that the fingerprint information of the key stored locally does not match the fingerprint information of the ECDSA key sent by the remote server. If the error occurs when it is not trusted for modification, there may be a security risk of a man-in-the-middle attack.

Because the reason for the error here is that we have updated the user login key on the server, so we only need to refresh the key fingerprint information saved locally. The solution can be any of the following:

  • Manually delete ~/.ssh/known_hoststhe information corresponding to the remote server IP in the file in the user directory .
  • Use the ssh-keygen -R <ip>command to update, after running, there will be the following similar prompt:$ ssh-key -R ***.***.***.*** # Host ***.***.***.*** found: line xxx /home/***/.ssh/known_hosts updated. Original contents retained as /home/***/.ssh/known_hosts.old


Be the first to comment on "AWS EC2 Linux user key loss recovery"

Leave a comment

Your email address will not be published.