In this blog, I will show how to recover the AWS EC2 Linux user key. We will discuss Preface in which how key got lost and then we will look into solutions.
A colleague mistakenly deleted the files in the
ec2-user user directory of an AWS EC2 Linux instance
.ssh/authorized_keys, which made it impossible to log in using SSH, so there is this record of reconfiguring the SSH connection key. This method requires the corresponding operation permissions in the AWS EC2 console and is suitable for the case where the private key of the local connection is lost or the key saved by the remote server is lost.
First, you need to log in to the AWS EC2 console page, follow the steps below to generate a new key pair and attach the root device volume of the instance to a temporary instance.
Network & SecurityCreate a new key pair on the EC2 console page. If the server loses the
authorized_keyskey file instead of the local key file, it is not necessary to generate a new key pair or generate a new key pair for use.
- Find the corresponding EC2 instance
Root device name(root device name), the name of a recording apparatus (e.g.,
Block devicesfind the corresponding case
Volume ID(Volume ID).
- Stop the original instance in the EC2 console (
- Create a new temporary instance, and select the key pair created in the first step as the key pair for the connection (if the server-side key is lost, the original key pair can be used here). If there are other existing instances available in the same area, you can choose not to create a new instance. In the case of using different key pairs, the operation in step 9 will be different, and there is no need to copy the secret in the instance. The key public key, but the corresponding public key is generated according to the private key of the key. For the generation method, please refer to the following.
- Detach the root device volume of the original instance from the original instance (
availableand attach it to the temporary instance after waiting for its status to change (
Attach Volume), and write down the device name set when attaching (for example
After performing the above steps, you need to connect to the temporary instance via SSH for subsequent operations:
lsblkDetermine whether the volume has been partitioned through the command, and the partitioned volume will have the
parttype of partition. For example, the
vxdadevice in the following example has partitions
vxda1, but the
vxdgdevice is not partitioned. If the volume has been partitioned, you will need to mount the corresponding partition (such as a partition
/dev/xvdf1) when mounting in the following steps , otherwise you will need to mount the original device (such as a device
$ lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT xvda 202:0 0 8G 0 disk └─xvda1 202:1 0 8G 0 part / xvdf 202:80 0 101G 0 disk └─xvdf1 202:81 0 101G 0 part xvdg 202:96 0 30G 0 diskNote: Different Linux distributions device name naming rules vary, for example, it may be
/dev/xvdkand so on.
- Create a temporary directory for mounting the volume, or you can directly use the current directory (used in the example, you
/mnt/tempvolcan replace it with other directories).
$ sudo mkdir /mnt/tempvol
mountcommands to mount the volume to a temporary directory. The commands used in different releases may be different.
# Amazon Linux、Ubuntu、Debian $ sudo mount /dev/xvdf1 /mnt/tempvol # Amazon Linux 2、CentOS、SUSE Linux 12、RHEL 7. $ sudo mount -o nouuid /dev/xvdf1 /mnt/tempvolIf the
mountfile system is damaged error is prompted when the command is executed, the command can be used
fsckto repair it.
$ sudo fsck /dev/xvdf1
- Update the
authorized_keysfile in the mounted volume. The location of the file will be different in different releases. If the directory is in Amazon Linux
/home/ec2-user/.ssh/authorized_keys, other distributions may use different locations or different user names, such as those used in Ubuntu
ubuntu. The target location in the mounted volume needs to add the mounted directory before the corresponding file directory, for example
- If you use the key public key in the temporary instance, copy it to the corresponding location in the mounted volume. $ cp .ssh/authorized_keys /mnt/tempvol/home/ec2-user/.ssh/authorized_keys If there is insufficient authority, please sudocopy and then modify the user and group to which the key public key in the mounted volume belongs. # $ ls /mnt/tempvol/home/ec2-user/.ssh total 4 -rw——- 1 < user> <group> 381 Dec 23 2019 authorized_keys # Copy key public key $ sudo cp .ssh/authorized_keys /mnt/tempvol/home/ec2-user/.ssh/authorized_keys # Modify user and user group $ sudo chown <user> :<group> /mnt/tempvol/home/ec2-user/.ssh/authorized_keys
- If you are using a newly generated unbound key, you need to use a
ssh-keygencommand to obtain the corresponding key public key.
$ ssh-keygen -y -f key-pair.pem
- Use the
unmountcommand to unmount the mounted volume.
$ sudo unmount /mnt/tempvol
After unmounting the mounted volume, you can disconnect the SSH connection and return to the AWS EC2 console for subsequent operations.
- Detach the original volume from the temporary instance, and
availablereattach it to the original instance after its state changes back . When attaching, the device name needs to be filled in as the original root device name recorded in step 2 (for example
- Use the
sshconnection test, if you no longer need to use the temporary instance after success, you can terminate it to avoid additional costs.
SSH connection appears
REMOTE HOST IDENTIFICATION HAS CHANGED!and the connection fails
After resetting the server-side user
authorized_keysand reconnecting to the server, the connection may fail and the following message will appear:
$ ssh ***@***.***.***.*** @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @ @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY! Someone could be eavesdropping on you right now (man-in-the-middle attack)! It is also possible that a host key has just been changed. The fingerprint for the ECDSA key sent by the remote host is SHA256:*******************************************. Please contact your system administrator. Add correct host key in /home/***/.ssh/known_hosts to get rid of this message. Offending ECDSA key in /home/***/.ssh/known_hosts:3 ECDSA host key for ***.***.***.*** has changed and you have requested strict checking. Host key verification failed.
The reason for this error is that the fingerprint information of the key stored locally does not match the fingerprint information of the ECDSA key sent by the remote server. If the error occurs when it is not trusted for modification, there may be a security risk of a man-in-the-middle attack.
Because the reason for the error here is that we have updated the user login key on the server, so we only need to refresh the key fingerprint information saved locally. The solution can be any of the following:
- Manually delete
~/.ssh/known_hoststhe information corresponding to the remote server IP in the file in the user directory .
- Use the
ssh-keygen -R <ip>command to update, after running, there will be the following similar prompt:
$ ssh-key -R ***.***.***.*** # Host ***.***.***.*** found: line xxx /home/***/.ssh/known_hosts updated. Original contents retained as /home/***/.ssh/known_hosts.old