deleted glibc x86_64 on a VPS, centOS 6

Can you please help me solve this problem.

I’ve “accidentally” rpm -e the glibc.x86_64 libraries and made my VPS unusable as all commands throw one of the following errors:

[root@panel lib64]# yum
bash: /usr/bin/yum: /usr/bin/python: bad interpreter: No such file or directory
[root@panel lib64]# ls 
bash: /bin/ls: /lib64/ld-linux-x86-64.so.2: bad ELF interpreter: No such file or directory

If any of the following can help:

1) I am still connected to the shell

2) I can’t boot up to recovery as it is a remote VPS

3) there is a i686 glibc still installed

4) I have the x86_64 versions .rpm file in my home dir.

5) There is /lib/ld-linux.so.2, can I somehow point the system to using that one?

6) I have a /opt/glibc-2.14/lib/ld-linux-x86-64.so.2 and I can execute commands with prefixing it, but the crucial ones, like rpm2cpio or wget throw shared object errors.

Can I get this solved using static commands or putting some other lib in the path or any other way?

Thank you all in advance.

Answer

No additional programs can be executed as you removed the linker. All you really have is the running bash shell. No wget, no cp.

This can go two ways. A routine backup restore and rebuild of what this instance did, start this process now. Or very creative heroics in extracting data off of it, and getting glibc back on.

Need to copy in binaries at least as new as EL6’s glibc version 2.12. Only using bash. In theory it is possible using only built-ins. On some other host, extract with rpm2cpio, use /dev/tcp/ to download them to the broken host. In practice, this is ugly and difficult to get functional. Need to use unencrypted protocols, and manually type the request in.

Instead, attach (a snapshot of) the VM disk image to a different, working EL6 VM and operate on it there. Mount it somewhere, say /mnt. Use yum to get glibc back, but install it in the broken root:

 yum --installroot=/mnt install glibc

Back up any important data while you have it mounted. Swing it over to the broken instance again, and boot.


Now for the postmortem.

Fixing a broken instance is nice to get your data back, but may not be possible. Backups remain the recovery that must always work. Be sure rebuild and restore are possible.

You had to use rpm -e --nodeps to get in this situation. Never do that. Dependencies exist because packages require one another. glibc’s rpm package description (rpm -qi glibc) mentions its importance, although to be fair after a lot of words:

This particular package contains the most important sets of shared
libraries: the standard C library and the standard math library.
Without these two libraries, a Linux system will not function.

If you cannot solve a rpm transaction with yum, stop. Question the integrity of the package database. Question the sanity of the yum repos that made someone think --nodeps was ever a good idea.

Attribution
Source : Link , Question Author : Vedran B , Answer Author : John Mahowald

Leave a Comment