A server restart produced this output on the console:
ZFS: i/o error - all block copies unavailable ZFS: failed to read pool zroot directory object qptzfsboot: failed to mount default pool zroot FreeBSD/x86 boot ZFS: i/o error - all block copies unavailable ZFS: can't fild dataset 0 Default: zroot/<0x0> boot:
I booted the host from the usb livecd and mounted the /etc directory under /tmp to enable ssh access:
ifconf -a # get available i/f names ifconf em0 inet 192.168.216.46 route add default 192.168.216.1 hostname vhost06.internal mkdir /tmp/etc mount_unionfs /tmp/etc /etc echo 'PermitRootLogin yes' >> /etc/sshd_config passwd Changing local password for root New Password: Retype New Password: service sshd onestart
There are no zfs pools available to import:
root@vhost06:~ # zpool status no pools available root@vhost06:~ # zpool list no pools available root@vhost06:~ # zfs list no datasets available
gpart
shows this geometry:gpart show
=> 40 15628053088 ada0 GPT (7.3T) 40 1024 1 freebsd-boot (512K) 1064 984 - free - (492K) 2048 16777216 2 freebsd-swap (8.0G) 16779264 15611273216 3 freebsd-zfs (7.3T) 15628052480 648 - free - (324K) => 40 15628053088 ada1 GPT (7.3T) 40 1024 1 freebsd-boot (512K) 1064 984 - free - (492K) 2048 16777216 2 freebsd-swap (8.0G) 16779264 15611273216 3 freebsd-zfs (7.3T) 15628052480 648 - free - (324K) => 40 15628053088 ada2 GPT (7.3T) 40 1024 1 freebsd-boot (512K) 1064 984 - free - (492K) 2048 16777216 2 freebsd-swap (8.0G) 16779264 15611273216 3 freebsd-zfs (7.3T) 15628052480 648 - free - (324K) => 40 15628053088 ada3 GPT (7.3T) 40 1024 1 freebsd-boot (512K) 1064 984 - free - (492K) 2048 16777216 2 freebsd-swap (8.0G) 16779264 15611273216 3 freebsd-zfs (7.3T) 15628052480 648 - free - (324K) => 40 15628053088 diskid/DISK-VAGWJ6VL GPT (7.3T) 40 1024 1 freebsd-boot (512K) 1064 984 - free - (492K) 2048 16777216 2 freebsd-swap (8.0G) 16779264 15611273216 3 freebsd-zfs (7.3T) 15628052480 648 - free - (324K) => 40 15628053088 diskid/DISK-VAGWV89L GPT (7.3T) 40 1024 1 freebsd-boot (512K) 1064 984 - free - (492K) 2048 16777216 2 freebsd-swap (8.0G) 16779264 15611273216 3 freebsd-zfs (7.3T) 15628052480 648 - free - (324K) => 40 15628053088 diskid/DISK-VAHZAD2L GPT (7.3T) 40 1024 1 freebsd-boot (512K) 1064 984 - free - (492K) 2048 16777216 2 freebsd-swap (8.0G) 16779264 15611273216 3 freebsd-zfs (7.3T) 15628052480 648 - free - (324K) => 40 15628053088 diskid/DISK-VAH3PXYL GPT (7.3T) 40 1024 1 freebsd-boot (512K) 1064 984 - free - (492K) 2048 16777216 2 freebsd-swap (8.0G) 16779264 15611273216 3 freebsd-zfs (7.3T) 15628052480 648 - free - (324K) => 1 30240767 da0 MBR (14G) 1 1600 1 efi (800K) 1601 2012560 2 freebsd [active] (983M) 2014161 28226607 - free - (13G) => 0 2012560 da0s2 BSD (983M) 0 16 - free - (8.0K) 16 2012544 1 freebsd-ufs (983M) => 1 30240767 diskid/DISK-00241D8CE51BB011B9A694C1 MBR (14G) 1 1600 1 efi (800K) 1601 2012560 2 freebsd [active] (983M) 2014161 28226607 - free - (13G) => 0 2012560 diskid/DISK-00241D8CE51BB011B9A694C1s2 BSD (983M) 0 16 - free - (8.0K) 16 2012544 1 freebsd-ufs (983M)
How do I recover from here?
<—— end of original question
I have made some progress and managed to import and mount one root dataset – iocage. This is a boot on zfs system but I cannot find the dataset that contains the root filesystem so I cannot get at /var/log to see if anything is there:
mkdir /tmp/zroot # /tmp is a writable file system zpool -f zroot # force the zpool import zfs set mountpoint=/tmp/zroot zroot # mount the imported pool in a writable fs zfs mount -a # find and mount all the datasets ll /tmp/zroot total 12 drwxr-xr-x 9 root wheel 11 Feb 27 13:09 iocage/
Fortunately, all of the absolutely critical stuff is in
/zroot/iocage
as the host simply acts as a platform for the jail. However the absence of the root dataset is bothersome to me.A
zpool status
showed zroot with no errors.I next transferred the iocage dataset to another system using
zfs send
.zfs snapshot -r zroot/iocage@vh6iocsend1 zfs send -R zroot/iocage@vh6iocsend1 | ssh 192.168.216.45 zfs receive zroot/iocagev6
This took a while but it has completed successfully.
Now I need to get the problem host started. This host was restarted yesterday at noon without a problem. I do not recall running
freebsd-update fetch
, but even if I had there was nothing for fetch to deliver as the system was already at 12.1p2I still need help getting the host to boot.
<———-
Additional notes:
I was able to mount the entire
zpool
using thealtroot
option ofzpool import
:
- Boot into the live cd shell.
- Import the zfs pool(s) but do not allow
import
to auto mount any datasets:zpool import -o altroot=/tmp/altroot -N -a
.- Mount the root
/
dataset first:zfs mount zroot/ROOT/default
.- Now mount the remaining datasets:
zfs mount -a
.The entire
zroot
pool’s file system is now accessible at/tmp/altroot
.I have used this with
zfs send
to move the contents of/var
to another host. Actually, I sent the entire pool.However, the original system still will not boot.
Answer
As a last resort I pulled the four hdds from the host that failed to boot and placed them in an identically configured server. That server booted from those hdds. Evidently a hardware problem.
There is still an anomaly on the new host: if bay 1 is occupied then the system will only boot if at least one of bay 0 or bay 2 is likewise occupied. The configuration 0-,1A,2-,3B will not boot. I do not have a clue as to why this is so.
Attribution
Source : Link , Question Author : James B. Byrne , Answer Author : James B. Byrne