bhyve, zvol, zfs, and importing zpool from host

I’ve been having problem with importing zpool of a vm which was using a zvol as its disk image.

I don’t know the exact detail but the only way I can import it so far is by dd-ing the image to a file and then attaching it as md device as such:

# mdconfig -a -S 512 -f /mnt/vm/somevm/disk0.img
md0

And then the pool can be normally imported from the md0 device.

Of course, had the vm been created as image file on zfs filesystem, the dd step can be skipped and just md the file right away.

Conclusion?

Just use plain image file for vm disk. Either that or use ufs for the vm filesystem instead (ew).

Other benefit of using image file includes less noisy gpart output.

zpool hourly status check

I’m setting up cron job for a storage server using ZFS. There’s zpool status -x but it returns “all pools are healthy” (or “no pools available”) on no error and prints errors to stdout (instead of stderr), rendering it annoying for cron job.

#!/bin/sh

set -e
set -u

zstatus="`zpool status -x 2>&1`"
case "${zstatus}" in
  "all pools are healthy"|"no pools available")
    return 0
  ;;
  *)
    printf "%sn" "${zstatus}" >&2
    return 1
  ;;
esac

Put it in a file (e.g. /root/bin/zpool-status-cron), make it executable, and add it to crontab.

This ZFS thingy

Apparently a RAIDZ2 with 9 disks is computationally expensive: my system got as low as 34% CPU Idle when doing one disk rebuild (and one other disk offline). The CPU isn’t exactly fast (Athlon 64 X2 4600+) but I didn’t expect the usage to be this high. I’ll see how it goes after rebuild finished. Good thing I cancelled my plan of using an Atom for my other file server.

last pid: 62412;  load averages:  1.34,  1.28,  1.25                                                                   up 0+02:39:21  01:32:32
54 processes:  1 running, 52 sleeping, 1 zombie
CPU:  0.0% user,  0.0% nice, 63.2% system,  1.9% interrupt, 34.9% idle
Mem: 46M Active, 32M Inact, 4654M Wired, 104K Cache, 85M Buf, 1160M Free
Swap: 400M Total, 400M Free

ZFS on Desktop – Which OS?

I’ve been thinking about this lately, especially after seeing this:

[root@einhart ~]# pkg_version -vIL=>
binutils-2.21 < needs updating (index has 2.21.1)
bitstream-vera-1.10_4 < needs updating (index has 1.10_5)
chromium-12.0.742.112 < needs updating (index has 12.0.742.124)
droid-fonts-ttf-20100214_1 < needs updating (index has 20110324)
exiv2-0.21,1 < needs updating (index has 0.21.1,1)
gawk-3.1.8 < needs updating (index has 4.0.0)
gcc-4.6.2.20110701 < needs updating (index has 4.6.2.20110708)
gnutls-2.12.7_1 < needs updating (index has 2.12.7_2)
libidn-1.19 < needs updating (index has 1.22)
openjdk6-b22_6 < needs updating (index has b23)
p5-Net-DBus-0.33.6 < needs updating (index has 1.0.0)
p5-libwww-5.837 Fetching ‘/usr/ports/www/chromium’

To build Chromium, you should have around 1 GB of memory
and a fair amount of free diskspace (~ 1.5GB).

===> Vulnerability check disabled, database not found
===> License BSD LGPL21 MPL accepted by the user
===> Found saved configuration for chromium-12.0.742.112
=> chromium-courgette-redacted-12.0.742.124.tar.xz doesn’t seem to exist in /usr/ports/distfiles/.
=> Attempting to fetch http://download.goodking.org/downloads/chromium-courgette-redacted-12.0.742.124.tar.xz
chromium-courgette-redacted-12.0.742.124.tar.x 0% of 114 MB 35 kBps^C

And did you notice that “you should have around 1 GB of memory and a fair amount of free diskspace”? Yeah, it does use that much memory to update a web browser. Something that takes few seconds on Windows and Fedora and few minutes on Ubuntu takes almost hours to do on FreeBSD. Downloading source notwithstanding, compiling it also takes long, long time.

If anything, this is why I hate FreeBSD. And you will mention use package except that there’s no H.264 video support and possible chance of conflicting package difference between my system and the buildbot. As much as I want to use binary packages, they come in unwanted forms:

* GNOME 2 pulls Samba4 for God knows why reason
* Some pacakges don’t have optimization enabled
* Mplayer doesn’t have VDPAU
* Some packages must be installed manually (Java, Opera)
* Portupgrade takes minutes to compute what to do (something which is done in seconds in Ubuntu)

As much as I can tolerate this shit, the very fact that VirtualBox under FreeBSD takes much more CPU certainly doesn’t help.

So, I guess I’ll try yet another OS. There are several options so here are overview of current choices.

FreeBSD
——-
Pros:

* Extremely simple
* Up to date packages
* Highly customizable
* Stable ZFS

Cons:

* Customization comes at cost: compile time
* No good virtualization option
* A package management that is comparable with slowpoke
* No flash (yes there’s linux flashplugin ports but it’s a joke)
* Awesomely slow video playback even optimized on Firefox

PC-BSD
——
Pros:

* Based on FreeBSD, has some of familiarity
* Binary packages that’s not too slow
* Stable ZFS

Cons:

* Also no good virtualization
* Epic size of packages (eg. [a hundred megabytes for Firefox](http://www.pbidir.com/bt/pbi/49/firefox))

OpenIndiana
———–
Pros:

* Stable, greatest ZFS
* Zones
* Good VirtualBox support
* Flash support

Cons:

* Complex
* Doesn’t survive trivial hardware change without Live CD/USB
* ACL
* Outdated packages
* Non-existent packages
* Manual compiling of `mplayer` is a must since the defaults on various repositories are crap
* Still beta
* Doesn’t recognize FreeBSD’s GPT (at least on `oi_148`)
* Ugly freetype2 (no support for subpixel hinting apart of manual compile)

Ubuntu Linux
————
Pros:

* Latest, greatest software
* Good software support
* Good virtualization

Cons:

* Unstable ZFS
* Linux

Note that I only include Ubuntu in Linux category since it’s the only OS having both working nvidia driver and precompiled ZFS module, saving me lots of headaches.