windy/my-vault
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
Files
40f3fe3eb84a39504a6fa40c192e8f60572bada3
my-vault/00_Inbox/Clippings/2024/05/Fedora Root on ZFS — OpenZFS documentation.md
T

14 KiB
Raw Blame History

page-title, url, date
page-title url date
Fedora Root on ZFS — OpenZFS documentation https://openzfs.github.io/openzfs-docs/Getting%20Started/Fedora/Root%20on%20ZFS.html 2024-05-29 14:22:28

find /dev/disk/by-id/

Fedora Root on ZFS

Notes

  • As an alternative to the below method of installing Fedora Linux on a ZFS root filesystem, you can use the unofficial script fedora-on-zfs, which is more automated and can generate a Fedora Linux installation that is closer to an official Fedora Linux configuration. The fedora-on-zfs script is different from the below method in that it uses one of Fedora’s official kickstarts (fedora-disk-minimal.ks, fedora-disk-workstation.ks, fedora-disk-kde.ks, etc.) to guide the installation, but with a few overrides to add the ZFS functionality. Bug reports should be submitted to Greg’s fedora-on-zfs GitHub repo.

ZFSBootMenu

ZFSBootMenu is an alternative bootloader free of such limitations and has support for boot environments. Do not follow instructions on this page if you plan to use ZBM, as the layouts are not compatible. Refer to their site for installation details.

Customization

Unless stated otherwise, it is not recommended to customize system configuration before reboot.

Only use well-tested pool features

You should only use well-tested pool features. Avoid using new features if data integrity is paramount. See, for example, this comment.

UEFI support only

Only UEFI is supported by this guide.

Preparation

  1. Disable Secure Boot. ZFS modules can not be loaded if Secure Boot is enabled.

  2. Because the kernel of latest Live CD might be incompatible with ZFS, we will use Alpine Linux Extended, which ships with ZFS by default.

    Download latest extended variant of Alpine Linux live image, verify checksum and boot from it.

    gpg --auto-key-retrieve --keyserver hkps://keyserver.ubuntu.com --verify alpine-extended-*.asc

    dd if=input-file of=output-file bs=1M

  3. Login as root user. There is no password.

  4. Configure Internet

    setup-interfaces -r # You must use "-r" option to start networking services properly # example: network interface: wlan0 WiFi name: ip address: dhcp manual netconfig: n

  5. If you are using wireless network and it is not shown, see Alpine Linux wiki for further details. wpa_supplicant can be installed with apk add wpa_supplicant without internet connection.

  6. Configure SSH server

    setup-sshd # example: ssh server: openssh allow root: "prohibit-password" or "yes" ssh key: "none" or ""

  7. Set root password or /root/.ssh/authorized_keys.

  8. Connect from another computer

  9. Configure NTP client for time synchronization

  10. Set up apk-repo. A list of available mirrors is shown. Press space bar to continue

  11. Throughout this guide, we use predictable disk names generated by udev

apk update
apk add eudev
setup-devd udev
  1. Target disk
List available disks with

If virtio is used as disk bus, power off the VM and set serial numbers for disk. For QEMU, use `-drive format=raw,file=disk2.img,serial=AaBb`. For libvirt, edit domain XML. See [this page](https://bugzilla.redhat.com/show_bug.cgi?id=1245013) for examples.

Declare disk array

DISK\='/dev/disk/by-id/ata-FOO /dev/disk/by-id/nvme-BAR'

For single disk installation, use

DISK\='/dev/disk/by-id/disk1'

  1. Set a mount point

  2. Set partition size:

Set swap size in GB, set to 1 if you don’t want swap to take up too much space

Set how much space should be left at the end of the disk, minimum 1GB

  1. Install ZFS support from live media:

  2. Install partition tool

apk add parted e2fsprogs cryptsetup util-linux

System Installation

  1. Partition the disks.

    Note: you must clear all existing partition tables and data structures from target disks.

    For flash-based storage, this can be done by the blkdiscard command below:

    partition_disk () { local disk="${1}" blkdiscard -f "${disk}" || true

    parted --script --align=optimal "${disk}" -- \ mklabel gpt \ mkpart EFI 1MiB 4GiB \ mkpart rpool 4GiB -$((SWAPSIZE + RESERVE))GiB \ mkpart swap -$((SWAPSIZE + RESERVE))GiB -"${RESERVE}"GiB \ set 1 esp on \

    partprobe "${disk}" }

    for i in ${DISK}; do partition_disk "${i}" done

  2. Setup temporary encrypted swap for this installation only. This is useful if the available memory is small:

    for i in ${DISK}; do cryptsetup open --type plain --key-file /dev/random "${i}"-part3 "${i##*/}"-part3 mkswap /dev/mapper/"${i##*/}"-part3 swapon /dev/mapper/"${i##*/}"-part3 done

  3. Load ZFS kernel module

  4. Create root pool

    • Unencrypted:

      # shellcheck disable=SC2046 zpool create \ -o ashift=12 \ -o autotrim=on \ -R "${MNT}" \ -O acltype=posixacl \ -O canmount=off \ -O dnodesize=auto \ -O normalization=formD \ -O relatime=on \ -O xattr=sa \ -O mountpoint=none \ rpool \ mirror \ $(for i in ${DISK}; do printf '%s ' "${i}-part2"; done)

  5. Create root system container:

    # dracut demands system root dataset to have non-legacy mountpoint zfs create -o canmount=noauto -o mountpoint=/ rpool/root

    Create system datasets, manage mountpoints with mountpoint=legacy

    zfs create -o mountpoint=legacy rpool/home zfs mount rpool/root mount -o X-mount.mkdir -t zfs rpool/home "${MNT}"/home

  6. Format and mount ESP. Only one of them is used as /boot, you need to set up mirroring afterwards

    for i in ${DISK}; do mkfs.vfat -n EFI "${i}"-part1 done

    for i in ${DISK}; do mount -t vfat -o fmask=0077,dmask=0077,iocharset=iso8859-1,X-mount.mkdir "${i}"-part1 "${MNT}"/boot break done

System Configuration

  1. Download and extract minimal Fedora root filesystem:

    apk add curl curl --fail-early --fail -L \ https://dl.fedoraproject.org/pub/fedora/linux/releases/39/Container/x86_64/images/Fedora-Container-Base-39-1.5.x86_64.tar.xz \ -o rootfs.tar.gz curl --fail-early --fail -L \ https://dl.fedoraproject.org/pub/fedora/linux/releases/39/Container/x86_64/images/Fedora-Container-39-1.5-x86_64-CHECKSUM \ -o checksum

    # BusyBox sha256sum treats all lines in the checksum file # as checksums and requires two spaces " " # between filename and checksum

    grep 'Container-Base' checksum \ | grep '^SHA256' \ | sed -E 's|.*= ([a-z0-9]*)$|\1 rootfs.tar.gz|' > ./sha256checksum

    sha256sum -c ./sha256checksum

    rootfs_tar=$(tar t -af rootfs.tar.gz | grep layer.tar) rootfs_tar_dir=$(dirname "${rootfs_tar}") tar x -af rootfs.tar.gz "${rootfs_tar}" ln -s "${MNT}" "${MNT}"/"${rootfs_tar_dir}" tar x -C "${MNT}" -af "${rootfs_tar}" unlink "${MNT}"/"${rootfs_tar_dir}"

  2. Enable community repo

    sed -i '/edge/d' /etc/apk/repositories sed -i -E 's/#(.*)community/\1community/' /etc/apk/repositories

  3. Generate fstab:

    apk add arch-install-scripts genfstab -t PARTUUID "${MNT}" \ | grep -v swap \ | sed "s|vfat.*rw|vfat rw,x-systemd.idle-timeout=1min,x-systemd.automount,noauto,nofail|" \

    "${MNT}"/etc/fstab

  4. Chroot

    cp /etc/resolv.conf "${MNT}"/etc/resolv.conf for i in /dev /proc /sys; do mkdir -p "${MNT}"/"${i}"; mount --rbind "${i}" "${MNT}"/"${i}"; done chroot "${MNT}" /usr/bin/env DISK="${DISK}" bash

  5. Unset all shell aliases, which can interfere with installation:

  6. Install base packages

    dnf -y install @core kernel kernel-devel

  7. Install ZFS packages

    dnf -y install \ https://zfsonlinux.org/fedora/zfs-release-2-4"$(rpm --eval "%{dist}"||true)".noarch.rpm

    dnf -y install zfs zfs-dracut

  8. Check whether ZFS modules are successfully built

    tail -n10 /var/lib/dkms/zfs/**/build/make.log

    # ERROR: modpost: GPL-incompatible module zfs.ko uses GPL-only symbol 'bio_start_io_acct' # ERROR: modpost: GPL-incompatible module zfs.ko uses GPL-only symbol 'bio_end_io_acct_remapped' # make[4]: [scripts/Makefile.modpost:138: /var/lib/dkms/zfs/2.1.9/build/module/Module.symvers] Error 1 # make[3]: [Makefile:1977: modpost] Error 2 # make[3]: Leaving directory '/usr/src/kernels/6.2.9-100.fc36.x86_64' # make[2]: [Makefile:55: modules-Linux] Error 2 # make[2]: Leaving directory '/var/lib/dkms/zfs/2.1.9/build/module' # make[1]: [Makefile:933: all-recursive] Error 1 # make[1]: Leaving directory '/var/lib/dkms/zfs/2.1.9/build' # make: [Makefile:794: all] Error 2

    If the build failed, you need to install an Long Term Support kernel and its headers, then rebuild ZFS module

    # this is a third-party repo! # you have been warned.

    # select a kernel from # https://copr.fedorainfracloud.org/coprs/kwizart/

    dnf copr enable -y kwizart/kernel-longterm-VERSION dnf install -y kernel-longterm kernel-longterm-devel dnf remove -y kernel-core

    ZFS modules will be built as part of the kernel installation. Check build log again with tail command.

  9. Add zfs modules to dracut

    echo 'add_dracutmodules+=" zfs "' >> /etc/dracut.conf.d/zfs.conf echo 'force_drivers+=" zfs "' >> /etc/dracut.conf.d/zfs.conf

  10. Add other drivers to dracut:

if grep mpt3sas /proc/modules; then
  echo 'force\_drivers+=" mpt3sas "'  \>> /etc/dracut.conf.d/zfs.conf
fi
if grep virtio\_blk /proc/modules; then
  echo 'filesystems+=" virtio\_blk "' \>> /etc/dracut.conf.d/fs.conf
fi
  1. Build initrd
find \-D exec /lib/modules \-maxdepth 1 \\
-mindepth 1 \-type d \\
-exec sh \-vxc \\
'if test -e "$1"/modules.dep;
   then kernel=$(basename "$1");
   dracut --verbose --force --kver "${kernel}";
 fi' sh {} \\;

  1. For SELinux, relabel filesystem on reboot:

  2. Enable internet time synchronisation:

systemctl enable systemd-timesyncd
  1. Generate host id
zgenhostid \-f \-o /etc/hostid
  1. Install locale package, example for English locale:
dnf install \-y glibc-minimal-langpack glibc-langpack-en
  1. Set locale, keymap, timezone, hostname
rm \-f /etc/localtime
rm \-f /etc/hostname
systemd-firstboot \\
--force \\
--locale\=en\_US.UTF-8 \\
--timezone\=Etc/UTC \\
--hostname\=testhost \\
--keymap\=us || true
  1. Set root passwd
printf 'root:yourpassword' | chpasswd

Bootloader

  1. Install rEFInd boot loader:

    # from http://www.rodsbooks.com/refind/getting.html # use Binary Zip File option curl -L http://sourceforge.net/projects/refind/files/0.14.0.2/refind-bin-0.14.0.2.zip/download --output refind.zip

    dnf install -y unzip unzip refind.zip mkdir -p /boot/EFI/BOOT find ./refind-bin-0.14.0.2/ -name 'refind_x64.efi' -print0 \ | xargs -0I{} mv {} /boot/EFI/BOOT/BOOTX64.EFI rm -rf refind.zip refind-bin-0.14.0.2

  2. Add boot entry:

    tee -a /boot/refind-linux.conf <<EOF "Fedora" "root=ZFS=rpool/root" EOF

  3. Exit chroot

  4. Unmount filesystems and create initial system snapshot You can later create a boot environment from this snapshot. See Root on ZFS maintenance page.

    umount -Rl "${MNT}" zfs snapshot -r rpool@initial-installation

  5. Export all pools

  6. Reboot

Post installaion

  1. Install package groups

    dnf group list --hidden -v # query package groups dnf group install gnome-desktop

  2. Add new user, configure swap.

  3. Mount other EFI system partitions then set up a service for syncing their contents.

Reference in New Issue View Git Blame Copy Permalink