Gentoo Wiki ArchivesGentoo Wiki

HOWTO_Install_Gentoo_on_an_LVM2_root_partition

This article is part of the HOWTO series.
Installation Kernel & Hardware Networks Portage Software System X Server Gaming Non-x86 Emulators Misc

This is where the community will put in a "HOWTO Install Gentoo on an LVM2 root partition" It will cover making an initrd and configuring boot scripts.

Contents

Needed Packages

If you intend to use RAID, you will also need sys-fs/mdadm

Preamble

Ok, first ask the question, what do you need to mount a lvm partition?

  1. Kernel with dm-mod loaded
  2. Kernel with initrd support and CONFIG_BLK_DEV_RAM_SIZE > 4096
  3. /dev/mapper/control Device hook for dm-mod
  4. vgscan and vgchange
  5. mount and the kernel drivers for the filesystem (ext2 must be built in for initrd to work)

Install

For our example we will use a single ide harddrive 40GB in size. The root filesystem will be reiserfs and the boot partition will be ext2.

We will use one volume group (system) and three logical volumes, usr, home and slash.

First, boot the gentoo liveCD and prepare the hard drive.

Prepare three partitions the first with id of 83 of at least 10MB (I usually make it 512MB to make it nice and round). The second will be your swap partion, mine is 512mb, use id 82. The third partition use the rest of the free space and use id 8e.

Now load the device mapper kernel module.

modprobe dm-mod

Now activate lvm (ignore any error messages about volume group not found at this point)

vgscan
vgchange -ay

Now make a physical volume on /dev/hda3

pvcreate /dev/hda3

Create a volume group on /dev/hda3, we will call it "system". The default physical extent size of 4MB is too little, so we set it to a more realistic 32MB here. We do this because our maximum VG size is determined by the size of our extents. Maximum VG size = Extent Size * 64K. E.g. 32MB * 64K = 2097152MB = 2048GB = 2TB. Think ahead; this number should be big enough to cover the largest size your array could possibly become...changing this later is a real pain (think backup / restore). On the other hand, we don't want to set this too large either because this is also the granularity with which we can allocate space to our Logical Volumes (i.e. if we used the maximum of 16GB, we'd only be able to allocate space in chunks of 16GB).

Note: the above restriction on extent size _only applies to LVM1_, there's no problem with 4MB extents and VG of any size in LVM2. Small extents in a very big VG will decrease the lvm tools performance, though. Source: vgcreate manpage.

vgcreate -s 32M system /dev/hda3

Note: only for LVM1, changing the extent size will possibly cause errors when extracting the portage snapshot.

Now make the logical volumes, we will leave some room to grow later.

lvcreate -L10G -nslash system
lvcreate -L10G -nusr system
lvcreate -L10G -nhome system

We have 9GB left, I'll show you how to use it later.
Let's format all out partitions now.

mkfs.ext2 /dev/hda1
mkswap /dev/hda2
mkfs.reiserfs /dev/system/slash
mkfs.reiserfs /dev/system/usr
mkfs.reiserfs /dev/system/home

Now let's mount these puppies!

swapon /dev/hda2
mkdir /gentoo
mount -t reiserfs /dev/system/slash /gentoo
mkdir /gentoo/boot
mount -t ext2 /dev/hda1 /gentoo/boot
mkdir /gentoo/usr
mount -t reiserfs /dev/system/usr /gentoo/usr
mkdir /gentoo/home
mount -t reiserfs /dev/system/home /gentoo/home

Download your gentoo stage tarball to /gentoo and install it.

cd /gentoo; tar -xvpjf ./stage{insertnamehere}.tar.bz2

You can alternatively extract the stage tarballs directly from the internet, without downloading it first.

cd /gentoo; wget http://www.yourfavoritemirror.org/gentoo/stageXYZ.tar.bz2 -O - | tar xvpjf -

Here we mount the proc filesystem for availability within the chroot env. This is need for many 2.6 kernel apps. We also bind the physical devices found by the live cd and provide it to the chroot env via the mount --bind method.

mkdir /gentoo/{dev,proc}
mount --bind /dev /gentoo/dev
mount -t proc /proc /gentoo/proc

Now chroot into your new install and continue as you would normally.

chroot /gentoo /bin/bash
env-update && source /etc/profile

Now continue as in Gentoo Handbook remember to enable the device mapper kernel module (dm-mod.ko).

/etc/fstab

You need to set this correctly.

File: /etc/fstab
/dev/mapper/system-slash   /           reiserfs        noatime,notail         0 0
/dev/mapper/system-usr     /usr        reiserfs        noatime                0 0
/dev/mapper/system-home    /home       reiserfs        noatime                0 0
/dev/hda2                  none        swap            sw                     0 0
/dev/hda1                  /boot       ext2            noauto,noatime         0 0

Make your initrd

You have three options here - each one gives a varying level of automation and control over the kernel and initrd configurations. You need an initrd, because you need userspace software to assemble the LVM/RAID volumes. Choose one of these sections based on your needs and preference.

The Easiest Way

In light of the gentoo way, Kernel/genkernel can be used to generate an initrd with lvm2 support built in. I built this with genkernel 3.1.0d and the following packages:

LVM2.2.00.25.tgz
device-mapper.1.00.17.tgz
[try running: "emerge lvm2" first, it may get lvm2 and device-mapper going right away. If portage complains about packages being blocked, try updating udev first]
Note: Before running genkernel you need to make sure that the symlink in /usr/src are pointing to the correct version of kernel you are installing for.
cd /usr/src
rm linux
ln -sf linux-{version} linux

Now, all you need to do to generate the initrd with lvm2 support is the following:

genkernel --dmraid --lvm --splash=emergence initrd

The splash argument is for the splash theme when booting up, there are several options to choose from, but I chose emergence since it looks cool.

One last thing you need to do is tell the initrd that you want to use lvm2 and additionally raid if you use the lvm2 on top of a raid system. In grub, append dolvm2 to the kernel line in /boot/grub/grub.conf and be sure to set real_root=/dev/VOLUME_GROUP/ROOT_VOLUME where the VOLUME_GROUP AND ROOT_VOLUME are the devices for your system. In my case, I used /dev/vg/root.

Here is an example /boot/grub/grub.conf

File: /boot/grub/grub.conf
#
# Sample boot menu configuration file
#

# Boot automatically after 10 secs.
timeout 10
# By default, boot the first entry.
default 0
# Fallback to the second entry.
fallback 1
splashimage=(hd0,0)/grub/splash.xpm.gz

# Gentoo Linux 2.6.9-r9 lvm
title  Gentoo Linux (2.6.9-r9 lvm2)
root   (hd0,0)
kernel /kernel-2.6.9-gentoo-r9 udev dolvm2 root=/dev/ram0 real_root=/dev/vg/root init=/linuxrc <other options>
initrd /initrd-2.6.9-gentoo-r9
# If you are using devfs instead of udev you should replace the kernel-line above with this line:
#kernel /kernel-2.6.9-gentoo-r9 devfs dolvm2 root=/dev/ram0 real_root=/dev/vg/root init=/linuxrc....

grub-install into mirror

If your boot partition is also on software raid, you will run into problems running grub-install. The problem is that grub-install does a 'df' to determine the real physical device corresponding with /boot. To get around this, you should edit /etc/mtab and change md0 to the 1st half of the mirror, run grub-install on the first half of the mirror, change it to the second half, run grub-install on the second half, then put /etc/mtab back. Assuming that /dev/md0 consists of /dev/sda1 and /dev/sdb1:

cp -p /etc/mtab /etc/mtab.orig
sed s:/dev/md0:/dev/sda1: /etc/mtab.orig > /etc/mtab
grub-install /dev/sda1
sed s:/dev/md0:/dev/sdb1: /etc/mtab.orig > /etc/mtab
grub-install /dev/sdb1
mv /etc/mtab.orig /etc/mtab

possible issue with above example

I am unfamiliar with grub, but dolvm2 did not work in lilo as of sys-kernel/genkernel-3.4.9_pre6. However, dolvm did work.

/etc/lilo.conf excerpt:

image=/boot/kernel-genkernel-x86-2.6.23-gentoo-r3
       label=2.6.23-r3
       root=/dev/ram0
       initrd=/boot/initramfs-genkernel-x86-2.6.23-gentoo-r3
       vga=798
       read-only
       append="dolvm real_root=/dev/sys/root init=/linuxrc ramdisk_size=8192"

[sireasoning]

Adding RAID with genkernel

If your lvm root device exists on a software md raid, you must tell your initrd about this so that the devices can be initialized before vgscan is run. To do this pass the line lvmraid=[md devices] either in the kernel line of your grub.conf or on the append line of your lilo.conf [fati]

Problems with this way

As of 2007-01-31, the genkernel way does not work out of the box (LVM on top of RAID), since it could not mount the root filesystem. It tried the following kernel arguments.

/boot/kernel-genkernel-x86-2.6.19-gentoo-r4 udev dodmraid dolvm2 root=/dev/ram0 init=/linuxrc lvmraid=/dev/md1 real_root=/dev/vg/root init=/linuxrc

The devices are found, but the userspace utilities (mdstart from busybox?) do not work. This seems to be a problem with newer versions of busybox. Try the 2nd easiest way if this one fails.

The Second Easiest Way (lvm2create_initrd)

This method uses the LVM2 script that automatically creates the initrd you need for LVM2. This way, you have more control over your kernel than genkernel gives you, and you don't have to make the initrd from scratch. The lvm2create_initrd script is available in the LVM2 source tree.

First, download the initrd script.

Next, as root, mount your /boot partition, if you need to:

mount /boot

Now, configure and compile your kernel however you want, keeping the following in mind:

Linux Kernel Configuration: Enabling DM and Snapshots
Device Drivers  --->
  [*] Multiple devices driver support (RAID and LVM) --->
    <*> Device mapper support
    <*> Snapshot target (EXPERIMENTAL)

Make sure to select the appropriate RAID type for your configuration. If in doubt, select all of these:

Linux Kernel Configuration: Enabling RAID
Device Drivers  --->
  [*] Multiple devices driver support (RAID and LVM) --->
    <*> RAID support
    <*> Linear (append) mode
    <*> RAID-0 (striping) mode 
    <*> RAID-1 (mirroring) mode
    <*> RAID-10 (mirrored striping) mode (EXPERIMENTAL)
    <*> RAID-4/RAID-5/RAID-6 mode
Linux Kernel Configuration: RAM disk size
Device Drivers  --->
  Block Devices  --->
    <*> RAM disk Support
    (8192) Default RAM disk size (kbytes)
Linux Kernel Configuration: Enabling loopback
Device Drivers  --->
  Block Devices  --->
    <*> Loopback device support 
 
Linux Kernel Configuration: Enabling initrd
   General setup --->
    [*] Initial RAM filesystem and RAM disk (initramfs/initrd) support
 

Compile and install your kernel and modules. Go to the directory where the lvm2create_initrd script was downloaded. You have to specify the -M gentoo option to tell it which architecture to use. This will use the kernel pointed to by /usr/src/linux.

sh ./lvm2create_initrd -M gentoo

Specific kernel

If you have an specific kernel you would like to use, tell lvm2create_initrd about it by adding it to the command line, like this:

 sh ./lvm2create_initrd -M gentoo 2.6.9-gentoo-r4

lvm.conf

If you have an lvm.conf file already on your system, tell lvm2create_initrd about it by using the -c option, like this:

 sh ./lvm2create_initrd -M gentoo -c /etc/lvm/lvm.conf 2.6.9-gentoo-r4

The script should create the initrd, and move it to /boot, and give you some handy hints for setting up GRUB or LILO to boot your kernel and initrd. If all worked, make the necessary modifications to your bootloader.

Adding RAID

There are two options you need to pass to lvm2create_initrd to add software RAID support - that is, if your LVM2 root is on a RAID device. First, double check that the kernel support for the RAID level you are using is compiled into the kernel (this was done in an earlier step). Then, add the -r option to specify the name of the raid device you wish to have started, and the -R option to specify the path to the mdadm.conf file.

lvm2create_initrd -M gentoo -c /etc/lvm/lvm.conf -r /dev/md0 -R /etc/mdadm.conf 2.6.9-gentoo-r4

Create /initrd

Create the "initrd" directory under /:

mkdir -p /initrd

or you will get the error

Changing roots
\t*FAILED*
pivot_root: File not found (or something like this)

grub.conf

If you are using GRUB, you'll need something like this in grub.conf. Obviously, you'll need to replace your kernel, initrd and root volumes with the appropriate values for your system: VG is the name of the volume group containing the real root filesystem, and LV is the name of the logical volume within that volume group.

title Gentoo with LVM
    kernel /boot/vmlinuz-lvm2-2.6.9-gentoo-r4 root=/dev/ram0 lvm2root=/dev/VG/LV <other parameters>
    initrd /boot/initrd-lvm2-2.6.9-gentoo-r4.gz

Lilo users will want something like this. Again, modify the paths to the kernel, initrd, and lvm root volume as necessary:

image=/boot/vmlinuz-lvm2-2.6.9-gentoo-r4
     label="Gentoo_With_LVM"
     initrd=/boot/initrd-lvm2-2.6.9-gentoo-r4.gz
     append="root=/dev/ram0 lvm2root=/dev/VG/LV <other parameters>"

Ensure that your ramdisk size is sufficient to hold the initrd. Check your kernel configuration or override using the "ramdisk_size" (in kb) kernel parameter. The uncompressed size of your ramdisk can be found by passing the '-v' (verbose) option to lvm2create_initrd.

Then, continue on with this HOWTO, - you need to do the stuff in the "System Side" and "Cleanup" sections.

Potential Problems

bash fails on libncurses.so.5

On AMD64 platform I experienced this problem:

/bin/bash: error while loading shared libraries: libncurses.so.5: cannot open shared object file: No such file or directory

This appears to be a minor bug in Red Hat's script not liking our "lib64" directory. In case you haven't learned this yet, the initrd file is just a gzipped file containing an ext2 file system that the kernel loads into a ramfs. To solve this problem, I had to uncompress and mount the initrd file and make fix the "lib" directory.

cd /boot
gzip -d initrd-lvm2-2.6.21-gentoo-r1.gz
mkdir tmp
mount initrd-lvm2-2.6.21-gentoo-r1 tmp -o loop
cd tmp
mv lib/* lib64
rmdir lib
ln -s lib64 lib
cd ..
umount tmp
rmdir tmp
gzip initrd-lvm2-2.6.21-gentoo-r1

The Cowboy Way

If you don't like automation, and want to make your own initrd and kernel from scratch, this is the section for you!

Warning: Most people don't use the Cowboy Way, and so it isn't well tested. If you have time, please test these instructions and remove this warning! There is room for improvement, too; perhaps you can use ash from busybox instead of bash. Also, recent lvm builds seem to always be static; copy it to avoid a rebuild here
Note: Your kernel needs to load the device mapper kernel module and run userland tools - compiled with static option - before it can mount "/" !!!
CC="gcc -static" emerge lvm2
emerge mkinitrd
mkdir /initrd
Warning: If you don't create directory initrd in the root directory then you will get this error:
pivotroot: pivot_root(/sysroot,/sysroot/initrd) failed: 2
umount /initrd/proc failed: 2

My kernel is 2.6.9 use your own number for that...

mkinitrd --preload=dm-mod --with=dm-mod /boot/initrd-2.6.9.gz 2.6.9

Now mount the initrd.

mkdir /mnt/loop
gunzip /boot/initrd-2.6.9.gz
mount -o loop /boot/initrd-2.6.9 /mnt/loop

Now /mnt/loop has a miniture root filesystem. You need to copy a few utilites to it. Namley bash test mknod mkdir sed insmod

Static Executables

Build static versions of bash, busybox, lvm2 and mdadm into a different ROOT. Busybox and mdadm can build static versions using the static USE flag; the other two seem to work with CC="gcc -static", as follows:

mkdir /root/static
ROOT=/root/static USE="static -readline make-symlinks" emerge busybox mdadm
ROOT=/root/static CC="gcc -static" USE="-readline -gpm" emerge lvm2 bash
If you're using selinux, you may have some problems with busybox since libsepol is not included, and it is needed by the default busybox configuration. If so, you can work around this problem with this command:
ROOT=/root/static USE="static -readline make-symlinks" LIBRARIES="-lsepol" ACCEPT_KEYWORDS="~amd64" emerge --ask -v busybox
Now, take the static versions of the executables and merge them into /mnt/loop/bin:
cd /root/static
cp -a bin/bash bin/busybox bin/test /mnt/loop/bin
cp -a bin/mknod /bin/mkdir bin/sed /mnt/loop/bin
cp -a sbin/lvm sbin/mdadm /mnt/loop/sbin
cp -a /sbin/insmod.static /mnt/loop/sbin/insmod

(* mknod and mkdir are included in nash, which mkinitrd includes in the initrd. I guess in this case you should merge the below described devmap_mknod.sh into linuxrc?)

Warning: Your busybox executable may be too large for initrd. You'll have to increase the size (make a new one) or reconfigure busybox. Busybox only needs a few executables (it should be a short list; any takers?)


Tip: If necessary, you can use this to resize your initrd partition:
dd if=/boot/initrd-2.6.9 of=/dev/ram0
# above, /dev/ram0 represents any available/flushed ram device
# then resize, depending on the ramdisk size you have (to 6000 x 1K), then fsck, just in case
# your kernel default ramdisk size must support your image.
# otherwise you can specify it; and now ramdisk=XX in grub.conf is deprecated?
resize2fs -f /dev/ram0 6000
dd if=/dev/ram0 of=/boot/initrd-2.6.9 count=6000 ibs=1024
sync
e2fsck /boot/initrd-2.6.9

Make the script executable (chmod +x devmap_mknod.sh)

Add the script devmap_mknod.sh to /mnt/loop/bin

File: /mnt/loop/bin/devmap_mknod.sh
#! /bin/bash

# Startup script to create the device-mapper control device
# on non-devfs systems.
# Non-zero exit status indicates failure.
 
# These must correspond to the definitions in device-mapper.h and dm.h
DM_DIR="mapper"
DM_NAME="device-mapper"

set -e

DIR="/dev/$DM_DIR"
CONTROL="$DIR/control"

# Check for devfs, procfs
if test -e /dev/.devfsd ; then
        echo "devfs detected: devmap_mknod.sh script not required."
        exit
fi

if test ! -e /proc/devices ; then
        echo "procfs not found: please create $CONTROL manually."
        exit 1
fi

# Get major, minor, and mknod
MAJOR=$(sed -n 's/^ *\([0-9]\+\) \+misc$/\1/p' /proc/devices)
MINOR=$(sed -n "s/^ *\([0-9]\+\) \+$DM_NAME\$/\1/p" /proc/misc)

if test -z "$MAJOR" -o -z "$MINOR" ; then
        echo "$DM_NAME kernel module not loaded: can't create $CONTROL."
        exit 1
fi

mkdir -p -m 755 $DIR
test -e $CONTROL && rm -f $CONTROL

echo "Creating $CONTROL character device with major:$MAJOR minor:$MINOR."
mknod -m 600 $CONTROL c $MAJOR $MINOR

Edit /mnt/loop/linuxrc before the "pivot_root /sysroot /sysroot/initrd" line, the finished script should look like this.

File: /mnt/loop/linuxrc
#!/bin/bash

echo "Loading dm-mod.ko module"
insmod /lib/dm-mod.ko
echo Mounting /proc filesystem
mount -t proc /proc /proc
echo Creating block devices
mkdevices /dev
echo Creating root device
mkrootdev /dev/root
echo 0x0100 > /proc/sys/kernel/real-root-dev
echo "Making Device Mapper Nodes."
devmap_mknod.sh
echo "Scanning for volume groups."
lvm vgscan
lvm vgchange -ay
echo "Mounting root filesystem"
mount -o noatime --rw -t reiserfs /dev/mapper/system-slash /sysroot
pivot_root /sysroot /sysroot/initrd
umount /initrd/proc

Your initrd is now completed! Run this to finish up.

umount /mnt/loop
gzip /boot/initrd-2.6.9

System Side

You still have a little bit to do so gentoo can properly boot your system. Add these lines after "ebegin "Remounting root filesystem read-only (if necessary)"" in /etc/init.d/checkroot

vgscan --mknodes --ignorelockingfailure
vgchange -ay --ignorelockingfailure

Clean Up

Run these commands for good mesure.

emerge lvm2
emerge coreutils
vgchange -a n

Now install grub, and boot your new kernel with /boot/initrd-2.6.9 as your initrd, cross you fingers and hope for the best!

Links

Retrieved from "http://www.gentoo-wiki.info/HOWTO_Install_Gentoo_on_an_LVM2_root_partition"

Last modified: Mon, 25 Aug 2008 06:43:00 +1000 Hits: 93,663

Created by NickStallman.net, Luxury Homes Australia
Real estate agents should list their apartments, townhouses and units in Australia.