Gentoo Wiki ArchivesGentoo Wiki

HOWTO_build_a_LiveCD_from_scratch


This article is part of the HOWTO series.
Installation Kernel & Hardware Networks Portage Software System X Server Gaming Non-x86 Emulators Misc

Contents

Introduction

This mini-HowTo will show you how to create your own LiveCD. It will also allow you to have the following advantages over the catalyst way:

Requirements

You need to have enough disc space on your system. Depending on which packages you include in your LiveCD, this will vary (more packages increases the size). I'd say in average a 4GB free disc space ought to be enough.

You must be familiar with installing Gentoo using a stage2 tarball because that's what we're going to use in this mini-HowTo. If you're in doubt, please consult the Gentoo Handbook.

I also recommend quite some knowledge about the Gentoo workings and Linux in general. Especially the new build script things can get nasty if you get it wrong.

It is possible to do a stage1 install, but since this takes much more time than a stage2 install, this is not recommended. Most people will do something like emerge -e world anyway.

You'll also need app-cdr/cdrtools and sys-fs/squashfs-tools installed to build properly.

Note: Be sure that squashfs-tools and in-kernel driver have equal versions, otherwise it might refuse to mount

Setting up the build environment

Our build environment is just a directory that we will 'chroot' and we install Gentoo in (much like a installation from a normal LiveCD). Its contents will make the LiveCD later on. I will use the name 'source' for it, and will create it under a directory in my home called livecd.


emerge squashfs-tools cdrtools
cd ~
mkdir -p ~/livecd/source

This entire guide expects you to make the livecd directory in /root (or whatever ~ is set to), the build process will fail if you use another directory. I update the build script from time to time, so it is best to follow this and be able to create an up-to-date livecd.

(Comment - Hanni) This is not strictly true I have a number of simultaneous livecd builds scattered across nummerous directories. In order to use any directory you must keep all your scripts within whatever name you give your equivalent livecd directory and change the LIVECD variable in the build script to ${PWD}. i.e. change

LIVECD=~/livecd

to

LIVECD=${PWD}

pwd is an internal bash variable representing the working directory. (End Comment)

To get started download a stage2 tarball from one of the Gentoo mirrors to your home directory. I'm using an i686 stage2 in this mini-HowTo since it's most likely to run on most PCs these days. Once downloaded, untar it to source. Then create the necessary newroot directory which will be used during initial boot.

wget <<YOUR FAVORITE MIRROR>>/releases/x86/current/stages/stage2-x86-2006.0.tar.bz2
wget <<YOUR FAVORITE MIRROR>>/snapshots/portage-latest.tar.bz2
cd livecd/source
tar jxvpf ~/stage2-x86-2006.0.tar.bz2
mkdir newroot

(Comment) Because glibc 2.4 supports only nptl it's impossible to use CHOST i386, which is the case with stage2-x86-*. Instead use stage2-i686-2006.1.tar.bz2 or stage2-i586-2006.1.tar.bz2. (End Comment)

Now, download the latest portage snapshot to your home directory and untar it to your new build directory.


cd usr/
tar jxvf ~/portage-latest.tar.bz2


If you need to download packages during the installation, then setup whatever files in livecd/source/etc required to connect to the Internet. For example resolv.conf.

Your source directory is now ready to start installing Gentoo.

Installing Gentoo

Setup necessary mount points before you chroot to your build directory and start installation. On my system I keep distfiles under the default /usr/portage directory. I'm going to use it instead of downloading distfiles again during the installation.

cd ~/livecd/source
mkdir -p proc dev sys
mount --bind /proc proc
mount --bind /dev dev
mount --bind /sys sys
mkdir -p usr/portage/distfiles
mount --bind /usr/portage/distfiles usr/portage/distfiles 
cp /etc/resolv.conf etc/resolv.conf

(Comment - DKS) Instead of cross-mounting /usr/portage/distfiles as stated above, mount /usr/portage:

mkdir -p usr/portage
mount --bind /usr/portage usr/portage

(End Comment)

(Comment - carramba) When it's impossible to bind /usr/portage or /usr/portage/distfiles, You can export it through nfs (or other smb,ftpfs,sshfs whatever) readwrite and mount it in your ~/livecd/source/usr/portage/ (~/livecd/source/usr/portage/distfiles/)

echo "/usr/portage/distfiles 127.0.0.1(rw,no_root_squash,nohide,async,no_subtree_check,crossmnt)" >> /etc/exports
exportfs -r
mount -t nfs 127.0.0.1:/usr/portage/distfiles ~/livecd/source/usr/portage/distfiles

(End Comment)

Now chroot to source and start the installation. A few points about modifying /etc/make.conf

  1. Be conscious with USE variables. The more you use, the bigger your environment will get. It might even be so big that it won't fit on a CD! I use kde only, and don't care about docs and java, so I added the follwoing: '-gnome -gtk -java -doc'
  2. Make sure you add 'livecd' to your USE variables. This, along with a 'cdroot' parameter that we'll pass to the kernel at boot time, will make init scripts aware of the fact that we're booting off a cdrom, so it doesn't do inappropriate things like checking the root filesystem!
  3. You also might want to consider adding 'minimal' and 'userlocales' to USE (and don't forget to edit /etc/locales.build )

(Comment - DKS) If you employ the USE variable 'minimal', you will need to add an exemption for the perl package:

echo "dev-lang/perl -minimal" >> ~/livecd/source/etc/portage/package.use

The command above is for use *outside* the chroot. Inside the chroot the target is /etc/portage/package.use. (End Comment)

Here are the installation steps roughly

  1. cd livecd/source
  2. chroot . /bin/bash --login
  3. env-update
  4. source /etc/profile
    (Comment - BringerOfDarkness) Instead of doing a global search for the entire system updates as the next step leads you to do, first modify /etc/make.conf, adding/modifying the line ... GENTOO_MIRRORS=http://something ... to someplace close to you from the list at http://www.gentoo.org/main/en/mirrors.xml (End Comment)
  5. emerge -e system
    (Comment - Darkproger) It's better to make /etc/lo?ale.gen list before, or after emerging glibc it will generate >=374 locales (End Comment)
  6. emerge -e world # If part of the systems were updated, like GCC, re-emerge the whole system and check http://www.gentoo.org/doc/en/gcc-upgrading.xml
  7. passwd # It is a good idea to set a password for root before continuing!
  8. ln -sf /usr/share/zoneinfo/<path to time zone file> /etc/localtime
    (Comment - Octothorn) As of Feb 2007 this is not the way it is done. This should now be set in /etc/conf.d/clock - replace the FACTORY setting with the timezone of your choice.
  9. echo "sys-fs/dmraid ~x86" >> /etc/portage/package.keywords # you might need this line to use dmraid as requested later
    (Comment - Goyira) At present (9/10/06), adding dmraid to package keywords won't be enough. It is necessary to add it to package.unmask with echo "=sys-fs/dmraid-1.0.0_rc8-r1" >> /etc/portage/package.unmask [the vesion number may change] (End Comment)
  10. emerge memtest86+ localepurge genkernel gentoolkit dmraid livecd-tools #<required packages> (NOTE: sys-fs/fuse has been built into the kernel as of 2.6.16)
    (Comment - Hanni) livecd-tools appears to be a necessary package when using the newest baselayout (10/Nov/06) without it services won't start, additionally it adds a number of other functions which are useful for livecd's (End Comment)
  11. emerge #<required packages, as you like, e.g. logger hotplug udev gentoolkit>
    (Comment - grunthus) I have just discovered on forums.gentoo.org that hotplug's functionality has been removed - it does nothing. Confirmed by looking at the hotplug script in init.d (End Comment)
    (Comment - Arek75) Hotplug's secondary functionality (boot-time hardware detection) was replaced by a seperate coldplug script around this time. If you haven't already done so, doing "emerge coldplug && rc-update add coldplug default" as root should give you the same functionality hotplug once had. (End Comment)
    ((Comment - Octothorn) Coldplug functionality has been taken over by udev. If your attempt to emerge coldplug is being blocked by udev, this is fine. You already have all thefunctionality of coldplug in udev.) (End Comment)
  12. rc-update add <the services required, e.g. hotplug, net.eth0, sysklogd> default <or boot, as fits>
  13. emerge <Recommended optional packages sys-fs/dmraid and slocate, these are used in further scripts. You might need to add them to /etc/portage/package.keywords>
  14. emerge mingetty # <Required for autologin>
  15. emerge # <optional packages, as you like, e.g. kde, mc>

Before we merge the kernel and boot loader, let's modify important files in /etc. First, /etc/fstab

File: /etc/fstab
/dev/loop0              /               squashfs        defaults             0 0
none                    /proc           proc            defaults                0 0
none                    /dev/shm        tmpfs           defaults                0 0

Don't forget an empty line at the end or the start or the LiveCD will complain!

Notice that we're mounting root over a loop device with filesystem type squashfs. More on that later. Now, modify other /etc files as you see fit (e.g. hostname, rc.conf, conf.d/*, and so on)

It is for example a good idea to change the keyboard layout one time : edit /etc/conf.d/keymaps. For french keyboard including € character support, change to the following layout :

KEYMAP="fr-latin9"
SET_WINDOWKEYS="yes"


If you want network support, edit /etc/conf.d/net. This example sets dhcp configuration for eth0 :

config_eth0=( "dhcp" )

Kernel and initrd time

Then it's time to install the kernel sources and configure them.

emerge gentoo-sources #add patches after this
#if you use the vanilla sources you must patch them with the squashfs-patch from http://squashfs.sourceforge.net
cd /usr/src/linux
make allmodconfig   # If you want to have ALL kernel options compiled as modules (to be able to recognize most hardware), then use this 


Now merge your desired kernel. I'm using gentoo-sources(2.6). We're going to use squashfs as our root filesystem. So, be sure to patch your kernel with it if it's not already included (standard gentoo-sources have it these days).

Note: If you don't have SquashFS support listed in your kernel, look here or use Google, I googled "squashfs kernel patch"

When configuring your kernel (see below, using genkernel --menuconfig) , make sure the following is compiled in:

Linux Kernel Configuration: kernel configuration
File systems  --->
  <*> Ext3 journalling file system support
  <*> Reiserfs support
  <*> JFS filesystem support
  <*> XFS filesystem support
  Miscellaneous filesystems  --->
    <*> SquashFS 2.0 - Squashed file system support
  CD-ROM/DVD Filesystems  --->
    <*> ISO 9660 CDROM file system support
  Pseudo filesystems  --->
    [*] Virtual memory file system support (former shm fs)
Device Drivers  --->
  Block devices  --->
    <*> Loopback device support
    <*> RAM disk support
    (16)  Default number of RAM disks
    (9000) Default RAM disk size (kbytes)
    [*]   Initial RAM disk (initrd) support
  ATA/ATAPI/MFM/RLL support  --->
    <*>     Include IDE/ATAPI CDROM support
Note: ext2 filesystem support (which we use for our initrd image), (it is a good idea if it is an rescue cd of some sort to include support for most systems, may come in handy)


(Comment by moz667 (thank you Edna for help me in translation)) If your planning boot through usb you´ll need to compile the kernel with the USB Mass Storage support. It works when I use my usb stick with this configuration.

Linux Kernel Configuration: kernel configuration
Device Drivers --->
  USB support --->
    <*> USB Mass Storage support
      [ ] USB Mass Storage verbose debug
      [*] Datafab Compact Flash Reader support (EXPERIMENTAL)
      [*] Freecom USB/ATAPI Bridge support
      [*] ISD-200 USB/ATA Bridge support
      [*] Microtech/ZiO! CompactFlash/SmartMedia support
      [ ] USBAT/USBAT02-based storage support (EXPERIMENTAL)
      [*] SanDisk SDDR-09 (and other SmartMedia) support (EXPERIMENTAL)
      [*] SanDisk SDDR-55 SmartMedia support (EXPERIMENTAL)
      [*] Lexar Jumpshot Compact Flash Reader (EXPERIMENTAL)
      [ ] Olympus MAUSB-10/Fuji DPC-R1 support (EXPERIMENTAL)


IEEE 1394 ethernet support may disturb your system : sometimes an ethernet interface is recognized first, that is to say before "Classic" ethernet interface. If you don't need IEEE ethernet support, don't activate it :

Linux Kernel Configuration: kernel configuration2
Device Drivers  --->
  IEEE 1394 (FireWire) support  --->
   < >   Ethernet over 1394

Note: If you want to let make mkxf86config.sh an usable xorg.conf at startup, you have to build usbmouse ps2mouse etc as modules (and not compiled in!) since the script is searching for them.

Optionally, you can add any convenient option you use. For example I use Magic SysRq Keys, which allows rebooting instantly when no file system is mounted (or you might get corrupted filesystems).

Linux Kernel Configuration: kernel configuration3
Kernel hacking ->
 Magic SysRq Key [*]


I wanted something like the gentoo 2005.0 for it is much more flexible, looks 1000x better, and boots more systems. After some research I found out that it was genkernel that makes such an initrd. (believe me, using the initrd from the 2005.0 cd is not possible).

The old can still be used, but is not recommended. Also this howto is from now on based on genkernel.

(Comment by Kimchi_sg) If you have already configured AND compiled the kernel, tell genkernel to only create initrd by running:

genkernel initrd

And then skip the rest of this section.

Otherwise if you have not compiled kernel, do the following: (End of comment)

genkernel all --no-bootsplash --no-clean --menuconfig

Then don't forget to copy/rename the file to reflect the grub configuration (remember that as stated before, iso9660 doesn't handle symbolic links) :

For example, if you are working with a 2.6.15-gentoo-r1 type :

cp /boot/initramfs-genkernel-x86-2.6.15-gentoo-r1 /boot/initrd
cp /boot/kernel-genkernel-x86-2.6.15-gentoo-r1 /boot/vmlinuz

Or leave the menuconfig bit if you have a working .config. I found that the genkernel .config was good, at least after I adjusted the file system support.

We now have a suitable kernel and ramdisk !

(Comment by NyZeta) if you are having problems building a 32bit (i386) kernel on a 64bit (x86_64) system do:

cp .config backupconfig
make mrproper
cp backupconfig .config
ARCH="i386" make menuconfig

Check if the config has the right settings. (check CPU and ARCH). Then do:

ARCH="i386" make

If you are using the genkernel scripts try:

genkernel all --no-bootsplash --no-clean --arch-override=x86 --utils-arch=x86 --makeopts='ARCH="i386"'

(End of comment)

(Comment by PAulHewlett) The above coment by NyZeta did not work for me. The correct solution is to emerge setarch in the 64 bit environment and when chrooting prefix with linux32 viz:

linux32 chroot . /bin/bash --login

(End of comment)

Cleaning some files

It is a good idea to clean the kernel tree, keeping only ".config" file :

cd /usr/src/linux
make clean

Edit /etc/locale.nopurge configuration file : remove the line 'NEEDSCONFIGFIRST' and just leave the locales you need. Mine looks like that (for both english and french support) :

File: /etc/locale.nopurge
 MANDELETE
 SHOWFREEDSPACE
 VERBOSE
 en
 en_GB
 en_US.UTF-8
 fr
 fr_FR
 fr_FR@euro
 fr_FR.UTF-8

Then run localepurge to save space and after that reindex whatis :

localepurge
makewhatis -u

Finally, delete ".keep" files that are necessary to restore empty directories from stage2 and portage TAR archives :

find / -type f -xdev -name ".keep" -print -exec rm {} \;

Using "locate"

Locate is a convenient and fast tool to know if some file is present on your system. If you want to use it, you have to build the file database :

localhost / #  cat /proc/mounts > /etc/mtab
localhost / #  updatedb

Then you'll be able to search file using locate :

localhost / # locate whatisRCU.txt
/usr/src/linux-2.6.17-gentoo-r8/Documentation/RCU/whatisRCU.txt

Installing and configuring the CD bootloader

The final step is merging and configuring the boot loader. We're going to use GRUB since it has support for booting a cdrom.

emerge grub 

And here is the /boot/grub/menu.lst, but more on this after the script.

File: /boot/grub/menu.lst
default 0
timeout 30
splashimage=/boot/grub/splash.xpm.gz

title=LiveCD
        kernel /boot/kernel-genkernel root=/dev/ram0 real_root=/dev/loop0 vga=788 looptype=squashfs loop=/livecd.squashfs udev nodevfs cdroot dodmraid 
        initrd /boot/initramfs-genkernel

title=LiveCD NO-FB
        kernel /boot/kernel-genkernel root=/dev/ram0 real_root=/dev/loop0 looptype=squashfs loop=/livecd.squashfs udev nodevfs cdroot dodmraid
        initrd /boot/initramfs-genkernel-x86-2.6.15-gentoo-r1

title=Memtest86+
        kernel /boot/memtest86plus/memtest.bin
Note: Don't forget to add the splash image OR remove the line from menu.lst

DO NOT forget to delete the symlink /boot/grub/menu.lst and then move grub.conf to menu.lst, because iso9660 can not handle symlinks, otherwise grub will just give a dumb commandline! (this one cost me a few hours!).

THIS COST ME A LOT OF TIME: The requirement by grub is that menu.lst should be accessible. Because of iso9660, you should create the file menu.lst and have the link for grub.conf. i.e. menu.lst should be a file (and grub.conf can point to it if you are really used to editing grub.conf).

You can discard the 'vga' kernel parameter if you want, but the rest of the paramters are necessary. Our next step is to create the 'initrd' image.

Edit above from Maurice Volaski Note that the script had real_root=/dev/loop/0, but there should be no second /; it's just /dev/loop0. In the above grub.conf the kernel should be /boot/vmlinuz and the initrd should be /boot/initrd, as to previous installation steps! End edit above

Final cleanings

Inside the chrooted environment, don't forget to remove old kernel modules. For example :

localhost modules # cd /lib/modules
localhost modules # du -sh *
29M     2.6.16-gentoo-r12
29M     2.6.16-gentoo-r13
29M     2.6.17-gentoo-r4
28M     2.6.17-gentoo-r7
28M     2.6.17-gentoo-r8

Here we can remove 4 obsolete modules directories (delete all directories except the kernel(s) the livecd shall be booting) :

localhost modules # rm -rf  2.6.16-gentoo-r12 2.6.16-gentoo-r13 2.6.17-gentoo-r4  2.6.17-gentoo-r7

Alternative (?) : we can also use emerge to remove old source trees :

equery l | grep "sources"
emerge -C =sys-kernel/gentoo-sources-2.6.17-r8 =sys-kernel/gentoo-sources-2.6.18-r3 =sys-kernel/gentoo-sources-2.6.18-r6 =sys-kernel/gentoo-sources-2.6.19-r5


Then we can leave the chrooted environment :

exit
cd ~/livecd/source
umount sys proc dev usr/portage/distfiles
env-update
source /etc/profile

Then we make a copy of our root sourcetree ; this way we will be able to make more agressive deletions on the copy without harming the original which can be useful for updates

Comment by Daniel Kühl If you receive a lot of warnings about existing directories when creating the squashfs, do not use the rsync command line, instead use the first, the second, fourth and the last line. End of comment

#rm -rf ~/livecd/target
#mkdir ~/livecd/target
rsync --delete-after --delete-excluded --archive --hard-links --quiet ~/livecd/source/boot ~/livecd/target/ 
#cp -a ~/livecd/source/boot ~/livecd/target/
#If you plan to use syslinux uncomment the following line
#cp -a ~/livecd/source/isolnux ~/livecd/target/
mkdir -p ~/livecd/target/files/source

Copy the source tree using cp OR rsync :

# cp -p -R -P -d ~/livecd/source/ ~/livecd/target/files ## CP IS OBSOLETE since RSYNC IS MUCH FASTER !!!
#rsync is faster since it doesn't copy many useless files
rsync --delete-after --delete-excluded --archive --hard-links --quiet --exclude "var/tmp/*" --exclude "var/cache/*" --exclude "*.h" --exclude "usr/portage" --exclude "etc/portage" --exclude "usr/share/doc" --exclude "var/db" --exclude "usr/src" ~/livecd/source/ ~/livecd/target/files/source/


Then we clean some files :

cd ~/livecd/target/files/source/
rm -rf var/tmp/*
#rm -rf var/run/*  #can't be deleted because some program use it
rm -rf var/lock/*
rm -rf var/cache/*
rm -rf var/db
rm -rf tmp/*
rm -f etc/mtab
touch etc/mtab
rm -rf var/log
mkdir var/log
mkdir var/lib/
mkdir var/lib/dhcpc
#                             Kernel tree and portage tree might be useless on your livecd
rm -rf usr/portage
rm -rf etc/portage
rm -rf usr/share/doc
rm -rf usr/src/
rm root/.bash_history

More agressive cleanings

The goal here is to remove all software that ain't mandatory. For example portage, automake and gcc will be useless on the Live CD.

Inside the chroot do :

#Note that the first lines include only one ">" and the others include two of them.
equery f gcc | grep -v "* Contents of " | grep "^/usr/lib/gcc/i486-pc-linux-gnu/4.1.2/include/" > ~/USELESSFILELIST
equery f  portage | grep -v "* Contents of " >> ~/USELESSFILELIST
equery f automake | grep -v "* Contents of " >> ~/USELESSFILELIST
equery f rpm2targz | grep -v "* Contents of " >> ~/USELESSFILELIST
equery f gentoolkit | grep -v "* Contents of " >> ~/USELESSFILELIST
equery f texinfo | grep -v "* Contents of " >> ~/USELESSFILELIST
equery f genkernel | grep -v "* Contents of " >> ~/USELESSFILELIST
equery f autoconf | grep -v "* Contents of " >> ~/USELESSFILELIST
equery f automake | grep -v "* Contents of " >> ~/USELESSFILELIST
equery f automake-wrapper | grep -v "* Contents of " >> ~/USELESSFILELIST
equery f bison | grep -v "* Contents of " >> ~/USELESSFILELIST
equery f flex | grep -v "* Contents of " >> ~/USELESSFILELIST
equery f gcc-config | grep -v "* Contents of " >> ~/USELESSFILELIST
equery f make | grep -v "* Contents of " >> ~/USELESSFILELIST
equery f m4 | grep -v "* Contents of " >> ~/USELESSFILELIST
equery f patch | grep -v "* Contents of " >> ~/USELESSFILELIST


This will provide a complete list of the desired packages.


Outside the chroot and after building the target execute a script like this :


#!/bin/bash
cd /root/livecd
for i in `cat source/root/USELESSFILELIST`
do
if [ -f target/files/source/${i} ]
then
rm target/files/source/${i} ;
fi
if [ -L target/files/source/${i} ]
then
rm target/files/source/${i}
fi
if [ -d target/files/source/${i} ]
then
rmdir target/files/source/${i}
fi
done

This will remove files, symbolic links, and empty directories (as stated, rmdir will only remove empty directories)

This saved me 80Mb before compression just removing gcc, portage, automake, rpm2targz, gentoolkit and texinfo.

devsk: Are the static libraries needed on the livecd? I saved about 85MB by doing

find ~/livecd/target/files/source/usr/lib -name "*.a" -print -exec rm {} \;

Better yet, you can put "*.a" in the rsync --exclude command line.

Building the LiveCD

Building the LiveCD involves the following steps

1 Using squashfs tools (emerge -av squashfs-tools if you need to), convert the source directory to a squashed image

2 Create the required empty livecd file. --Clarification added by Maurice Volaski

3 And finally, make the iso image


cd ~/livecd/target/files
rm -f ~/livecd/target/livecd.squashfs
time mksquashfs source/ ~/livecd/target/livecd.squashfs
touch ~/livecd/target/livecd       # !!! This file must be on livecd root, because the init script in initramfs uses this file to identify if cd is mounted or not !!!
# we don't delete the "files" directory no more : it is excluded by using mkisofs and can save a lot of time for furthers rsync operaions
# rm -rf ~/livecd/target/files
cd ~/livecd/
time mkisofs -R -b boot/grub/stage2_eltorito -no-emul-boot -boot-load-size 4 -boot-info-table -iso-level 4 -hide-rr-moved -c boot.catalog -o ~/livecd/livecd.iso -x files ~/livecd/target/


Now you can :

1 - test the file ~/livecd/livecd.iso using qemu ; see section Testing the LiveCD without wasting a blank CD

2 - burn the file ~/livecd/livecd.iso

FINISHED :D

Updating the CD

You can chroot again to the CD tree and make some updates :

cd ~/livecd/source
mount -o bind /proc proc
mount -o bind /sys sys
mount -o bind /dev dev
mount -o bind /usr/portage/distfiles usr/portage/distfiles
chroot . /bin/bash --login
#Chroot to 32 bit environnment if you plan to use the livecd on different arch
linux32 chroot . /bin/bash --login  
env-update
source /etc/profile
# LIVECD UPDATE STARTS HERE
# MAKE HERE THE DESIRED UPDATES, for example :
emerge --sync
emerge world
freshclam
# LIVECD UPDATE ENDS HERE
exit
umount sys proc dev usr/portage/distfiles
env-update
source /etc/profile

Now you can go back to the step Cleaning some files and follow the step to create a new ISO image.

Tips

Mounting /dev is important to get things like X to work from a chroot environment.

Another thing to consider is the size of your build source. As you can see, there're things that take a lot of space that are strictly not needed on a LiveCD. Two major candidates are /usr/src and /usr/portage. If you don't want to include them on your LiveCD don't delete them, you can utilize an option in mksquashfs to exclude any directory from the squashed image. This will keep your build source directory intact.

If booting the CD fails at some point during linuxrc script execution, you can always change your grub line to init=/bin/sh instead of linuxrc. This will give you a nice little shell inside the initrd image at boot time. From there you can type the commands in the linuxrc script manually and try to find out with one is failing and why.

I hope I didn't miss anything, and hope you find this howto useful. Your corrections/feedback are always welcome.

It is not necessary to bootstrap the system; the cd wil be able to boot from non-bootstrapped or bootsrapped systems.

Generate Xorg.conf at startup

emerge -av mkxf86config hwdata-gentoo hwsetup
rc-update add mkxf86config default

Note: mkxf86config realy depends on hwsetup, so you need to edit /etc/init.d/mkxf86config and add /usr/sbin/hwsetup before /usr/sbin/mkxf86config.sh . It is also recomended to use newest versions of mkxf86config and hwsetup (currently lastest is mkxf86config 0.9.8 and hwsetup 1.2, but you need to write ebuild manualy). If you want mkxf86config to detect hardware properly - you need to edit /etc/init.d/mkxf86config or it will not detect your hardware. Also it is useful to look through /usr/sbin/mkxf86config.sh... If you are building LiveCD for x86 compatible PCs - you can uncomment ddcxinfo-knoppix and emerge this tool (ddcxinfo-knoppix - it will automaticlly probe monitor for information, but it may have some troubles with old and non-x86 hardware

Comment-alchark: hwsetup will actually be called by autoconfig script from the livecd-tools package, and it will also call mkxf86config as required.

emerge -av livecd-tools
rc-update add autoconfig default

--end comment

Auto Mounting partitions spiced up

I have made an automounter script that looks up which partitions are available and then tries to mount them. But it is not a stupid loop.

  1. find all available partitions
  2. skip unmountables such as extended and grep error's
  3. use swap as swap instead of mounting
  4. mount ntfs with captive-ntfs
  5. if none of the above, just try to mount
Update: Use ntfs-3g to avoid licensing issues and have much faster/better ntfs support.

I do this by greping fdisk -l, and using a second loop in my loop (because of the * that fdisk puts in front of the bootable partitions).

Getting captive-ntfs to run from the live-cd is a bit of a problem most of the time. It requires the screen program, and the fuse kernel module (in the before-build script it is (re)compiled so we do not forget it). Also it needs the ntfs.sys etc, you can deliver them preinstalled on the live-cd but I do not know if this is legal or not. But it is best to edit it a bit to autodownload them when they are needed and only start this script when the user requests it when you do not want any legal problems.

I have put my script online here, do not edit it using notepad or wordpad because they put a ^M at the end of each line which will break your livecd completely!

I could use some help perfecting the 2>/dev/null. It still displays some output sometimes.

Auto Login

If you want root to be automatically logged in, there's nothing to do, because usually, the inittab is being fixed during the boot in order to autologin root on a LiveCD.

But: Most LiveCDs have the purpose to show Linux novices the principles of a Linux system, so it might be undesired to autlogin as root, which is a large security threat. So if you want to autologin as non-privileged user, here's how to do it.

First emerge mingetty. Then edit /etc/inittab on the cd (be sure to make a backup), find:

File: /etc/inittab
c1:12345:respawn:/sbin/agetty 38400 tty1 linux

replace with

c1:12345:respawn:/sbin/mingetty --autologin joe --noclear tty1

for every tty you want autologon for a user named "joe".

Then you need to manipulate /sbin/rc in order to keep it from rebuilding the inittab. This has become somewhat more difficult to do in openrc and baselayout2 but here is how to do it;

Extract ex /usr/portage/distfiles/openrc-0.2.2.tar.bz2 to a test location, then edit;

File: openrc-0.2.2/sh/init.sh.Linux.in
if [ -f /sbin/livecd-functions.sh -a -n "${CDBOOT}" ]; then
       ebegin "Updating inittab"
       livecd_fix_inittab
       eend $?
       telinit q &>/dev/null
fi

replace with

if [ -f /sbin/livecd-functions.sh -a -n "${CDBOOT}" ]; then
       ebegin "Updating inittab"
       /bin/true # livecd_fix_inittab
       eend $?
       /bin/true # telinit q &>/dev/null
fi

If you want to display and extra message next to "OpenRC 0.2.2 is starting up Gentoo-Linux(i686)" now is the time to edit openrc-0.2.2/src/rc/rc.c;

File: openrc-0.2.2/src/rc/rc.c

after

printf(BRANDING " (%s)", uts.machine);

insert

printf("\n   %sYour own message",
                      ecolor(ECOLOR_GOOD));

After this we need to get this back to the tar.bz2 and get it verified

tar cjf openrc-0.2.5.tar.bz2 openrc-0.2.5 
mv openrc-0.2.5.tar.bz2 /usr/portage/distfiles
ebuild /usr/portage/sys-apps/openrc/openrc-0.2.5.ebuild digest
emerge openrc

(this makes the ~ display after automatic login, instead of /root/)

Note: if you would like to disable autologin for all users, simply make the changes described above to /sbin/rc, without making the changes to /etc/inittab. When the LiveCD finishes booting, you'll then see the normal login prompt. Make sure that you have previously set password for root with passwd, or you'll be unable to login.

Optimizing the booting process

Note: No optimization is required if app-misc/livecd-tools is installed (in the chroot environment). That package provides scripts to detect whether a live CD is running and to disable unneeded instructions like the ones described below.

I have removed these optimizations since they were obsolete for some time and the new baselayout2 and openrc made them completly obsolete.

Another neat trick: edit /etc/rc:

Change rc_parallel="no" to rc_parallel="yes"

This will start all the services at the same moment, not one after another. Some people say this has no effect yet, but it looks cool and maybe there will be a time it has a purpose. But when using autoconfig from the livecd tools it will break your boot sequence.

"Defragmenting" the LiveCD to make it faster

By default, the files inside the squashfs filesystem are stored continuously, but unordered: this means that the cdrom drive will potentially have to seek them at very distant disk offsets, leading to higher cold start read times -- this is especially apparent at boot time if the LiveCD contains desktop environments such as GNOME or KDE, or applications such as Firefox or GIMP that use large number of files.

Fortunately, mksquashfs comes with a -sort option that makes it possible to change files' order on the filesystem. The easiest way to build the needed sort file is to include openlog [1] in your LiveCD kernel image, then run the resulting LiveCD. openlog makes the kernel able to timely log all the successfully opened files; it also includes a script, openlog.py, that makes it easy to reuse this information to create the sort file. openlog comes fully documented: see the README or README.html file provided with it.

Note: I have moved the openlog page to my own host since I have been unable to contact the author, also I have added a new patch file for newer kernel versions. The orriginal site: [2]

The quick and dirty howto:

Boot the livecd with logfiles=1 and get dmesg from it to your build environment and extract openlog.py as well.

File: dmesg
cat dmesg | ./openlog.py --root=/<path-to-livecd-sources> -o /<path-to-store-order-results>/load.order.new

And change your build script to let the ordering take effect in the next build

File: build
mksquashfs source ${TARGET}/rescuecd.squashfs -noappend -sort /<path-to-store-order-results>/load.order.new

Checking/editing the Initramfs

If you want to edit or check the initramfs generated, you can do something like this.

initramfs source

/usr/share/genkernel/pkg/${ARCH}/cpio

or

s=$(gzip -l $cur | ...) ; dd if=$cur bs=1 skip=$s of=$next ; cur=$next ; next=$next+1
 -- Thanx to robbat2

Initrd:

gzip -cd initrd > unzipped
mount -o loop -t ext2 unzipped /mnt/cdrom

USB Stick drive

(evildick 2005-08-09)

build-usb.sh

#!/bin/bash
LIVECD=~/livecd
TARGET=${LIVECD}/target
TARGET_SOURCE=${TARGET}/files/source/
SOURCE=${LIVECD}/source

#USB settings
USBDEV=/dev/sda
MNTDIR=/mnt/usbstick

#USB
echo "Warning\!  You are about to reformat your usb device on ${USBDEV}\!"
echo "The following partitions will be overwritten:"
sfdisk -l ${USBDEV}
echo "Are you certain that this is what you intend?"
echo "Press enter to continue or CTRL-C to quit."
read user_reply
echo "Please wait, preparing usb device . . ."
# if you want to completely wipe the USB drive, issue the following command :
# dd if=/dev/zero of=${USBDEV}
sfdisk -q ${USBDEV} << EOF
0,,L,*
;
;
;
EOF
mke2fs -m0 ${USBDEV}1
mkdir ${MNTDIR}
mount ${USBDEV}1 ${MNTDIR}

echo "Copying files to USB device . . ."
cp -p -R -P -d ${TARGET}/boot ${TARGET}/livecd.squashfs ${MNTDIR}

echo "Modifying ${MNTDEV}/boot/grub/menu.lst . . ."
sed 's/(cd)/(hd0,0)/g' ${MNTDIR}/boot/grub/menu.lst > ${MNTDIR}/boot/grub/menu.lst-tmp && mv -f ${MNTDIR}/boot/grub/menu.lst-tmp ${MNTDIR}/boot/grub/menu.lst
# This may need to be changed to replace the cdroot with something better, like usbroot?
#sed 's/cdroot//g' ${MNTDIR}/boot/grub/menu.lst > ${MNTDIR}/boot/grub/menu.lst-tmp && mv ${MNTDIR}/boot/grub/menu.lst-tmp ${MNTDIR}/boot/grub/menu.lst
echo "Installing Grub . . ."
grub-install --root-directory=${MNTDIR} ${USBDEV} --recheck

echo "Unmounting ${MNTDIR} . . ."
umount ${MNTDIR}

echo "Your bootable USB drive should be complete."

  • For the most part this works. Problems may be experienced due to bios limitations. Problems can range but usually it will just lock up shortly around initialization of the kernel (during or on the initramfs). This so far only occurs with a Supermicro P4SCi (it is likely that it will also occur with a P4SC8 - similar boards, but I have not tested it)
  • The script was designed to be used in conjuction with the above livecd scripts (this script being used last or replacing the mkiso command). The script above will format your usb device with an ext2 filesystem and write the squash image along with the grub loader files to it. After copying the grub boot loader menu it will alter it to work with the usb stick as a bootable device. Be sure to change the device settings in beginning of the script to reflect the mount directory you wish to use, as well as the usb stick device. You may also need to compile your kernel with USB devices (UHCI and OHCI) not as modules.

PXE Booting

  • Using the above instructions to build a squash image you can boot into the image via PXE (grub works but does not support as many network cards as pxelinux). NFS and tftp are required on the server end for this method. The following is required to be enabled in the kernel on the Livecd or client end.
Note: : Don't forget to add to your kernel built-in (no modules!)
- NFS filesystem client support as ;
- NFS ROOT support ;
- your network card(s) support ;


Copy the contents of your target directory into a your tftp/nfs directory. Use the following parameters in your PXELinux default configuration file:

label gentoo
   menu label Gentoo LivePXE 
   kernel /livecd/boot/kernel-genkernel-x86-2.6.14-gentoo-r2
   append initrd=/livecd/boot/initramfs-genkernel-x86-2.6.14-gentoo-r2 looptype=squashfs root=/dev/ram0 real_root=/dev/nfs nfsroot=172.16.4.7:/diskless/livecd ip=dhcp udev nodevfs dodmraid cdroot

Other methods to Copy files

  • I have heard that tar and cpio are supposed to copy files faster than cp. I have not confirmed this yet, but here is a little bit of code to try out if you like.
  • Comment out the line in your build script:
cp -p -R -P -d ${SOURCE}/ ${TARGET}/files

And add this above or below it:

cd ${SOURCE}
tar cf - * | ( cd ${TARGET_SOURCE}; tar xfp -)

(Comment - evermind) why don't you use tar cpf - * before the pipe? Isn't that neccesary to preserve the permissions? (End Comment)

(Comment - letum) The p option is needless. Yes, it preserves permissions, but only when you extract the archive. Tar saves permissions by default - p is necessary only during the extraction process. (End Comment)

I have not fully tested this, so be careful. You can also add; '--exclude=usr/src --exclude=var/tmp* --exclude=usr/portage exclude=etc/portage --exclude=var/db --exclude=tmp/* --exclude=var/cache/*' to the first tar if you don't mind leaving the unemerge programs in that the before-build.sh scripts removes. This cuts the time down quite a bit.

Tested it, it seemed to work, but tar does not retain the permissions correctly so in the end it is a bad choise. Which is to bad because it was fast, I am currently looking into cpio. (man tar : tar [ -p, --same-permissions, --preserve-permissions ]

Also, rsync works very well as it only updates what has been updated.



Note : the following tube should work all the time. With the first line, it makes a mini-script that is as error-safe as can be. It should preserve ownerships.

test $# == 2 || exit 2 ; SOURCE=$1 ; shift ; DEST=$1 ; shift ;

( cd $SOURCE && tar c . ) | ( cd $DEST && tar xpvf - )

Testing the LiveCD without wasting a blank CD

If you're trying out some weird stuff with your LiveCD and don't want to waste a blank CD (and don't have a rewritable) each time you've built a new ISO-image, you can emulate the LiveCD with qemu, a very nice open-source emulator.

If you don't have qemu already, of course you need to emerge it (kqemu is an accelerator module)

emerge qemu kqemu
  1. Notice from falcon15: Perhaps you should do 'echo "app-emulation/qemu kqemu" >> /etc/portage/package.use' before merging qemu in order to make use of kqemu

To use it, there are several possibilities:

# first, load the kqemu kernel module
# in order to get /dev/kqemu in udev on most distros you'll need to do:
# modprobe kqemu major=0
modprobe kqemu

#simple example of qemu usage
qemu -cdrom livecd.iso

#give more resources on the host systems (you need to be root for this)
nice -n 2 qemu -cdrom livecd.iso

#give more (emulated) memory, here: 512 megs
qemu -cdrom livecd.iso -m 512

It may be that some exotic functions are not supported by qemu, yet this shouldn't happen too often. If emulation starts failing at a very basic point, you might want to try emulating something like a Knoppix image in order to see if your qemu install is ok. However, booting sometimes hangs for about ten minutes or even more at

>>> Copying read-write image contents to tmpfs

This is due to qemu's memory management. But apart from the waiting, this is a harmless effect, and it doesn't seems to occur on a burned LiveCD.

Making a bootable floppy boot from a boot option in your cd's bootloader

This is very simple after I figured this out, you just need a few basic things;

Get syslinux

 emerge syslinux

Then copy the memdisk to your boot folder

 cp /usr/lib/syslinux/memdisk /boot/memdisk

Then make an image of your bootable floppy

 dd if=/dev/fd0 of=/boot/floppy

And after all these hard steps just make an boot option (example in grub)

 nano -w /boot/grub/grub.conf

and addhave not compiled kernel, do the following: (End of comment)

 title=Bootable floppy from livecd
        root (cd)
        kernel (cd)/boot/memdisk
        initrd (cd)/boot/floppy

That's all there is to it!

32bit chroot on a 64bit installation

Important Note: if you are following this guide from within a 64-bit environment, consult the "32Bit Chroot Guide for Gentoo/AMD64" at [3] for how to properly chroot.

To modify the build script change (the fast way);

chroot ${TARGET_SOURCE}/ /bin/bash --login <<CHROOTED
to
linux32 chroot ${TARGET_SOURCE}/ /bin/bash --login <<CHROOTED

Comment: If you don't have linux32 at hand, you could also just modify the folowing: make.conf for right architecture, run make ARCH=i386 (menuconfig) or genkernel --arch-override=x86 initrd/all, (depends on what you use)!

Floppy tips

If you have a floppy that you have made bootable, but need to make it bigger so more files will fit on it, that can be done very simple.

Create a new file as big as you need it to be

dd if=/dev/zero of=/new_image bs=18k seek=200 count=75

Make it a file system (dos for this instance).

mkfs.msdos /new_image

Mount the new image and fill it (redo the above steps if you still have not got enough space).

mount -o loop /new_image /mnt/something

Fill it with the files you need. The next step is to get the old bootsector (orriginal_floopy can be a dd image or a /dev node);

dd if=/orriginal_floppy of=/orriginal_boot_sector bs=512 count=1

Then simply merge the bootsector and new_image;

cat /orriginal_boot_sector /new_image > /new_bootable_image

After that you can simply start it as the old floppy!

This can also be done for hd's, I recommend using something like vmware make a small hd and install whatever you want on it (needs to be bootable). The trick is now to make a dd image of the complete hd (example: dd if=/dev/hda of=~/hd.dd).

This is now bootable as in the previous floppy example, but since it takes the whole hd the dd image can be large, luckily memdisk supports gzip. So gzip -9 your-dd-image.dd does a neat trick here.

COMPLETE BUILD SCRIPT

I had problems running this scripts : mounting/umounting different parts (sys, proc, ...), so I change to a plain text method. This way, reader can understand step by step what's happening and can check what's wrong.

build script

To help automate these steps, create a simple 'build' script inside ~/livecd This builds the complete livecd from scratch at once, be careful it replaces menu.lst with grub.conf (so we do not forget!)


Then create the build script :

cd ~/livecd
touch build
chmod +x build 

And here's the script

#!/bin/bash
LIVECD=~/livecd
TARGET=${LIVECD}/target
TARGET_SOURCE=${TARGET}/files/source/
SOURCE=${LIVECD}/source

# It is important, that you don't have mounted these filesystems - cp will fail in /proc when it's mounted. >>>
umount ${SOURCE}/sys
umount ${SOURCE}/dev
umount ${SOURCE}/proc
umount ${SOURCE}/usr/portage
# <<<

rm -rf ${TARGET}
mkdir ${TARGET}
cp -a ${SOURCE}/boot ${TARGET}/
mkdir -p ${TARGET}/files/source
#make sure the menu.lst is up to date (ONLY EDIT grub.conf!!)
rm ${TARGET}/boot/grub/menu.lst
cp ${TARGET}/boot/grub/grub.conf ${TARGET}/boot/grub/menu.lst

#Copy all the files for processing
cp -p -R -P -d ${SOURCE}/ ${TARGET}/files

cd ${TARGET}/files
mount -o bind /sys ${TARGET_SOURCE}/sys
mount -o bind /dev ${TARGET_SOURCE}/dev
mount -o bind /proc ${TARGET_SOURCE}/proc
mount -o bind /usr/portage ${TARGET_SOURCE}/usr/portage
chroot ${TARGET_SOURCE} /bin/bash --login <<CHROOTED
env-update
/sbin/depscan.sh
update-modules
localepurge
makewhatis -u
#You can uncomment the following lines if you want, I use those so I do not forget to merge these packages (kernel updates etc).
# emerge sys-fs/fuse dmraid alsa-driver captive nvidia-kernel nvidia-glx
# emerge unmerge localepurge genkernel ccache
find / -xdev -name ".keep" -exec rm -rf {} \;
CHROOTED
umount ${TARGET_SOURCE}/sys
umount ${TARGET_SOURCE}/dev
umount ${TARGET_SOURCE}/proc
umount ${TARGET_SOURCE}/usr/portage
env-update
source /etc/profile

cd ${TARGET_SOURCE}
rm -rf var/tmp/*
#rm -rf var/run/*
rm -rf var/lock/*
rm -rf var/cache/*
mkdir -p var/lock/subsys
mkdir -p var/cache/edb/deb
mkdir var/cache/hald
rm -rf var/db
rm -rf tmp/*
rm -f etc/mtab
touch etc/mtab
rm -rf usr/portage
rm -rf etc/portage
rm -rf usr/share/doc
rm root/.bash_history
rm root/.zcompdump
rm -rf var/log
mkdir var/log
rm etc/make.profile
rm -rf boot
rm -rf root/.ccache
rm -rf root/.mc
rm -rf initrd
rm before-build

cd ${TARGET}/files
mksquashfs source/ ${TARGET}/livecd.squashfs
cd ${TARGET}
touch livecd
rm -rf ${TARGET}/files/
cd ${LIVECD}
mkisofs -R -b boot/grub/stage2_eltorito -no-emul-boot -boot-load-size 4 -boot-info-table -iso-level 4 -hide-rr-moved -c boot.catalog -o ${LIVECD}/livecd.iso ${TARGET}/

The emerge fuse and dmraid makes sure we do not forget to built these modules when we updated the kernel.

Make both executable, and be sure to read the warning message. If you do not check if it has succesfully chrooted to the ~/target/files/source directory and you run the file it might DESTROY your whole system!

Squashing a large source directory takes some time, so be patient. Once done, just burn your ISO image, boot it, and enjoy :)

Work script

A basic work script which enables you to easily work in the chroot, a more advanced automated script is below. work.

cd livecd
touch work
chmod +x work 

Here it is:

File: Work script, automatically mount and chroot the livecd
#!/bin/bash
mount -o bind /proc source/proc
mount -o bind /sys source/sys
mount -o bind /dev source/dev
mount -o bind /dev/pts source/dev/pts
mount -o bind /usr/portage source/usr/portage
chroot source/ /bin/bash --login
umount source/proc
umount source/sys
umount source/dev/pts
umount source/dev
umount source/usr/portage

Do not forget, after running this script when you get in the environment, to manually run;

env-update
source /etc/profile

Because a simple bash script is not able to give commands to the new environment, but it does unmount , run env-update and source /etc/profile after you leave the environment.

Comment: To automate this step just remove the --login part and add the folowing to source/root/.bash_rc

File: additional script, run inside the chroot
env-update
source /etc/profile
cd ~

Here is what I use as 'work' script. This makes sure that you can invoke 'work' multiple times, quit and still have a sane environment to work in the rest of the windows.

#!/bin/bash

# change to my directory
cd /home/root/livecd

#
# mount if not already mounted
#
grep -q /home/root/livecd/source/proc /etc/mtab
if [ "$?" == "1" ]
then
       mount -o bind /proc source/proc
       mount -o bind /sys source/sys
       mount -o bind /dev source/dev
       mount -o bind /dev/pts source/dev/pts
       mount -o bind /usr/portage source/usr/portage
       mount -o bind /tmp source/tmp
       sed -i -e "s:31m\\\u:31mCHROOT-64:" source/root/.bashrc
       sed -i -e "s:31m\\\u:31mCHROOT-64:" source/root/.bash_profile
fi
chroot source/ /bin/bash

# find out if there is anybody using the chroot
count=0
for i  in `ps -aef| grep $0 | grep -v grep | awk '{print $2}'`
do
       rln=`readlink /proc/$i/cwd`
       if [ "$rln" == "/home/root/livecd" ]
       then
               count=$((count+1))
       fi
done
if [ "$count" == "1" ]
then
       sed -i -e "s:31mCHROOT-64:31m\\\u:" source/root/.bashrc
       sed -i -e "s:31mCHROOT-64:31m\\\u:" source/root/.bash_profile
       umount source/proc
       umount source/sys
       umount source/dev/pts
       umount source/dev
       umount source/usr/portage
       umount source/tmp
fi
Retrieved from "http://www.gentoo-wiki.info/HOWTO_build_a_LiveCD_from_scratch"

Last modified: Thu, 04 Sep 2008 13:53:00 +1000 Hits: 144,918

Created by NickStallman.net, Luxury Homes Australia
Real estate agents should list their apartments, townhouses and units in Australia.