HOWTO_Install_Gentoo_or_another_Linux_distro_from_USB_flash_drive_using_syslinux

What this HOWTO about?

This HOWTO is a step by step instruction needed to create bootable Linux USB flash drive. It describes how to make bootable USB flash drive with files taken from Gentoo minimal installation CD image. Such bootable flash drive may be used in place of Gentoo installation CD to install Gentoo. Also you may copy Gentoo Installer LiveCD to your USB flash drive and then use this flash drive to boot and run Gentoo from.

Is this HOWTO Gentoo specific?

This HOWTO shows how to build bootable USB flash drive with files copied from Gentoo minimal installation CD image. Also you may copy files from Gentoo Installer LiveCD. If you do not need Gentoo installation USB flash, you may write another distribution installation files to your USB flash drive. For example, you may wish to copy kernel file and initrd image from Fedora or Redhat installation CD. Then you can boot from USB flash drive with Fedora/Redhat kernel and initrd image so you can install Fedora or Redhat using Anaconda.

Probably every modern Linux distribution provides installation CD. These CDs use isolinux to boot kernel from CD image and load initrd image to RAM disk. After expanding compressed initrd image to RAM disk these CDs start installation program. This HOWTO shows by Gentoo example how to boot from USB flash drive and how to load almost any distribution kernel and initrd.

Procedure described in this HOWTO has been successfully tested on CentOS, Debian Lenny and Gentoo.

Why do we need bootable Linux USB flash drive?

You may need bootable Linux USB flash drive in cases such as:

• You have laptop without CD/DVD drive and want to install Linux.
• You have laptop without CD/DVD drive and need to boot rescue image in order to repair installed Linux.
• You are tracking some distro's -current branch and you know that any static installation image will become obsolete soon.
• You build custom bootable image tailored for your hardware or filled with software you need. You do not want to write CD disk after any minor change made to your custom bootable image.
• You want to see how some Linux distribution works with you hardware. You want to boot many test LiveCD images and then select only one of many revisions or snapshots. You do not want to write lot of CD disks to throw away soon.
• You care of environment and you know that industry harms our planet when making each non-reusable CD/DVD disk.

Prerequisites.

Before you begin, you need:

• Any working Linux distribution.
• USB flash drive. Obviously USB flash drive should be big enough to fit Gentoo Minimal Installation CD in and should be at least some kilobytes bigger than Gentoo Minimal Installation CD iso image file.
• Software:
  • fdisk or sfdisk
  • grub
  • syslinux
  • squashfs-tools (this is Gentoo specific requirement)
• ISO image

You need to download Gentoo Minimal Installation CD. By the time of writing this is an iso image of 58 megabytes:

[a@nx6110 gentoo]$ ls -hl install-x86-minimal-2007.0-r1.iso 
-rw-r--r-- 1 a a 58M Dec 22 11:08 install-x86-minimal-2007.0-r1.iso

Partitioning and formatting.

Warning: all information stored on you flash drive will be lost. Make a backup if necessary. Check twice to be sure you partitioning your USB flash drive but not another drive.

• Plug your USB flash drive into USB port. If your flash drive has LED, this LED should be lit when drive is properly connected to USB port and your hardware has been recognized by Linux.
• Do you know you USB flash drive device name? Some modern Linux distros name both hard drives and USB flash drive as /dev/sdX devices where X is an alphabet letter. Be sure you partition your USB flash drive, but not hard drive. To be absolutely sure, see your kernel bootup messages:

[a@nx6110 ~]$ dmesg | less

• Search for usb-storage keyword:

usb 5-1: new high speed USB device using ehci_hcd and address 2
usb 5-1: configuration #1 chosen from 1 choice
scsi0 : SCSI emulation for USB Mass Storage devices
usb-storage: device found at 2
usb-storage: waiting for device to settle before scanning
  Vendor: Generic   Model: STORAGE DEVICE    Rev: 9144
  Type:   Direct-Access                      ANSI SCSI revision: 00
SCSI device sda: 1984000 512-byte hdwr sectors (1016 MB)
sda: Write Protect is off
sda: Mode Sense: 02 00 00 00
sda: assuming drive cache: write through
SCSI device sda: 1984000 512-byte hdwr sectors (1016 MB)
sda: Write Protect is off
sda: Mode Sense: 02 00 00 00
sda: assuming drive cache: write through
 sda: sda1
sd 0:0:0:0: Attached scsi removable disk sda
usb-storage: device scan complete
sd 0:0:0:0: Attached scsi generic sg0 type 0
SELinux: initialized (dev sda1, type vfat), uses genfs_contexts
usb 5-1: USB disconnect, address 2
usb 5-1: new high speed USB device using ehci_hcd and address 3
usb 5-1: configuration #1 chosen from 1 choice
scsi1 : SCSI emulation for USB Mass Storage devices
usb-storage: device found at 3
usb-storage: waiting for device to settle before scanning
  Vendor: Generic   Model: STORAGE DEVICE    Rev: 9144
  Type:   Direct-Access                      ANSI SCSI revision: 00
SCSI device sda: 3920896 512-byte hdwr sectors (2007 MB)
sda: Write Protect is off
sda: Mode Sense: 02 00 00 00
sda: assuming drive cache: write through
SCSI device sda: 3920896 512-byte hdwr sectors (2007 MB)
sda: Write Protect is off
sda: Mode Sense: 02 00 00 00
sda: assuming drive cache: write through
 sda: sda1
sd 1:0:0:0: Attached scsi removable disk sda
sd 1:0:0:0: Attached scsi generic sg0 type 0
usb-storage: device scan complete

As you can see here I have /dev/sda device associated with my USB flash drive. Check your kernel messages to see yours.

• Login as a root. Run fdisk on your USB drive. Enter p fdisk command to see how your flash drive partitioned and check total drive size to see you editing partition table of correct drive. If you are not sure you partitioning you USB flash drive, enter q fdisk command now to quit without saving changes. Remember, all data on partitioned media will be lost! fdisk commands you have to enter show in bold letters below.

[root@nx6110 ~]# /sbin/fdisk /dev/sda
Command (m for help): p
 Disk /dev/sda: 2007 MB, 2007498752 bytes
 62 heads, 62 sectors/track, 1020 cylinders
 Units = cylinders of 3844 * 512 = 1968128 bytes
   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1        1020     1960409    6  FAT16

• Now create a new empty DOS partition table (o), add a new partition (n), change a partition's system id (t) to FAT32 and toggle a partition bootable flag (a):

Command (m for help): o
Building a new DOS disklabel. Changes will remain in memory only,
until you decide to write them. After that, of course, the previous
content won't be recoverable.
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)
Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-1020, default 1): 
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-1020, default 1020): 
Using default value 1020
Command (m for help): t
Selected partition 1
Hex code (type L to list codes): 0B
Changed system type of partition 1 to 0B (W95 FAT32)
Command (m for help): a
Partition number (1-4): 1
Command (m for help):

When fdisk asks you for first and last cylinder numbers, just press Enter to accept default values.

• Now check partition you just created (p) and save changes (w) if all looks okay. If you want to start partitioning over, enter o command to build empty partition table. If you want to quit without saving changes, enter q command.

Command (m for help): p
Disk /dev/sda: 2007 MB, 2007498752 bytes
62 heads, 62 sectors/track, 1020 cylinders
Units = cylinders of 3844 * 512 = 1968128 bytes
   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1        1020     1960409    6  FAT32
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
The kernel still uses the old table.
The new table will be used at the next reboot.
WARNING: If you have created or modified any DOS 6.x
partitions, please see the fdisk manual page for additional
information.
Syncing disks.

If you prefer sfdisk, please run sfdisk as shown below:

sfdisk -D /dev/sda <<-EOF
  0;50;0x0B;*
  ;
  ;
  ;
  y
EOF

As you may see, we use 0-50 "cylinders" for 1st partition marked as FAT32 (type 0x0B), mark this partition as bootable and confirm writing.

• Now format you new FAT32 partition. Note we access /dev/sda1 as a first partition while /dev/sda is a whole device which may contain several partitions.

[root@nx6110 ~]# mkdosfs -vF 32 /dev/sda1
mkdofs 2.11 (12 Mar 2005)
mkdofs: /dev/sda1 contains a mounted file system.
[root@nx6110 ~]# umount /dev/sda1
[root@nx6110 ~]# mkdosfs -vF 32 /dev/sda1
mkdofs 2.11 (12 Mar 2005)

First attempt to format USB flash drive failed because Linux has automatically mounted new partition. I have unmounted this volume before second attempt. If you current Linux environment does not mount plugged in volumes automatically, you will succeed on first attempt without unmounting you new flash drive partition.

• Now you have USB flash drive with empty FAT32 filesystem.

Writing syslinux bootloader.

Check you syslinux version. If you use Gentoo, install fresh syslinux:

[root@nx6110 ~]# ACCEPT_KEYWORDS="~x86" emerge sys-boot/syslinux

Note that older versions of syslinux does not support directories on bootable image. We use "~x86" keyword above to ask Gentoo installer to select newer version of syslinux. If your distribution offers syslinux older than 3.50, probably you should compile syslinux from sources. If you can't, then you must place all installation files in root directory and edit syslinux.cfg accordingly. Usually it is easier to install fresh syslinux version.

Now look for MBR image:

nx6110 ~ # updatedb
nx6110 ~ # locate mbr.bin
/usr/lib/syslinux/mbr.bin

And copy MBR to your flash drive:

nx6110 ~ # cat /usr/lib/syslinux/mbr.bin > /dev/sda

The MBR you have written seeks for syslinux bootloader placed in filesystem. So now you could write syslinux bootloader to you empty FAT32 partition.

[root@nx6110 ~]# syslinux /dev/sda1
[root@nx6110 ~]# mount /dev/sda1
[root@nx6110 ~]# ls -l /media/flash/
total 32
-r-xr-xr-x 1 root root 9660 Dec 22 15:58 ldlinux.sys

As you can see above, now you have written ldlinux.sys file to your empty partition. MBR executable code looks for this file and pass control to the code written in this file. Syslinux can read FAT32 filesystem, can load kernels and copy initrd images to ramdisk and boot linuces.

Stop point.

Now it is good time to check everything go well and your computer's BIOS understand code you have written to USB flash drive. Enter computer BIOS setup and enable booting from USB drives. In many cases you have to enable BIOS feature called like "Enable booting from USB hard drive" or so. Plug your USB flash drive into USB port and reboot. If you see BIOS boot menu, select boot device called "USB hard drive" or so. Wait while computer boots and look what error mesage will be written last. In case everything is okay your will see message like that on black and white screen:

SYSLINUX 3.52 EBIOS Copyright (C) 1994 H. Peter Anvin
Could not find kernel image: linux
boot:

What this message means for you? It's meaning is very important.

1. Your computer is okay and could successfully boot.
2. Your computer's BIOS is fresh enough to boot from USB devices.
3. Your computer's BIOS recognizes your USB flash drive as a compatible storage device.
4. Yuor computer's BIOS recognizes your USB flash drive as a correct bootable (and somewhat DOS-compatible) device.
5. Your computer's BIOS has succesfully passed control to your USB flash drive MBR executable code. The decision is made upon your boot device menu selection.
6. MBR code ran well and has found syslinux file on FAT16 filesystem.
7. When received control from MBR code, syslinux code ran well too.

Syslinux defaults to load linux kernel from file named "linux" placed in the root directory of boot media. You have not placed linux kernel to your USB flash drive yet so no surprise syslinux has not found linux kernel there. Your computer did not boot from USB flash drive and you had not see mesage like shown above? Try following:

• Try to add -s switch to syslinux command.

syslinux -s /dev/sda1

• Your computer's BIOS is too old. Could you download and install update to your computer's BIOS?

Copying kernel, initrd image and syslinux config files.

If you are reading these lines then hopefully your computer's BIOS is compatible with MBR written by ms-sys and with syslinux code. Now it is a good time to write Linux kernel, initrd compressed image and other files needed to boot USB flash drive. The easiest way to get these files is downloading installation CD iso image to copy files from. Most installation CDs use isolinux to boot Linux. syslinux and isolinux are two parts of one project. So you can use most of isolinux specific files with syslinux and successfully boot linux.

In general you need three files:

1. syslinux.cfg
2. linux kernel file
3. initrd image

When syslinux gets control from MBR executable code, it looks for syslinux.cfg in the root directory of boot drive. Then syslinux loads linux kernel and initrd files. The names of kernel and initrd files and their placement are specified in syslinux.cfg file. Hopefully you already know what the Linux kernel is. The initrd image is an usually compressed set of files needed for Linux kernel. Usually initrd image files are relatively small. Most important role of initrd files is to be a place to store kernel modules in.

When computer boots up, kernel loads modules read from initrd image. Usually only small subset of loadable kernel modules being stored in initrd image. Initrd makers select modules needed to mount boot media and read root filesystem.

• For example, when you boot from SCSI device, obviously kernel need to load SCSI modules to access SCSI hardware. In this case you have to store all SCSI related kernel modules in initrd image. syslinux reads initrd image, copies them to RAM, then kernel loads SCSI related modules and kernel mounts root SCSI filesystem, reads and executes startup scripts and so on.
• On the other hand, when you boot your root filesystem from IDE drive and mount SCSI drives later, you do not need to store SCSI drivers in initrd image. In later case initrd image contain modules needed only to boot from IDE disk and SCSI modules could be loaded later by the system startup scripts.

So in general you need to write at least three files to your USB flash drive now.

• syslinux.cfg If you want to make Slackware bootable USB flash drive, copy isolinux.cfg file from isolinux directory of Slackware distribution. If you use Redhat or Fedora, copy isolinux.cfg from os/i386/isolinux/ directory and name the copy of isolinux.cfg file as syslinux.cfg.
• kernel image If you want to make Slackware bootable USB flash drive, copy bzImage file from preferred subdirectory located in kernels directory of Slackware distribution. If you use Redhat or Fedora, copy vmlinuz file from os/i386/isolinux/ directory.
• initrd image If you want to make Slackware bootable USB flash drive, copy initrd.img file from isolinux directory of Slackware distribution. If you use Redhat or Fedora, copy initrd.img file from os/i386/isolinux/ directory.

For Debian (and probably for Ubuntu) download small mini.iso image from main/installer-i386/current/images/netboot/ directory of Debian/Ubuntu repository. Then mount this image as loop filesystem and extract required files from mounted filesystem.

Now edit syslinux.cfg file written to your USB flash drive (copied from isolinux.cfg file) and fix paths to kernel and initrd image files. See Gentoo specific example below. Save edited file, unmount USB flash drive to commit cached files and reboot. After reboot you should see your preferred distribution installation program running. Consult your distribution manual to see installation or repair instructions as if you boot install or repair CD disk.

Copying Gentoo installation files.

When you have your USB flash drive ready with MBR and syslinux file (ldlinux.sys) written and your system has succesfully passed boot test, copy Gentoo installation files from Gentoo Minimal Installation CD to your flash drive. Assume you have mounted USB flash drive as /media/flash/:

[a@nx6110 gentoo]$ mount
/dev/hda1 on / type ext3 (rw)
...
/dev/sda1 on /media/flash type vfat (rw,noexec,nosuid,nodev,user=a)

Then mount Gentoo Minimal Installation CD iso image as a readonly loop system:

[root@nx6110 gentoo]# mount -r -o loop install-x86-minimal-2007.0-r1.iso /mnt/
[root@nx6110 gentoo]# ls -l /mnt/
total 51468
-rw-r--r-- 1 root root   7008256 May  8  2007 gentoo.efimg
-rw-r--r-- 1 root root     21776 May  8  2007 Getting_Online.txt
-rwx------ 1 root root  45662208 May  8  2007 image.squashfs
drwxr-xr-x 2 1000 users     4096 May  8  2007 isolinux
-rw-r--r-- 1 root root         0 May  8  2007 livecd
-rw-r--r-- 1 root root      6222 May  8  2007 README.txt

Now copy installation files:

[root@nx6110 gentoo]# cp -r -v /mnt/* /media/flash/
`/mnt/gentoo.efimg' -> `/media/flash/gentoo.efimg'
`/mnt/Getting_Online.txt' -> `/media/flash/Getting_Online.txt'
`/mnt/image.squashfs' -> `/media/flash/image.squashfs' # compressed filesystem
`/mnt/isolinux' -> `/media/flash/isolinux'
`/mnt/isolinux/boot.cat' -> `/media/flash/isolinux/boot.cat'
`/mnt/isolinux/boot.msg' -> `/media/flash/isolinux/boot.msg'
`/mnt/isolinux/elilo.efi' -> `/media/flash/isolinux/elilo.efi'
`/mnt/isolinux/F2.msg' -> `/media/flash/isolinux/F2.msg'
`/mnt/isolinux/F3.msg' -> `/media/flash/isolinux/F3.msg'
`/mnt/isolinux/F4.msg' -> `/media/flash/isolinux/F4.msg'
`/mnt/isolinux/F5.msg' -> `/media/flash/isolinux/F5.msg'
`/mnt/isolinux/F6.msg' -> `/media/flash/isolinux/F6.msg'
`/mnt/isolinux/F7.msg' -> `/media/flash/isolinux/F7.msg'
`/mnt/isolinux/gentoo' -> `/media/flash/isolinux/gentoo' # Linux kernel
`/mnt/isolinux/gentoo.igz' -> `/media/flash/isolinux/gentoo.igz' # initrd image
`/mnt/isolinux/isolinux.bin' -> `/media/flash/isolinux/isolinux.bin'
`/mnt/isolinux/isolinux.cfg' -> `/media/flash/isolinux/isolinux.cfg' # isolinux config
`/mnt/isolinux/kernels.msg' -> `/media/flash/isolinux/kernels.msg'
`/mnt/isolinux/memtest86' -> `/media/flash/isolinux/memtest86'
`/mnt/livecd' -> `/media/flash/livecd'
`/mnt/README.txt' -> `/media/flash/README.txt'

Most important files marked bold in listing above.

Now copy isolinux.cfg file used by isolinux.bin when booting from CD to syslinux.cfg used by syslinux when booting from USB flash.

[root@nx6110 gentoo]# cp -v /media/flash/isolinux/isolinux.cfg /media/flash/syslinux.cfg
`/media/flash/isolinux/isolinux.cfg' -> `/media/flash/syslinux.cfg'

Editing Gentoo syslinux.cfg file.

Paths in isolinux.cfg file are written relative to isolinux directory. But syslinux reads syslinux.cfg placed in root directory of boot drive. So we need to correct paths:

default gentoo
timeout 150
prompt 1
display boot.msg
F1 kernels.msg
F2 F2.msg
F3 F3.msg
F4 F4.msg
F5 F5.msg
F6 F6.msg
F7 F7.msg
label gentoo
  kernel gentoo
  append root=/dev/ram0 init=/linuxrc  dokeymap looptype=squashfs loop=/image.squashfs  cdroot initrd=gentoo.igz vga=791 splash=silent,theme:livecd-2007.0 CONSOLE=/dev/tty1 quiet
label gentoo-nofb
  kernel gentoo
  append root=/dev/ram0 init=/linuxrc  dokeymap looptype=squashfs loop=/image.squashfs  cdroot initrd=gentoo.igz
label memtest86
  kernel memtest86

Corrected syslinux.cfg file with absolute paths shown below:

default gentoo
timeout 150
prompt 1
display /isolinux/boot.msg
F1 /isolinux/kernels.msg
F2 /isolinux/F2.msg
F3 /isolinux/F3.msg
F4 /isolinux/F4.msg
F5 /isolinux/F5.msg
F6 /isolinux/F6.msg
F7 /isolinux/F7.msg
label gentoo
  kernel /isolinux/gentoo
  append root=/dev/ram0 init=/linuxrc dokeymap looptype=squashfs loop=/image.squashfs cdroot initrd=/isolinux/gentoo.igz vga=792 splash=silent,theme:livecd-2007.0 CONSOLE=/dev/tty1 quiet doload=nls_cp437,nls_iso8859-1
label gentoo-nofb
  kernel /isolinux/gentoo
 append root=/dev/ram0 init=/linuxrc dokeymap looptype=squashfs loop=/image.squashfs cdroot initrd=/isolinux/gentoo.igz doload=nls_cp437,nls_iso8859-1
label memtest86
  kernel memtest86

You see string doload=nls_cp437,nls_iso8859-1 added to kernel parameters line. The meaning of this string explained below.

Adding missing kernel modules to initrd image file.

Gentoo Minimal Installation CD's initrd image contains kernel modules needed to boot Gentoo from CD/DVD drive. If you reboot your computer now, you will see error messages:

>>No bootable medium found. Waiting for new devices...
...
!!Could not find CD to boot, something else needed!
...
!!The root block device is unspecified or not detected.

Booting fails because initrd image taken from Gentoo Minimal Installation CD has no NLS (National Language Support) kernel modules. These modules not needed to boot from CD/DVD drive but required to boot from USB flash drive formatted as FAT16 filesystem.

In order to add kernel modules to initrd image file you need to:

1. Unpack existing Gentoo initrd image file.
2. Unpack Gentoo squashed filesystem image.
3. Take missing modules from squashed filesystem and add them to initrd files.
4. Pack new initrd image file.
5. Compress initrd image file and copy it to USB flash drive.

Gentoo initrd image stored as compressed cpio archive. So we create new directory, descsnd to this new directory and unpack initrd image stored in Gentoo Minimal Installation CD iso image which we have mounted before as a loop filesystem:

[root@nx6110 gentoo]# mkdir -v initrd
mkdir: created directory `initrd'
[root@nx6110 gentoo]# cd initrd/
[root@nx6110 initrd]# zcat /mnt/isolinux/gentoo.igz | cpio -i
23274 blocks
[root@nx6110 initrd]# cd ..
[root@nx6110 gentoo]#

Gentoo installation CD files stored in compressed squashfs filesystem. Now you need to unpack them (unsquashfs defaults to unpack squashfs filesystem to squashfs-root directory).

[root@nx6110 gentoo]# /usr/sbin/unsquashfs /mnt/image.squashfs 
created 6193 files
created 503 directories
created 1917 symlinks
created 5138 devices
created 0 fifos

Now you have both source and destination directories to merge:

[root@nx6110 gentoo]# cp -rv squashfs-root/lib/modules/2.6.19-gentoo-r5/kernel/fs/nls/ initrd/lib/modules/2.6.19-gentoo-r5/kernel/fs/
`squashfs-root/lib/modules/2.6.19-gentoo-r5/kernel/fs/nls/' -> `initrd/lib/modules/2.6.19-gentoo-r5/kernel/fs/nls'
`squashfs-root/lib/modules/2.6.19-gentoo-r5/kernel/fs/nls/nls_iso8859-1.ko' -> `initrd/lib/modules/2.6.19-gentoo-r5/kernel/fs/nls/nls_iso8859-1.ko'
`squashfs-root/lib/modules/2.6.19-gentoo-r5/kernel/fs/nls/nls_cp437.ko' -> `initrd/lib/modules/2.6.19-gentoo-r5/kernel/fs/nls/nls_cp437.ko'
`squashfs-root/lib/modules/2.6.19-gentoo-r5/kernel/fs/nls/nls_utf8.ko' -> `initrd/lib/modules/2.6.19-gentoo-r5/kernel/fs/nls/nls_utf8.ko'

Now you need to create cpio archive in newc format and compress this archive to USB flash drive.

[root@nx6110 gentoo]# cd initrd/
[root@nx6110 initrd]# find . | sort | cpio -H newc -o | gzip > /media/flash/isolinux/gentoo.igz 
23306 blocks
[root@nx6110 gentoo]# umount install-x86-minimal-2007.0-r1.iso 
[root@nx6110 gentoo]# umount /dev/sda1

That is all. Remember you already have appended doload=nls_cp437,nls_iso8859-1 string to kernel parameters line when you edited syslinux.cfg file. This parameter makes kernel to load nls_cp437.ko and nls_iso8859-1.ko kernel modules.

Now you may boot from your USB flash drive and install Gentoo as when booting from installation CD disk. Consult with Gentoo Handbook for further installation instructions.

Related reading

Even if you are not using Slackware but interested in creating bootable flash images, take a look at Eric Hameleers, also known as Alien BOB pages. Particularly you may find useful USB installer for Slackware page and USB install script. Read BOB's self-explanatory shell scripts.