Gentoo Wiki ArchivesGentoo Wiki

Compaq_ProLiant_6500

This article is part of the Hardware series.
Laptops TV Tuner Cards Wireless Servers Storage Other Hardware Motherboards Related

Contents

Introduction

The Proliant brand & the 6500

The Proliant brand refers to Compaq's, and now HP's, range of x86 servers. They are typically high-end and multi-processor capable, featuring hardware RAID disk-arrays & hot-swappable power-supplies. The first 6500 was based on the Pentium Pro, but later models were slot-1, accommodating "one to four Intel Pentium II Xeon® or Pentium III Xeon processors" and these are now available remarkably cheaply on eBay.

When I stumbled across the price of the 6500, I was unable to resist, but I justify my purchase by reasoning that a hardware RAID array is just the perfect way to atone for my lethargy over backing-up data - I have about 4 years' of email on my current server and whilst none of it is mission-critical, I'd hate to lose it through carelessness & a drive-failure.

Specification of my machine

A Proliant 6500 with one of its hot-swappable hard-drive trays withdrawn
A Proliant 6500 with one of its hot-swappable hard-drive trays withdrawn

The specification I'll be using on which to draft this document is:

HP / Compaq's Linux Support

Linux is supported on Proliants, but it looks like the reason these older models are becoming available so cheap might be that HP's support is fading for them - if you go to their software & drivers download page and choose the Proliant 6500 you'll see that Suse 6.3 & 7 and RedHat 6.1, 6.2 & 7 are mentioned. I don't know what security updates are available for these distros - if we can get Gentoo on our beast, then it won't matter! - but at the time of writing Suse is on version 9.3, which is supported for current Proliant models.

Buying an older Proliant

These 6500s can be picked up so cheaply from knowledgeable dealers - who are often paid by the corporate previous owners for disposing of the server & securely wiping their data - that I really don't think it's worth buying one from anyone else. My first contact with a Proliant was about 2 weeks ago, when a friend of mine picked one up as part of an office clearance. It was looking pretty scrappy, and when he hit the power button the lights would come on but nothing else would happen.

Considering that the machine I ended up with cost me less than £100 and had a warranty, my mate did me a favour by refusing to flog me his - it turns out that many Proliants have "interlocks" on the chassis, switches which prevent them from booting unless the lid is on. At the time the lid was lying on the floor, but had I bought it I'd probably still be trying to start the thing - the last thing I normally do when I'm tinkering with a PC is close the case!

I'd advise you to steer clear of too-good-to-be-true bargains & look to buy from someone who shifts a lot of this kind of server, who knows these systems & knows how to run the diagnostic utilities. Even something as simple as the one or two components on a harddrive-tray can fail, and sometimes not until the system's been running for several hours & overheated; the symptoms of this look just like a knackered hard-drive - probably including data loss - so the classic mistake is to replace the disk, only to have the same thing happen a few hours later!


SmartStart & low-level system setup

An arcane BIOS experience

The Compaq System Partition gives you access to BIOS settings
The Compaq System Partition gives you access to BIOS settings

I haven't used software RAID in the past, but my understanding is that it can be configured from the o/s - my Macintosh's Disk Utility has a section which allows me to plug in Firewire drives & RAID them, for instance. Hardware RAID, as on the Proliants, is a different Kenwood of piscines, however - the RAID controller might typically present 3 SCSI disks as a single virtual drive and so the configuration has to be made at the firmware level before the o/s is able to see the logical device(s).

Other aspects of the 6500's BIOS are also quite unusual - you can't adjust the order of the boot-devices or change the date without software! Compaq provide a bootable "SmartStart" CD, which allows access to some of these settings, but to configure all of them you need to install utilities from that disk onto a "Compaq System Partition" which is then accessed at bootime by pressing F10.

Many Gentooers are happy to format their RAID arrays without a Compaq System Partion, but I'm a small, petty & obsessive person, and I want all the diagnostic tools that Compaq provide on the F10 partition, even if it makes the install more fiddly. It also allows us to do cool stuff like muck around with the Proliant's LCD display & send ourselves pager messages if the system encounters problems.

Boot with the SmartStart CD

As far as I can tell, the supported SmartStart CD for the Proliant 6500 is version 5.50 - that one worked for me, anyway. Usage is pretty straight forward:

To make sure everything is update i recommend downloading the Firmware Maintenace CD 7.30-0 it will update your board bios and add in cards.

Boot from the SmartStart CD & hit F1 when prompted

Run System Erase Utility

to take the system back to factory settings.

Run Array Configuration Utility

RAID stands for something like "Redundant Array of Inexpensive Disks", but RAID can actually be configured in a number of different ways, including some with no redundancy. Non-redundant RAID arrays are only useful when no protection against hard-drive failure is required, but speed is of the essence - for instance, in any action half of the data might read or written to each of two drives, thus halving the read/write times; this might be suitable for an ISP's web-proxy or news server, where the data stored has itself little value and can be recovered easily from other sites on the infrastructure, but access time is imperative.


Far more common are applications like mine, where redundancy is the whole point of the array - if one of the hard-drives fails I can be sure that my data is still safe on the other disks in the array. It is possible to configure RAID so that even if several disks fail one can be assured of data integrity. Apparently most people choose the same as me - RAID 5 - but you might find these definitions useful, and these pictures pretty

Update System Partition

which will install the Compaq System Partition. I think at this stage at you'll get the opportunity to say you'll be installing Linux later on, but you can also do that from the F10 menu.

Run System Configuration Utility

to set the SCSI bus boot-order.

Boot from a Linux CD & Partition

This is where things get interesting - did you notice how I didn't say boot from the Gentoo liveCD & partition?

The first time I installed on this box I had no problems at all using cfdisk /dev/ida/c0d0/disc, and if you skipped the instruction to Update System Partition above, neither will you. In that case, Gentoo will be seeing the RAID array as a blank hard-drive at this point, but we won't have F10 partition installed & I'll sulk.

If you've been following carefully & installed the Compaq utilities, you may run into problems at this stage - it seemed to me that either the tools shipped on Gentoo's 2003.2 LiveCD don't like Compaq's proprietary partition format, or perhaps something is missing from the LiveCD's kernel. My experience was that fdisk, cfdisk & partedit all b0rked on the logical drive at this stage, complaining that it wasn't a valid partition layout and refusing to allow me to add partitions.

After some Googling, all the references I could find on installing Linux on Proliants with the Compaq System Partition intact referred to RedHat, so I ended up booting from one of their CDs that I had lying around - I'd downloaded it for a friend, honest!

Fedora Core 1 is later than the versions of RedHat listed as supported on Compaq's webpage, but sure enough it handles the Compaq System Partition marvellously. If you'd like to give me a Proliant hard-drive tray or two, I'll be glad to swap out my disks & document this better, and I'd like to try some other approaches, like Knoppix and other/later Gentoo LiveCDs, too.

Basically, the process I used was:

If you have success with fdisk, cfdisk or partedit from a Gentoo liveCD then I'd be extremely grateful to hear about it. Also, if you have a Knoppix kicking around, I'd appreciate it if you could try partitioning from that before going the RedHat route & let me know how you get on.

-edit, this might work for the 6500 Series but it does work for the Proliant 8000 when booting gentoo (tested on 2006 liveCD) try these boot params.

"boot: gentoo-nofb nox nodetect mem=960M ncpus=1 i8042.nomux" the most inportant part is the "i8042.nomux" everything else is more or less optional, also there seems to be a bug in the 10/12/2001 Bios (hangs the liveCD @ IDE Detection), currently using 02/10/2001 and the system boots fine.

Compaq Downloads

You may find some of these Compaq utilities useful:

HP don't seem to know which of their CDs support which Proliant models, so you may need a different version - look at the SmartStart CD Server Support Matrix and the ProLiant Software Maintenance CD Matrix and take a random guess which one is correct.


Installing a Gentoo base-system

I'm assuming here that you've installed Gentoo before - if you haven't and you're still reading you might want to practice a few times on a cheap desktop PC before proceeding, because I might miss something out - I'm following the Gentoo Quick Install Guide.

Booting to the LiveCD

The hardware RAID which is so important to me is provided to this machine by a Compaq Smart-2 PCI card, which uses the cpqarray module. I found that the Gentoo 2004.2 liveCD detected this & modprobed it for me automagically, as long as I booted the LiveCD with the smp kernel, smp dokeymap as the liveCD boots.

I was pretty impressed at how smoothly the liveCD handled this all - after all, it probably wasn't tested on one of these servers. The biggest glitch for me was that I plugged my network cable into the wrong Ethernet port - apparently the liveCD doesn't expect us to have multiple network cards. Pah!

Now is a good time to check out exactly what hardware's in the system, so we know what modules to add to the kernel later on.

Code: Checking the system hardware
livecd root # lspci -tv
-+-[04]-+-01.0-[05]----00.0  Compaq Computer Corporation Smart-2/P RAID Controller
 |      +-04.0-[06]--+-00.0  Compaq Computer Corporation Netelligent Dual 10/100 TX PCI UTP
 |      |            \-01.0  Compaq Computer Corporation Netelligent Dual 10/100 TX PCI UTP
 |      \-0b.0  Compaq Computer Corporation PCI Hotplug Controller
 \-[00]-+-0b.0  Compaq Computer Corporation PCI Hotplug Controller
        +-0c.0  Compaq Computer Corporation Advanced System Management Controller
        +-0d.0  LSI Logic / Symbios Logic 53c875
        +-0d.1  LSI Logic / Symbios Logic 53c875
        +-0e.0  ATI Technologies Inc 3D Rage IIC 215IIC [Mach64 GT IIC]
        +-0f.0  Intel Corp. 82371AB/EB/MB PIIX4 ISA
        +-0f.1  Intel Corp. 82371AB/EB/MB PIIX4 IDE
        +-0f.2  Intel Corp. 82371AB/EB/MB PIIX4 USB
        +-0f.3  Intel Corp. 82371AB/EB/MB PIIX4 ACPI
        +-10.0  Intel Corp. 450NX - 82451NX Memory & I/O Controller
        +-12.0  Intel Corp. 450NX - 82454NX/84460GX PCI Expander Bridge
        \-14.0  Intel Corp. 450NX - 82454NX/84460GX PCI Expander Bridge

It's worth mentioning that there are no USB ports on my machine - the USB controller is obviously part of this Intel 82371 chipset, but really not much use on this kind of server, at least at the time it was manufactured.

Checking the partitions

If you're used to EIDE then the /dev entries for the RAID logical drive might look a bit unusual - they remind me of "Irix", which was shipped on my first Unix box, a blue one. A logical drive under cpqarray might be referred to as either /dev/cciss/c0d0 or /dev/ida/c0d0/, depending on your kernel.

ahoogerhuis explains why & describes partition addessing in this forums post:

• The DL360G3 uses the cciss-driver
• In plain 2.4 kernels the devices (with devfs) will appear as /dev/cciss/discX/partX
• In plain 2.6 kernels the devices (with devfs) will appear as /dev/cciss/hostX/discX/partX
• In plain 2.6 kernels the devices (with udev) will appear as /dev/cciss/cXdXpX

Fortunately as rumba explains in the same thread there is another way of addressing the drives:

I'm also using a Compaq DL 380 with a Compaq 431 Smart Array. I'm always using /dev/discs/...

Since the devfs introduction that's my preferred way to access my discs. No matter if it's IDE, SCSI, or some of the more exotic block devices like the mention RAID controllers, they just appear in /dev/discs/...

Let's try:

Code: Checking partitioning
livecd root # ls -d /dev/discs/*/*
/dev/discs/disc0/disc   /dev/discs/disc0/part2  /dev/discs/disc0/part4
/dev/discs/disc0/part1  /dev/discs/disc0/part3
livecd root # fdisk /dev/discs/disc0/disc

The number of cylinders for this disk is set to 4355.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
   (e.g., DOS FDISK, OS/2 FDISK)

Command (m for help): p

Disk /dev/discs/disc0/disc: 18.1 GB, 18194841600 bytes
255 heads, 32 sectors/track, 4355 cylinders
Units = cylinders of 8160 * 512 = 4177920 bytes

                Device Boot      Start         End      Blocks   Id  System
/dev/discs/disc0/part1   *          10          24       61200   83  Linux
/dev/discs/disc0/part2              25         281     1048560   82  Linux swap
/dev/discs/disc0/part3               1           9       36704   12  Compaq diagnostics
/dev/discs/disc0/part4             282        4355    16621920   83  Linux

Partition table entries are not in disk order

Command (m for help): q

livecd root # mkdir /mnt/compaq-diags
livecd root # mount -v /dev/discs/disc0/part3 /mnt/compaq-diags
mount: you didn't specify a filesystem type for /dev/discs/disc0/part3
       I will try type vfat
/dev/discs/disc0/part3 on /mnt/compaq-diags type vfat (rw)
livecd root # ls -C /mnt/compaq-diags/ | head -n 5
!!cfg.ndx     !cpq5255.cfg  !cpqa015.cfg  1014003e.pcf  cpqidecd.cmn
!adp0000.cfg  !cpq525d.cfg  !cpqa040.cfg  10222000.pcf  cpqivga.cmn
!bus4201.cfg  !cpq5281.cfg  !cpqa045.cfg  10222020.pcf  cpqls120.cmn
!cpq0521.cfg  !cpq5282.cfg  !cpqf000.cfg  102b0518.pcf  cpqmdm.cmn
!cpq0531.cfg  !cpq5287.cfg  !cpqf120.cfg  102b0519.pcf  cpqmem.cmn
livecd root # for foo in `ls --color=no /dev/discs/disc0/ | grep part` \
 ; do mkdir /mnt/$foo ; mount -v /dev/discs/disc0/$foo /mnt/$foo ; done
mount: you didn't specify a filesystem type for /dev/discs/disc0/part1
       I will try type ext3
/dev/discs/disc0/part1 on /mnt/part1 type ext3 (rw)
mount: you didn't specify a filesystem type for /dev/discs/disc0/part2
       and it looks like this is swapspace
/dev/discs/disc0/part2 looks like swapspace - not mounted
mount: you must specify the filesystem type
mount: you didn't specify a filesystem type for /dev/discs/disc0/part3
       I will try type vfat
/dev/discs/disc0/part3 on /mnt/part3 type vfat (rw)
mount: you didn't specify a filesystem type for /dev/discs/disc0/part4
       I will try type ext3
/dev/discs/disc0/part4 on /mnt/part4 type ext3 (rw)
livecd root # df -h
Filesystem            Size  Used Avail Use% Mounted on
tmpfs                 506M  2.9M  503M   1% /
/newroot/dev/cdroms/cdrom0
                      605M  605M     0 100% /mnt/cdrom
/dev/loop/0            70M   70M     0 100% /mnt/livecd
/dev/discs/disc0/part3
                       36M  6.4M   30M  18% /mnt/compaq-diags
/dev/discs/disc0/part1
                       58M  8.4M   47M  16% /mnt/part1
/dev/discs/disc0/part3
                       36M  6.4M   30M  18% /mnt/part3
/dev/discs/disc0/part4
                       16G  509M   15G   4% /mnt/part4
livecd root #

Format the partitions

The RedHat disk I used to partition didn't seem to support ReiserFS, and wrote its files all over the boot partition, too, so I just:

Code: Format the Gentoo partitions
livecd root # mkreiserfs /dev/discs/disc0/part4
livecd root # mke2fs -j /dev/discs/disc0/part1

Continue with the usual install stuff

Mount partitions, unpack stage, chroot in:

Code: Quick stage3 install from a universal CD
livecd root # mount /dev/discs/disc0/part4 /mnt/gentoo/
livecd root # mkdir /mnt/gentoo/boot
livecd root # mount /dev/discs/disc0/part1 /mnt/gentoo/boot/
livecd root # cd /mnt/gentoo/
livecd gentoo # tar -xvjpf /mnt/cdrom/stages/stage3-i686*
livecd gentoo # tar -xvjf /mnt/cdrom/snapshots/portage-*.tar.bz2 -C /mnt/gentoo/usr
livecd gentoo # mkdir /mnt/gentoo/usr/portage/distfiles
livecd gentoo # cp /mnt/cdrom/distfiles/* /mnt/gentoo/usr/portage/distfiles/

There is one very important addition when chrooting in - you need to also mount /dev, so that Grub will recognise the /dev/ida/c0d0/ entries

Code: Mount /proc & /dev, chroot in
livecd gentoo # mount -t proc none /mnt/gentoo/proc
livecd gentoo # mount -o bind /dev /mnt/gentoo/dev
livecd gentoo # chroot /mnt/gentoo /bin/bash
livecd / # env-update; source /etc/profile
 * Caching service dependencies...
livecd / #

/etc/fstab

devfs

File: /etc/fstab
# <fs>                  <mountpoint>    <type>          <opts>                  <dump/pass>

# NOTE: If your BOOT partition is ReiserFS, add the notail option to opts.
/dev/discs/disc0/part1  /boot           ext2            noauto,noatime          1 1
/dev/discs/disc0/part4  /               reiserfs        noatime                 0 0
/dev/discs/disc0/part2  none            swap            sw                      0 0
/dev/cdroms/cdrom0      /mnt/cdrom      iso9660         noauto,ro               0 0
#/dev/fd0               /mnt/floppy     auto            noauto                  0 0

# NOTE: The next line is critical for boot!
none                    /proc           proc            defaults                0 0

none                    /dev/shm        tmpfs           defaults                0 0

udev

devfs is depreciated as of 2.6.14 & udev becomes mandatory. Under udev the naming scheme for disks & partitions under the Compaq Array Controller have changed slightly, and /dev/discs/ no longer seems to refer to them:

File: /etc/fstab
# <fs>                  <mountpoint>    <type>          <opts>  <dump/pass>

# NOTE: If your BOOT partition is ReiserFS, add the notail option to opts.
/dev/ida/c0d0p1         /boot           ext2            noauto,noatime  1 2
/dev/ida/c0d0p2         none            swap            sw              0 0
/dev/ida/c0d0p4         /               reiserfs        noatime         0 1

kernel configuration

I'm sure that one or two of these options, such as the fake PCI hotplug driver, aren't absolutely necessary, but others such as SMP support and the Compaq SMART options we really don't want to be without. The legacy /proc/scsi/ stuff just looked interesting. In any case, if you're reading these options here, they haven't broken my system. I would like to know more about what the ACPI PCI Hotplug driver does - I'm of the if-in-doubt-compile-it-statically-into-the-kernel school of philospohy - so if you know, please feel free to get in touch.

CONFIG_MPENTIUMII=y
CONFIG_SMP=y
CONFIG_NR_CPUS=4
  Processor type and features  --->
    Processor family (Pentium-II/Celeron(pre-Coppermine))
    [*] Symmetric multi-processing support
    (4)   Maximum number of CPUs (2-255)

CONFIG_HOTPLUG=y
CONFIG_HOTPLUG_PCI=y
CONFIG_HOTPLUG_PCI_FAKE=y
CONFIG_HOTPLUG_PCI_COMPAQ=y
CONFIG_HOTPLUG_PCI_COMPAQ_NVRAM=y
CONFIG_HOTPLUG_PCI_ACPI=y
  Bus options (PCI, PCMCIA, EISA, MCA, ISA)  --->
    [*] PCI support
      PCI Hotplug Support  --->
        <*> Support for PCI Hotplug (EXPERIMENTAL)
        <*>   Fake PCI Hotplug driver
        <*>   Compaq PCI Hotplug driver
        [*]     Save configuration into NVRAM on Compaq servers
        <*>   ACPI PCI Hotplug driver

CONFIG_BLK_CPQ_DA=y
CONFIG_BLK_CPQ_CISS_DA=y
CONFIG_SCSI=y
CONFIG_SCSI_PROC_FS=y
  Device Drivers  --->
    Block devices  --->
      <*> Compaq SMART2 support
      <*> Compaq Smart Array 5xxx support
    SCSI device support  --->
      <*> SCSI device support
      [*]   legacy /proc/scsi/ support

The following is for my Netelligent network card, as shown above - if lsmod from the LiveCD shows "tlan", then you probably need this. For more information on which devices are supported by this module see /usr/src/linux/Documentation/networking/tlan.txt

CONFIG_TLAN=y
  Device Drivers  --->
     Networking support  --->
       [*] Networking support
       <*>   TI ThunderLAN support

The graphics card is an ATI 3D Rage [Mach64 GT IIC] - I think these are the right modules, but I'll probably be removing them tomorrow, as per the instruction to "do not enable nvidia or ATi specific options" in The Gentoo Bootsplash How-To.

CONFIG_FB_ATY=y
CONFIG_FB_ATY_CT=y
CONFIG_FB_ATY_GX=y
CONFIG_FB_ATY_XL_INIT=y
  Device Drivers  --->
    Graphics support  --->
      [*] Support for frame buffer devices
      <*>   ATI Mach64 display support
      [*]     Mach64 CT/VT/GT/LT (incl. 3D RAGE) support
      [*]     Mach64 GX support
      [*]       Rage XL No-BIOS Init support

Grub

Richard Black explains on his page General instructions for Installing Linux onto Compaq Servers:

The functionality of pressing F10 at startup to access the system partition is located in the MBR. If you overwrite the MBR with LILO, you will not be able to boot to the system partition by pressing F10 on startup. Here are the steps to recover your F10 functionality and still be able to boot Linux.
...
Configure LILO to install to /boot instead of the MBR

The GRUB manual tells us how to do this with our preferred bootloader. Ok, ok... my preferred bootloader... get over it:

Once started, GRUB will show the command-line interface (see Command-line interface). First, set the GRUB's root device4 to the partition containing the boot directory, like this:
grub> root (hd0,0)

If you are not sure which partition actually holds this directory, use the command find (see find), like this:
grub> find /boot/grub/stage1

Once you've set the root device correctly, run the command setup (see setup):
grub> setup (hd0)

This command will install the GRUB boot loader on the Master Boot Record (MBR) of the first drive. If you want to put GRUB into the boot sector of a partition instead of putting it in the MBR, specify the partition into which you want to install GRUB:
grub> setup (hd0,0)

You might also access the System Partition from your Bootloader as it is a simple FAT partition with a chainloader. With Grub it looks like this (or slightly different on your system)

File: grub.conf - Grub entry for the Compaq System Partition
title Systempartition
  map (hd2) (hd0)
  map (hd0) (hd2)
  rootnoverify (hd0,2)
  chainloader +1
  makeactive
  boot

If you configure the system using the SmartStart CD as I have done, then the Compaq System Partition will automatically chainload onto a bootloader placed in the active partition, IE /boot


Code: Installing GRUB into the boot sector of /boot
# mount /boot/
# grub
grub> find /boot/grub/stage1
 (hd0,0)
grub> root (hd0,0)
 Filesystem type is ext2fs, partition type 0x83
grub> setup (hd0,0)

Don't forget to mark /boot as bootable using fdisk.

References

Retrieved from "http://www.gentoo-wiki.info/Compaq_ProLiant_6500"

Last modified: Wed, 13 Aug 2008 12:48:00 +1000 Hits: 36,686

Created by NickStallman.net, Luxury Homes Australia
Real estate agents should list their apartments, townhouses and units in Australia.