|Laptops • TV Tuner Cards • Wireless • Servers • Storage • Other Hardware • Motherboards • Related|
The Proliant brand & the 6500
The Proliant brand refers to Compaq's, and now HP's, range of x86 servers. They are typically high-end and multi-processor capable, featuring hardware RAID disk-arrays & hot-swappable power-supplies. The first 6500 was based on the Pentium Pro, but later models were slot-1, accommodating "one to four Intel Pentium II Xeon® or Pentium III Xeon processors" and these are now available remarkably cheaply on eBay.
When I stumbled across the price of the 6500, I was unable to resist, but I justify my purchase by reasoning that a hardware RAID array is just the perfect way to atone for my lethargy over backing-up data - I have about 4 years' of email on my current server and whilst none of it is mission-critical, I'd hate to lose it through carelessness & a drive-failure.
Specification of my machine
The specification I'll be using on which to draft this document is:
- 4 x Xeon PII 450Mhz CPUs.
- 3 x hot-swappable SCSI 9.1gig hard-drives, with room for 4 more.
- 2 x hot-swappable power supplies.
- 1gig of RAM.
- warrantied & working for under £100!
HP / Compaq's Linux Support
Linux is supported on Proliants, but it looks like the reason these older models are becoming available so cheap might be that HP's support is fading for them - if you go to their software & drivers download page and choose the Proliant 6500 you'll see that Suse 6.3 & 7 and RedHat 6.1, 6.2 & 7 are mentioned. I don't know what security updates are available for these distros - if we can get Gentoo on our beast, then it won't matter! - but at the time of writing Suse is on version 9.3, which is supported for current Proliant models.
Buying an older Proliant
These 6500s can be picked up so cheaply from knowledgeable dealers - who are often paid by the corporate previous owners for disposing of the server & securely wiping their data - that I really don't think it's worth buying one from anyone else. My first contact with a Proliant was about 2 weeks ago, when a friend of mine picked one up as part of an office clearance. It was looking pretty scrappy, and when he hit the power button the lights would come on but nothing else would happen.
Considering that the machine I ended up with cost me less than £100 and had a warranty, my mate did me a favour by refusing to flog me his - it turns out that many Proliants have "interlocks" on the chassis, switches which prevent them from booting unless the lid is on. At the time the lid was lying on the floor, but had I bought it I'd probably still be trying to start the thing - the last thing I normally do when I'm tinkering with a PC is close the case!
I'd advise you to steer clear of too-good-to-be-true bargains & look to buy from someone who shifts a lot of this kind of server, who knows these systems & knows how to run the diagnostic utilities. Even something as simple as the one or two components on a harddrive-tray can fail, and sometimes not until the system's been running for several hours & overheated; the symptoms of this look just like a knackered hard-drive - probably including data loss - so the classic mistake is to replace the disk, only to have the same thing happen a few hours later!
SmartStart & low-level system setup
An arcane BIOS experience
I haven't used software RAID in the past, but my understanding is that it can be configured from the o/s - my Macintosh's Disk Utility has a section which allows me to plug in Firewire drives & RAID them, for instance. Hardware RAID, as on the Proliants, is a different Kenwood of piscines, however - the RAID controller might typically present 3 SCSI disks as a single virtual drive and so the configuration has to be made at the firmware level before the o/s is able to see the logical device(s).
Other aspects of the 6500's BIOS are also quite unusual - you can't adjust the order of the boot-devices or change the date without software! Compaq provide a bootable "SmartStart" CD, which allows access to some of these settings, but to configure all of them you need to install utilities from that disk onto a "Compaq System Partition" which is then accessed at bootime by pressing F10.
Many Gentooers are happy to format their RAID arrays without a Compaq System Partion, but I'm a small, petty & obsessive person, and I want all the diagnostic tools that Compaq provide on the F10 partition, even if it makes the install more fiddly. It also allows us to do cool stuff like muck around with the Proliant's LCD display & send ourselves pager messages if the system encounters problems.
Boot with the SmartStart CD
As far as I can tell, the supported SmartStart CD for the Proliant 6500 is version 5.50 - that one worked for me, anyway. Usage is pretty straight forward:
To make sure everything is update i recommend downloading the Firmware Maintenace CD 7.30-0 it will update your board bios and add in cards.
Boot from the SmartStart CD & hit F1 when prompted
Run System Erase Utility
to take the system back to factory settings.
Run Array Configuration Utility
RAID stands for something like "Redundant Array of Inexpensive Disks", but RAID can actually be configured in a number of different ways, including some with no redundancy. Non-redundant RAID arrays are only useful when no protection against hard-drive failure is required, but speed is of the essence - for instance, in any action half of the data might read or written to each of two drives, thus halving the read/write times; this might be suitable for an ISP's web-proxy or news server, where the data stored has itself little value and can be recovered easily from other sites on the infrastructure, but access time is imperative.
Far more common are applications like mine, where redundancy is the whole point of the array - if one of the hard-drives fails I can be sure that my data is still safe on the other disks in the array. It is possible to configure RAID so that even if several disks fail one can be assured of data integrity. Apparently most people choose the same as me - RAID 5 - but you might find these definitions useful, and these pictures pretty
Update System Partition
which will install the Compaq System Partition. I think at this stage at you'll get the opportunity to say you'll be installing Linux later on, but you can also do that from the F10 menu.
Run System Configuration Utility
to set the SCSI bus boot-order.
- I might have these last two in the wrong order.
Boot from a Linux CD & Partition
This is where things get interesting - did you notice how I didn't say boot from the Gentoo liveCD & partition?
The first time I installed on this box I had no problems at all using cfdisk /dev/ida/c0d0/disc, and if you skipped the instruction to Update System Partition above, neither will you. In that case, Gentoo will be seeing the RAID array as a blank hard-drive at this point, but we won't have F10 partition installed & I'll sulk.
If you've been following carefully & installed the Compaq utilities, you may run into problems at this stage - it seemed to me that either the tools shipped on Gentoo's 2003.2 LiveCD don't like Compaq's proprietary partition format, or perhaps something is missing from the LiveCD's kernel. My experience was that fdisk, cfdisk & partedit all b0rked on the logical drive at this stage, complaining that it wasn't a valid partition layout and refusing to allow me to add partitions.
After some Googling, all the references I could find on installing Linux on Proliants with the Compaq System Partition intact referred to RedHat, so I ended up booting from one of their CDs that I had lying around - I'd downloaded it for a friend, honest!
Fedora Core 1 is later than the versions of RedHat listed as supported on Compaq's webpage, but sure enough it handles the Compaq System Partition marvellously. If you'd like to give me a Proliant hard-drive tray or two, I'll be glad to swap out my disks & document this better, and I'd like to try some other approaches, like Knoppix and other/later Gentoo LiveCDs, too.
Basically, the process I used was:
- Boot from Fedora Core 1 CD
- Choose text installation process - I think the command was linux text at the first prompt.
- Choose manual install.
- Use RedHat's partitioning tool to add partitions - it seems to do this fine. The tool is kinda graphical as I recall, with drop-downs & stuff, so it's easy enough to use. I went with my standard layout of < 100meg boot, some swap, and a / partition.
- Tell the installer NOT to do that thing with GRUB in the MBR.
- Let RedHat partition the drive for you. It may not do this until after the packages selection screen, I just selected a minimal install.
- You can probably pull the plug & reboot once the installer shows the partitioning has finished, but I allowed it to continue & finish the RedHat install.
If you have success with fdisk, cfdisk or partedit from a Gentoo liveCD then I'd be extremely grateful to hear about it. Also, if you have a Knoppix kicking around, I'd appreciate it if you could try partitioning from that before going the RedHat route & let me know how you get on.
-edit, this might work for the 6500 Series but it does work for the Proliant 8000 when booting gentoo (tested on 2006 liveCD) try these boot params.
"boot: gentoo-nofb nox nodetect mem=960M ncpus=1 i8042.nomux" the most inportant part is the "i8042.nomux" everything else is more or less optional, also there seems to be a bug in the 10/12/2001 Bios (hangs the liveCD @ IDE Detection), currently using 02/10/2001 and the system boots fine.
You may find some of these Compaq utilities useful:
- Firmware Maintenance CD - version 7.20, 14 Feb 05
- HP SmartStart 5.50 CD &/or version 7.00 (12 Dec 03)
- Software 7.20, whatever that contains.
HP don't seem to know which of their CDs support which Proliant models, so you may need a different version - look at the SmartStart CD Server Support Matrix and the ProLiant Software Maintenance CD Matrix and take a random guess which one is correct.
Installing a Gentoo base-system
I'm assuming here that you've installed Gentoo before - if you haven't and you're still reading you might want to practice a few times on a cheap desktop PC before proceeding, because I might miss something out - I'm following the Gentoo Quick Install Guide.
Booting to the LiveCD
The hardware RAID which is so important to me is provided to this machine by a Compaq Smart-2 PCI card, which uses the cpqarray module. I found that the Gentoo 2004.2 liveCD detected this & modprobed it for me automagically, as long as I booted the LiveCD with the smp kernel, smp dokeymap as the liveCD boots.
I was pretty impressed at how smoothly the liveCD handled this all - after all, it probably wasn't tested on one of these servers. The biggest glitch for me was that I plugged my network cable into the wrong Ethernet port - apparently the liveCD doesn't expect us to have multiple network cards. Pah!
Now is a good time to check out exactly what hardware's in the system, so we know what modules to add to the kernel later on.
|Code: Checking the system hardware|
livecd root # lspci -tv -+--+-01.0-----00.0 Compaq Computer Corporation Smart-2/P RAID Controller | +-04.0---+-00.0 Compaq Computer Corporation Netelligent Dual 10/100 TX PCI UTP | | \-01.0 Compaq Computer Corporation Netelligent Dual 10/100 TX PCI UTP | \-0b.0 Compaq Computer Corporation PCI Hotplug Controller \--+-0b.0 Compaq Computer Corporation PCI Hotplug Controller +-0c.0 Compaq Computer Corporation Advanced System Management Controller +-0d.0 LSI Logic / Symbios Logic 53c875 +-0d.1 LSI Logic / Symbios Logic 53c875 +-0e.0 ATI Technologies Inc 3D Rage IIC 215IIC [Mach64 GT IIC] +-0f.0 Intel Corp. 82371AB/EB/MB PIIX4 ISA +-0f.1 Intel Corp. 82371AB/EB/MB PIIX4 IDE +-0f.2 Intel Corp. 82371AB/EB/MB PIIX4 USB +-0f.3 Intel Corp. 82371AB/EB/MB PIIX4 ACPI +-10.0 Intel Corp. 450NX - 82451NX Memory & I/O Controller +-12.0 Intel Corp. 450NX - 82454NX/84460GX PCI Expander Bridge \-14.0 Intel Corp. 450NX - 82454NX/84460GX PCI Expander Bridge
It's worth mentioning that there are no USB ports on my machine - the USB controller is obviously part of this Intel 82371 chipset, but really not much use on this kind of server, at least at the time it was manufactured.
Checking the partitions
If you're used to EIDE then the /dev entries for the RAID logical drive might look a bit unusual - they remind me of "Irix", which was shipped on my first Unix box, a blue one. A logical drive under cpqarray might be referred to as either /dev/cciss/c0d0 or /dev/ida/c0d0/, depending on your kernel.
ahoogerhuis explains why & describes partition addessing in this forums post:
• The DL360G3 uses the cciss-driver
Fortunately as rumba explains in the same thread there is another way of addressing the drives:
I'm also using a Compaq DL 380 with a Compaq 431 Smart Array. I'm always using /dev/discs/...
Since the devfs introduction that's my preferred way to access my discs. No matter if it's IDE, SCSI, or some of the more exotic block devices like the mention RAID controllers, they just appear in /dev/discs/...
|Code: Checking partitioning|
livecd root # ls -d /dev/discs/*/* /dev/discs/disc0/disc /dev/discs/disc0/part2 /dev/discs/disc0/part4 /dev/discs/disc0/part1 /dev/discs/disc0/part3 livecd root # fdisk /dev/discs/disc0/disc The number of cylinders for this disk is set to 4355. There is nothing wrong with that, but this is larger than 1024, and could in certain setups cause problems with: 1) software that runs at boot time (e.g., old versions of LILO) 2) booting and partitioning software from other OSs (e.g., DOS FDISK, OS/2 FDISK) Command (m for help): p Disk /dev/discs/disc0/disc: 18.1 GB, 18194841600 bytes 255 heads, 32 sectors/track, 4355 cylinders Units = cylinders of 8160 * 512 = 4177920 bytes Device Boot Start End Blocks Id System /dev/discs/disc0/part1 * 10 24 61200 83 Linux /dev/discs/disc0/part2 25 281 1048560 82 Linux swap /dev/discs/disc0/part3 1 9 36704 12 Compaq diagnostics /dev/discs/disc0/part4 282 4355 16621920 83 Linux Partition table entries are not in disk order Command (m for help): q livecd root # mkdir /mnt/compaq-diags livecd root # mount -v /dev/discs/disc0/part3 /mnt/compaq-diags mount: you didn't specify a filesystem type for /dev/discs/disc0/part3 I will try type vfat /dev/discs/disc0/part3 on /mnt/compaq-diags type vfat (rw) livecd root # ls -C /mnt/compaq-diags/ | head -n 5 !!cfg.ndx !cpq5255.cfg !cpqa015.cfg 1014003e.pcf cpqidecd.cmn !adp0000.cfg !cpq525d.cfg !cpqa040.cfg 10222000.pcf cpqivga.cmn !bus4201.cfg !cpq5281.cfg !cpqa045.cfg 10222020.pcf cpqls120.cmn !cpq0521.cfg !cpq5282.cfg !cpqf000.cfg 102b0518.pcf cpqmdm.cmn !cpq0531.cfg !cpq5287.cfg !cpqf120.cfg 102b0519.pcf cpqmem.cmn livecd root # for foo in `ls --color=no /dev/discs/disc0/ | grep part` \ ; do mkdir /mnt/$foo ; mount -v /dev/discs/disc0/$foo /mnt/$foo ; done mount: you didn't specify a filesystem type for /dev/discs/disc0/part1 I will try type ext3 /dev/discs/disc0/part1 on /mnt/part1 type ext3 (rw) mount: you didn't specify a filesystem type for /dev/discs/disc0/part2 and it looks like this is swapspace /dev/discs/disc0/part2 looks like swapspace - not mounted mount: you must specify the filesystem type mount: you didn't specify a filesystem type for /dev/discs/disc0/part3 I will try type vfat /dev/discs/disc0/part3 on /mnt/part3 type vfat (rw) mount: you didn't specify a filesystem type for /dev/discs/disc0/part4 I will try type ext3 /dev/discs/disc0/part4 on /mnt/part4 type ext3 (rw) livecd root # df -h Filesystem Size Used Avail Use% Mounted on tmpfs 506M 2.9M 503M 1% / /newroot/dev/cdroms/cdrom0 605M 605M 0 100% /mnt/cdrom /dev/loop/0 70M 70M 0 100% /mnt/livecd /dev/discs/disc0/part3 36M 6.4M 30M 18% /mnt/compaq-diags /dev/discs/disc0/part1 58M 8.4M 47M 16% /mnt/part1 /dev/discs/disc0/part3 36M 6.4M 30M 18% /mnt/part3 /dev/discs/disc0/part4 16G 509M 15G 4% /mnt/part4 livecd root #
Format the partitions
The RedHat disk I used to partition didn't seem to support ReiserFS, and wrote its files all over the boot partition, too, so I just:
|Code: Format the Gentoo partitions|
livecd root # mkreiserfs /dev/discs/disc0/part4 livecd root # mke2fs -j /dev/discs/disc0/part1
Continue with the usual install stuff
Mount partitions, unpack stage, chroot in:
|Code: Quick stage3 install from a universal CD|
livecd root # mount /dev/discs/disc0/part4 /mnt/gentoo/ livecd root # mkdir /mnt/gentoo/boot livecd root # mount /dev/discs/disc0/part1 /mnt/gentoo/boot/ livecd root # cd /mnt/gentoo/ livecd gentoo # tar -xvjpf /mnt/cdrom/stages/stage3-i686* livecd gentoo # tar -xvjf /mnt/cdrom/snapshots/portage-*.tar.bz2 -C /mnt/gentoo/usr livecd gentoo # mkdir /mnt/gentoo/usr/portage/distfiles livecd gentoo # cp /mnt/cdrom/distfiles/* /mnt/gentoo/usr/portage/distfiles/
There is one very important addition when chrooting in - you need to also mount /dev, so that Grub will recognise the /dev/ida/c0d0/ entries
|Code: Mount /proc & /dev, chroot in|
livecd gentoo # mount -t proc none /mnt/gentoo/proc livecd gentoo # mount -o bind /dev /mnt/gentoo/dev livecd gentoo # chroot /mnt/gentoo /bin/bash livecd / # env-update; source /etc/profile * Caching service dependencies... livecd / #
# <fs> <mountpoint> <type> <opts> <dump/pass> # NOTE: If your BOOT partition is ReiserFS, add the notail option to opts. /dev/discs/disc0/part1 /boot ext2 noauto,noatime 1 1 /dev/discs/disc0/part4 / reiserfs noatime 0 0 /dev/discs/disc0/part2 none swap sw 0 0 /dev/cdroms/cdrom0 /mnt/cdrom iso9660 noauto,ro 0 0 #/dev/fd0 /mnt/floppy auto noauto 0 0 # NOTE: The next line is critical for boot! none /proc proc defaults 0 0 none /dev/shm tmpfs defaults 0 0
devfs is depreciated as of 2.6.14 & udev becomes mandatory. Under udev the naming scheme for disks & partitions under the Compaq Array Controller have changed slightly, and /dev/discs/ no longer seems to refer to them:
# <fs> <mountpoint> <type> <opts> <dump/pass> # NOTE: If your BOOT partition is ReiserFS, add the notail option to opts. /dev/ida/c0d0p1 /boot ext2 noauto,noatime 1 2 /dev/ida/c0d0p2 none swap sw 0 0 /dev/ida/c0d0p4 / reiserfs noatime 0 1
I'm sure that one or two of these options, such as the fake PCI hotplug driver, aren't absolutely necessary, but others such as SMP support and the Compaq SMART options we really don't want to be without. The legacy /proc/scsi/ stuff just looked interesting. In any case, if you're reading these options here, they haven't broken my system. I would like to know more about what the ACPI PCI Hotplug driver does - I'm of the if-in-doubt-compile-it-statically-into-the-kernel school of philospohy - so if you know, please feel free to get in touch.
CONFIG_MPENTIUMII=y CONFIG_SMP=y CONFIG_NR_CPUS=4 Processor type and features ---> Processor family (Pentium-II/Celeron(pre-Coppermine)) [*] Symmetric multi-processing support (4) Maximum number of CPUs (2-255) CONFIG_HOTPLUG=y CONFIG_HOTPLUG_PCI=y CONFIG_HOTPLUG_PCI_FAKE=y CONFIG_HOTPLUG_PCI_COMPAQ=y CONFIG_HOTPLUG_PCI_COMPAQ_NVRAM=y CONFIG_HOTPLUG_PCI_ACPI=y Bus options (PCI, PCMCIA, EISA, MCA, ISA) ---> [*] PCI support PCI Hotplug Support ---> <*> Support for PCI Hotplug (EXPERIMENTAL) <*> Fake PCI Hotplug driver <*> Compaq PCI Hotplug driver [*] Save configuration into NVRAM on Compaq servers <*> ACPI PCI Hotplug driver CONFIG_BLK_CPQ_DA=y CONFIG_BLK_CPQ_CISS_DA=y CONFIG_SCSI=y CONFIG_SCSI_PROC_FS=y Device Drivers ---> Block devices ---> <*> Compaq SMART2 support <*> Compaq Smart Array 5xxx support SCSI device support ---> <*> SCSI device support [*] legacy /proc/scsi/ support
The following is for my Netelligent network card, as shown above - if lsmod from the LiveCD shows "tlan", then you probably need this. For more information on which devices are supported by this module see /usr/src/linux/Documentation/networking/tlan.txt
CONFIG_TLAN=y Device Drivers ---> Networking support ---> [*] Networking support <*> TI ThunderLAN support
The graphics card is an ATI 3D Rage [Mach64 GT IIC] - I think these are the right modules, but I'll probably be removing them tomorrow, as per the instruction to "do not enable nvidia or ATi specific options" in The Gentoo Bootsplash How-To.
CONFIG_FB_ATY=y CONFIG_FB_ATY_CT=y CONFIG_FB_ATY_GX=y CONFIG_FB_ATY_XL_INIT=y Device Drivers ---> Graphics support ---> [*] Support for frame buffer devices <*> ATI Mach64 display support [*] Mach64 CT/VT/GT/LT (incl. 3D RAGE) support [*] Mach64 GX support [*] Rage XL No-BIOS Init support
Richard Black explains on his page General instructions for Installing Linux onto Compaq Servers:
The functionality of pressing F10 at startup to access the system partition is located in the MBR. If you overwrite the MBR with LILO, you will not be able to boot to the system partition by pressing F10 on startup. Here are the steps to recover your F10 functionality and still be able to boot Linux.
The GRUB manual tells us how to do this with our preferred bootloader. Ok, ok... my preferred bootloader... get over it:
Once started, GRUB will show the command-line interface (see Command-line interface). First, set the GRUB's root device4 to the partition containing the boot directory, like this:
If you are not sure which partition actually holds this directory, use the command find (see find), like this:
Once you've set the root device correctly, run the command setup (see setup):
This command will install the GRUB boot loader on the Master Boot Record (MBR) of the first drive. If you want to put GRUB into the boot sector of a partition instead of putting it in the MBR, specify the partition into which you want to install GRUB:
You might also access the System Partition from your Bootloader as it is a simple FAT partition with a chainloader. With Grub it looks like this (or slightly different on your system)
|File: grub.conf - Grub entry for the Compaq System Partition|
title Systempartition map (hd2) (hd0) map (hd0) (hd2) rootnoverify (hd0,2) chainloader +1 makeactive boot
If you configure the system using the SmartStart CD as I have done, then the Compaq System Partition will automatically chainload onto a bootloader placed in the active partition, IE /boot
|Code: Installing GRUB into the boot sector of /boot|
# mount /boot/ # grub grub> find /boot/grub/stage1 (hd0,0) grub> root (hd0,0) Filesystem type is ext2fs, partition type 0x83 grub> setup (hd0,0)
Don't forget to mark /boot as bootable using fdisk.
- Gentoo-User Mailing List
- Gentoo on an old Compaq/SCSI RAID - probably not that relevant, but you might want to use this site to search the mailing-lists.
- Gentoo Forums
- Gentoo Installation guide for Grub/Compaq Smart2 controllers - I guess the reason I don't really use the forums is that I find the format of unthreaded questions & answers hard to read.
- Gentoo on a Compaq Proliant DL380 (with SmartArray 5300) - it's the discussion of the SmartArray that makes this interesting to us - it seems many Proliants of this era take quite similar configuration.
- Compaq Proliant 2500 Memory problems.... I experienced no problems like this on the 6500 - the 2500 seems to be quite a chunk older.
- where are my hardware raid drives?? (solved) - frankly, I like the answer given by Compaq in their RAID Array Setup Utility, "the blinking lights may help you to identify the drives".
- compaq (hp) Insight agents installation - appears to include information on ebuilds for the official HP management utilites.
- Installing Gentoo on a Proliant - getting RAID running on a 1600 & loadsof useful output from a 5000 quad Pentium-Pro.
- Installing with the Compaq System Partition
- Spark's SmartArray tools - the author says, "these tools are build by me to help monitor our servers' smartarray controllers. This Array Util thingy gives a nice overview of the configuration of your controller, and allows you to blink the leds of currently selected drives. This CPQ Array Daemon keeps on monitoring your controller and checks for abnormal conditions. By default is only reports to the syslogs, but you can make it send SNMP Traps to selected hosts." I'm hoping these will allow my server to send me an email should a hard-drive go down.
- Compaq Servers and Linux website - updated regularly, although perhaps not comprehensively, there's loads of info here.
- HP / Compaq website
- Proliant Support Team Number ( 1-800-652-6672 ) - added by Mi6
Created by NickStallman.net, Luxury Homes Australia
Real estate agents should list their apartments, townhouses and units in Australia.