TIP_AMD64-x86-distcc

Introduction

At first I tried the whole crossdev bit et al.. and now I feel a bit foolish, so I thought I'd save someone else some time...

On x86_64 you can build 32 bit x86 binaries..natively..

1. Make sure you're on the multilib amd64 profile. If you don't know what I'm talking about, then you already are so don't worry.
2. Setup distcc on host and client(s) where host is amd64.
3. Add '-m32' to CFLAGS='...' on your client systems.
4. Since gentoo now uses i686-pc-linux-gnu-gcc etc., you're are going have to run emerge as follows:

   CC=gcc CXX=g++ emerge <package>

5. Off you go.. (assuming you followed the distcc howto you're all set)

Setup (without crossdev)

If you want to check whether its working with the distcc monitors you have to set the DISTCC_DIR to /var/tmp/portage/.distcc for it to work thus we run distccmon-text for example as follows:

DISTCC_DIR="/var/tmp/portage/.distcc" distccmon-text N  (where N, an integer, is the update interval)

Don't use this method for emerge -e system.

Edit--
There isn't much point in using distcc for emerge -e system anyway as most of the packages disable distcc by default. However if you really want to, emerge -e system can be made to work by:
1. running emerge -e --pretend system > sys-emrg.sh
2. editing sys-emrg.sh to replace '[ebuild N ]' with 'emerge -1O =' {ensure that the '=' sign ends up immediately to the left of the package name}
3. edit sys-emrg.sh to have #!/bin/bash as the first line and remove the line containing emerge =sys-devel/gcc-w.x.y-rz {where w,x,y,z are the version and release numbers for your installed version of gcc eg. gcc-3.4.4-r1}
4. sh sys-emrg.sh and then find something to keep you entertained for quite some time!

This method is probably more appropriate for 'emerge -e world', as far more of the packages will allow distcc to act as the compiler.

Also the emerge seems to work fine without needing to use the prefix 'CC=gcc CXX=g++'

Setup (with crossdev)

Like the original author of this wiki page, I had built crossdev support on AMD64's to assist in the build of my Celeron 300.

On the AMD64's run

emerge crossdev (if you haven't already)

crossdev -t i686-pc-linux-gnu

This may simplify cross compiling later on. For example I don't need to prefix emerge with 'CC=gcc CXX=g++' each time which is nice.

Distcc Setup

To enable cross compiling while still preserving your existing distcc configuration, copy and edit the following files on your x86_64 machines.

cp /etc/init.d/distccd /etc/init.d/distccd-x86 edit /etc/init.d/distccd-x86 and change the path variable as shown in bold below (assumes i686 arch in this example)

#!/sbin/runscript
# $Header: /var/cvsroot/gentoo-x86/sys-devel/distcc/files/2.18/init,v 1.1 2004/10/12 17:21:43 lisa Exp $

depend() {
        need net
        use ypbind
}

start() {
        [ -e "${DISTCCD_PIDFILE}" ] && rm -f ${DISTCCD_PIDFILE} &>/dev/null

        ebegin "Starting distccd"
        chown distcc `dirname ${DISTCCD_PIDFILE}` &>/dev/null
        TMPDIR="${TMPDIR}" \
        PATH="$/usr/i686-pc-linux-gnu/bin:${PATH}" \
        /sbin/start-stop-daemon --start --quiet --startas ${DISTCCD_EXEC} \
        --pidfile ${DISTCCD_PIDFILE} -- \
        --pid-file ${DISTCCD_PIDFILE} -N ${DISTCCD_NICE} --user distcc \
        ${DISTCCD_OPTS}

        eend $?
}

stop() {
        ebegin "Stopping distccd"
        start-stop-daemon --stop --quiet --pidfile "${DISTCCD_PIDFILE}"
        rm -f "${DISTCCD_PIDFILE}"
        eend $?
}

cp /etc/conf.d/distccd /etc/conf.d/distccd-x86 edit /etc/conf.d/distccd-x86 and make the changes as shown in bold below

# Copyright 1999-2004 Gentoo Foundation
# $Header: /var/cvsroot/gentoo-x86/sys-devel/distcc/files/2.18/conf,v 1.2 2004/10/21 16:20:10 vapier Exp $
# distccd configuration file

DISTCCD_OPTS=""

# this is the distccd executable
DISTCCD_EXEC="/usr/bin/distccd"

# this is where distccd will store its pid file
DISTCCD_PIDFILE="/var/run/distccd/distccd-x86.pid"

# set this option to run distccd with extra parameters
# Default port is 3632.  For most people the default is okay.
DISTCCD_OPTS="${DISTCCD_OPTS} --port 3633"
#This is a change from the defaul of 3622
DISTCC_DIR="/dev/shm/.distcc-x86"
#I run the distcc_dir in memory for increased performance you will need to alter the DISTCC_DIR flag for distccmon accordingly
# Logging
# You can change some logging options here:
# --log-file FILE
# --log-level LEVEL  [critical,error,warning, notice, info, debug]
#
# Leaving --log-file blank will log to syslog
# example: --log-file /dev/null --log-level warning
# example: --log-level critical

DISTCCD_OPTS="${DISTCCD_OPTS} --log-level critical"

# SECURITY NOTICE:
# It is HIGHLY recomended that you use the --listen option
# for increased security. You can specify an IP to permit connections
# from or a CIDR mask
# --listen accepts only a single IP
# --allow is now mandatory as of distcc-2.18.
# example:  --allow 192.168.0.0/24
# example:  --allow 192.168.0.5 --allow 192.168.0.150
# example:  --listen 192.168.0.2
DISTCCD_OPTS="${DISTCCD_OPTS} --allow 192.168.0.1"
#This assumes your x86 machine has address 192.168.0.1
DISTCCD_OPTS="${DISTCCD_OPTS} --listen 192.168.0.2"
#This assumes your x86_64 host has address 192.168.0.2"
# set this for niceness
# Default is 15
DISTCCD_NICE="0"

On your x86 machine you need to make the following changes:

edit /etc/distcc/hosts

192.168.0.2:3633 192.168.0.3:3633 localhost
#:3633 ensures we are sending on the port number specified on the host machines

Run the x86 distcc version on each AMD64 host:

/etc/init.d/distcc-x86 start

Compiling x86_64 packages on 32bit x86

On the x86's run crossdev -t x86_64-pc-linux-gnu

To enable cross compiling while still preserving your existing distcc configuration, copy and edit the following files on your x86 machines.

cp /etc/init.d/distcc /etc/init.d/distcc-x86_64 edit /etc/init.d/distcc-x86_64 and change the path variable as shown in bold below (assumes x86_64 arch in this example)

#!/sbin/runscript
# $Header: /var/cvsroot/gentoo-x86_64/sys-devel/distcc/files/2.18/init,v 1.1 2004/10/12 17:21:43 lisa Exp $

depend() {
        need net
        use ypbind
}

start() {
        [ -e "${DISTCCD_PIDFILE}" ] && rm -f ${DISTCCD_PIDFILE} &>/dev/null

        ebegin "Starting distccd"
        chown distcc `dirname ${DISTCCD_PIDFILE}` &>/dev/null
        TMPDIR="${TMPDIR}" \
        PATH="$/usr/x86_64-pc-linux-gnu/bin:${PATH}" \
        /sbin/start-stop-daemon --start --quiet --startas ${DISTCCD_EXEC} \
        --pidfile ${DISTCCD_PIDFILE} -- \
        --pid-file ${DISTCCD_PIDFILE} -N ${DISTCCD_NICE} --user distcc \
        ${DISTCCD_OPTS}

        eend $?
}

stop() {
        ebegin "Stopping distccd"
        start-stop-daemon --stop --quiet --pidfile "${DISTCCD_PIDFILE}"
        rm -f "${DISTCCD_PIDFILE}"
        eend $?
}

cp /etc/conf.d/distcc /etc/conf.d/distcc-x86_64 edit /etc/conf.d/distcc-x86_64 and make the changes as shown in bold below

# Copyright 1999-2004 Gentoo Foundation
# $Header: /var/cvsroot/gentoo-x86_64/sys-devel/distcc/files/2.18/conf,v 1.2 2004/10/21 16:20:10 vapier Exp $
# distccd configuration file

DISTCCD_OPTS=""

# this is the distccd executable
DISTCCD_EXEC="/usr/bin/distccd"

# this is where distccd will store its pid file
DISTCCD_PIDFILE="/var/run/distccd/distccd-x86_64.pid"
 
# set this option to run distccd with extra parameters
# Default port is 3632.  For most people the default is okay.
DISTCCD_OPTS="${DISTCCD_OPTS} --port 3633"
#This is a change from the defaul of 3622
#EDIT: i couldnt find this line when i tried this step... so a left it out!!  
#DISTCC_DIR="/dev/shm/.distcc-x86_64" 
#I run the distcc_dir in memory for increased performance you will need to alter the DISTCC_DIR flag for distccmon accordingly
# Logging
# You can change some logging options here:
# --log-file FILE
# --log-level LEVEL  [critical,error,warning, notice, info, debug]
#
# Leaving --log-file blank will log to syslog
# example: --log-file /dev/null --log-level warning
# example: --log-level critical

DISTCCD_OPTS="${DISTCCD_OPTS} --log-level critical"

# SECURITY NOTICE:
# It is HIGHLY recomended that you use the --listen option
# for increased security. You can specify an IP to permit connections
# from or a CIDR mask
# --listen accepts only a single IP
# --allow is now mandatory as of distcc-2.18.
# example:  --allow 192.168.0.0/24
# example:  --allow 192.168.0.5 --allow 192.168.0.150
# example:  --listen 192.168.0.2
DISTCCD_OPTS="${DISTCCD_OPTS} --allow 192.168.0.1"
#This assumes your x86 machine has address 192.168.0.1
DISTCCD_OPTS="${DISTCCD_OPTS} --listen 192.168.0.2"
#This assumes your x86_64 host has address 192.168.0.2"
# set this for niceness
# Default is 15
DISTCCD_NICE="0"

On your amd64 machine you need to make the following changes:

edit /etc/distcc/hosts

192.168.0.2:3633 192.168.0.3:3633 localhost
#:3633 ensures we are sending on the port number specified on the host machines

Run the AMD64 distcc version on each x86 host:

/etc/init.d/distcc-x86_64 start

Happy Cross-Compiling!!