HOWTO_FS-Cache
Contents |
FS-Cache HowTo
CacheFS is written by David Howells / Red Hat, Inc.
It is a general cache facility and can be used to cache NFS mounts.
I'm currently working on an integration to a diskless Gentoo installation.
Like in http://mozart.informatik.fh-kl.de/download/Software/GentooDiskless/diskless2_en.pdf
In the End CacheFS should work together with UnionFS and minimize the network traffic.
Linux-cachefs mailing list https://www.redhat.com/mailman/listinfo/linux-cachefs
Kernel requirements
Gentoo MM-Sources
For this installation, the only working after my experience, you need the mm-sources.
emerge mm-sources
At this moment they are in version 2.6.19-rc5-mm1 available.
After merge you need to activate the Cache and NFS or AFS Support
cd /usr/src/linux-2.6.19-rc5-mm1
make menuconfig
The needed options are in section “File systems” > “Caches” >
Device Drivers --->
>File systems --->
Instrumentation Support --->
<M> Kernel automounter version 4 support (also supports v3)
<M> Filesystem in Userspace support
>Caches --->
CD-ROM/DVD Filesystems --->
DOS/FAT/NT Filesystems --->
<*> Filesystem caching on files
[*] Debug CacheFiles
and in section “File systems” > “Network File Systems”
Miscellaneous filesystems --->
>Network File Systems --->
Partition Types --->
<*> NFS file system support
[*] Provide NFSv3 client support
[ ] Provide client support for the NFSv3 ACL protocol extension [*] Provide NFSv4 client support (EXPERIMENTAL)
>[*] Provide NFS client caching support (EXPERIMENTAL)
[*] Allow direct I/O on NFS files
Now build your new kernel with
make && make install && make modules_install
Gentoo-Sources
Patches are available http://people.redhat.com/~dhowells/fscache/patches/
Extract Patches witch
tar xjf nfs+fscache-19.tar.bz2
Get fresh sources
emerge gentoo-sources cd /usr/src/linux
Now patch in order of apperance / patches have numbers
patch -p1 < /path/to/patchset/allPatches
Use the same Options as above in mm-sources
Userspace installation
Conditions
All CacheFS Sources are given as .rpm packages. In order to convert them you need these packages.
emerge rpm rpm2targz
Cachefilesd
Current version 0.8
http://people.redhat.com/~dhowells/fscache/
- cachefilesd-0.8.tar.bz2
- cachefilesd-0.8-15.fc7.src.rpm
Converting .rpm sources to tar.bz2
rpm2cpio cachefilesd-0.8-15.fc7.src.rpm |cpio -i
Extract & compile
tar xjf cachefilesd-0.8.tar.bz2
cd cachefilesd-0.8
make && make install
NFS-Utils
emerge libgssapi
http://people.redhat.com/~steved/fscache/nfs-utils/1.0.9-5/
mkdir nfs-utils
cd nfs-utils
rpm2cpio ../nfs-utils-1.0.9-5.fc6.src.rpm |cpio -i
Using the current Gentoo nfs-utils sources :
emerge -f nfs-utils
cp /usr/portage/distfiles/nfs-utils-1.0.10.tar.gz .
or use the given sources in the rpm “nfs-utils-1.0.9.tar.bz2”
I'm using the current Gentoo sources
tar xfz nfs-utils-1.0.10.tar.gz
cd nfs-utils-1.0.10
patch -p1 < ../allPatches
./configure –disable-gss
libtoolize --copy --force
make && make install
By this way, the subdirectory "utils/mount" wont be compiled, but you'll need it.
So you have to do it by your own.
cd utils/mount
make && make install
If you have later trouble mounting nfs-shares,
try
cp /usr/sbin/mount.nfs* /sbin/
Harddrive requirements
CacheFS can only be used with a EXT3 filesystem
Also you need to tune your partition with
tune2fs -o user_xattr /dev/hdaX
The Cache is declared in “/etc/cachefilesd.conf”
with the Option “dir”. The default is "/var/fscache"
So declare your partition in your "/etc/fstab"
/dev/hdaX /var/fscache ext3 defaults 0 0
Or try to find it at boot time. Therefore add a Label to the Cache
tune2fs -L CacheFS /dev/hdaX
A script could look like this
#!/bin/bash
label()
{
#while [ $(/sbin/e2label $1) != "" ]; do
while [ $1 ]; do
if [ $(/sbin/e2label $1) == "CacheFS" ]; then
/bin/mount $1 /var/fscache
# CacheFS needs portmap can't be started at this time
#/sbin/cachefilesd -s
#mount -t nfs -o fsc 10.0.8.17:/tftproot/gentoo /
echo "using CacheFS"
else
echo $1." is not a Cache"
fi
shift
done
}
label $(/bin/ls /dev/hda?)
Working with FS-Cache
localhost ~ # cachefilesd -s
About to bind cache
Bound cache
localhost ~ # mount -t nfs -o fsc ip:/source/dir /destination/dir
When the cache is bound properly, you'll find 2 directories under "/var/fscache"
- cache
- graveyard
Cache entries will be found under cache like cryptic "\@4a/I0nfs/\@41" and so on.
For Testing you can copy some files, like this
- From NFS to Hd, read from NFS and writing 2 times on Hd (cache init)
time cp source destination
- Read only from Cache to Ram or /dev/null,
time cp source /dev/null
- From Cache to Hd, read from Hd and write to Hd
time cp source destination
Stats using FS-Cache
One Big File
the File is about 350 MB
File is written 2 times on hd (cache init)
localhost ~ # time cp /nfsmount/oneBig.file . real 6m10.907s user 0m0.172s sys 0m12.161s
Copy from cache, file is read from hd
localhost ~ # time cp /nfsmount/oneBig.file /dev/null real 1m42.042s user 0m0.144s sys 0m52.467s
Copy from cache, read from and write to hd
localhost ~ # time cp /nfsmount/oneBig.file . real 3m33.246s user 0m0.176s sys 1m1.348s
many small files
1.8G Kernel sources
Normal NFS copy, no cache
localhost ~ # time cp -r /nfsmount/usr/src . real 33m47.274s user 0m6.192s sys 1m45.919s
Files are written 2 times on hd (cache init)
localhost ~ # time cp -r /nfsmount/usr/src/ . real 73m52.464s user 0m5.440s sys 3m55.663s
With cache, read from hd localhost ~ # time tar cf /dev/null /mnt/root/usr/src
tar: Removing leading `/' from member names real 18m59.711s user 0m2.640s sys 0m32.150s
Conclusion
The cache is good while reading files.
As soon as many small files are copied, causes the cache opposite. Particularly on slow harddrives. Solution: Put cache to another harddrive.
Created by NickStallman.net, Luxury Homes Australia
Real estate agents should list their apartments, townhouses and units in Australia.