Gentoo Wiki ArchivesGentoo Wiki

HOWTO_FS-Cache

Contents

FS-Cache HowTo

CacheFS is written by David Howells / Red Hat, Inc.
It is a general cache facility and can be used to cache NFS mounts.

I'm currently working on an integration to a diskless Gentoo installation.
Like in http://mozart.informatik.fh-kl.de/download/Software/GentooDiskless/diskless2_en.pdf
In the End CacheFS should work together with UnionFS and minimize the network traffic.

Linux-cachefs mailing list https://www.redhat.com/mailman/listinfo/linux-cachefs

Kernel requirements

Gentoo MM-Sources

For this installation, the only working after my experience, you need the mm-sources.

       emerge mm-sources

At this moment they are in version 2.6.19-rc5-mm1 available.
After merge you need to activate the Cache and NFS or AFS Support

       cd /usr/src/linux-2.6.19-rc5-mm1
make menuconfig

The needed options are in section “File systems” > “Caches” >

       Device Drivers  --->
>File systems --->
Instrumentation Support --->
       <M> Kernel automounter version 4 support (also supports v3)
<M> Filesystem in Userspace support
>Caches --->
CD-ROM/DVD Filesystems --->
DOS/FAT/NT Filesystems --->
       <*> Filesystem caching on files
[*] Debug CacheFiles

and in section “File systems” > “Network File Systems”

       Miscellaneous filesystems  --->
>Network File Systems --->
Partition Types --->
       <*> NFS file system support
[*] Provide NFSv3 client support
[ ] Provide client support for the NFSv3 ACL protocol extension [*] Provide NFSv4 client support (EXPERIMENTAL)
>[*] Provide NFS client caching support (EXPERIMENTAL)
[*] Allow direct I/O on NFS files

Now build your new kernel with

       make && make install && make modules_install


Gentoo-Sources

Patches are available http://people.redhat.com/~dhowells/fscache/patches/

Extract Patches witch

tar xjf nfs+fscache-19.tar.bz2 

Get fresh sources

emerge gentoo-sources
cd /usr/src/linux

Now patch in order of apperance / patches have numbers

patch -p1 < /path/to/patchset/allPatches

Use the same Options as above in mm-sources


Userspace installation

Conditions

All CacheFS Sources are given as .rpm packages. In order to convert them you need these packages.

emerge rpm rpm2targz

Cachefilesd

Current version 0.8

http://people.redhat.com/~dhowells/fscache/


Converting .rpm sources to tar.bz2

       rpm2cpio cachefilesd-0.8-15.fc7.src.rpm |cpio -i

Extract & compile

       tar xjf cachefilesd-0.8.tar.bz2
cd cachefilesd-0.8
make && make install

NFS-Utils

       emerge libgssapi 

http://people.redhat.com/~steved/fscache/nfs-utils/1.0.9-5/

       mkdir nfs-utils
       cd nfs-utils
       rpm2cpio ../nfs-utils-1.0.9-5.fc6.src.rpm |cpio -i

Using the current Gentoo nfs-utils sources :

       emerge -f nfs-utils
       cp /usr/portage/distfiles/nfs-utils-1.0.10.tar.gz .

or use the given sources in the rpm “nfs-utils-1.0.9.tar.bz2”

I'm using the current Gentoo sources

       tar xfz nfs-utils-1.0.10.tar.gz
       cd nfs-utils-1.0.10
       patch -p1 < ../allPatches
       ./configure –disable-gss
       libtoolize --copy --force
       make && make install

By this way, the subdirectory "utils/mount" wont be compiled, but you'll need it.
So you have to do it by your own.

       cd utils/mount
       make && make install

If you have later trouble mounting nfs-shares,

try

cp /usr/sbin/mount.nfs* /sbin/

Harddrive requirements

CacheFS can only be used with a EXT3 filesystem
Also you need to tune your partition with

       tune2fs -o user_xattr /dev/hdaX

The Cache is declared in “/etc/cachefilesd.conf”
with the Option “dir”. The default is "/var/fscache"
So declare your partition in your "/etc/fstab"

       /dev/hdaX  /var/fscache  ext3  defaults 0 0 

Or try to find it at boot time. Therefore add a Label to the Cache

       tune2fs -L CacheFS /dev/hdaX

A script could look like this

       #!/bin/bash
       label()
       {
               #while [ $(/sbin/e2label $1) != "" ]; do
               while [ $1 ]; do
       
                       if [ $(/sbin/e2label $1) == "CacheFS" ]; then
                               /bin/mount $1 /var/fscache
               # CacheFS needs portmap can't be started at this time	
                               #/sbin/cachefilesd -s
                               #mount -t nfs -o fsc 10.0.8.17:/tftproot/gentoo /
                               echo "using CacheFS"
                       else
                               echo $1." is not a Cache"
                       fi	
                       shift
               done
       }
       label $(/bin/ls /dev/hda?)

Working with FS-Cache

localhost ~ # cachefilesd -s
About to bind cache
Bound cache
localhost ~ # mount -t nfs -o fsc ip:/source/dir /destination/dir


When the cache is bound properly, you'll find 2 directories under "/var/fscache"

Cache entries will be found under cache like cryptic "\@4a/I0nfs/\@41" and so on.


For Testing you can copy some files, like this

time cp source destination
time cp source /dev/null
time cp source destination

Stats using FS-Cache

One Big File

the File is about 350 MB

File is written 2 times on hd (cache init)

localhost ~ # time cp /nfsmount/oneBig.file .
  real    6m10.907s
  user    0m0.172s
  sys     0m12.161s

Copy from cache, file is read from hd

localhost ~ # time cp /nfsmount/oneBig.file /dev/null
  real    1m42.042s
  user    0m0.144s
  sys     0m52.467s

Copy from cache, read from and write to hd

localhost ~ # time cp /nfsmount/oneBig.file .
  real    3m33.246s
  user    0m0.176s
  sys     1m1.348s

many small files

1.8G Kernel sources


Normal NFS copy, no cache

localhost ~ # time cp -r /nfsmount/usr/src .
  real    33m47.274s
  user    0m6.192s
  sys     1m45.919s


Files are written 2 times on hd (cache init)

localhost ~ # time cp -r /nfsmount/usr/src/ .
  real    73m52.464s
  user    0m5.440s
  sys     3m55.663s

With cache, read from hd localhost ~ # time tar cf /dev/null /mnt/root/usr/src

tar: Removing leading `/' from member names
  real    18m59.711s
  user    0m2.640s
  sys     0m32.150s

Conclusion

The cache is good while reading files.
As soon as many small files are copied, causes the cache opposite. Particularly on slow harddrives. Solution: Put cache to another harddrive.

Retrieved from "http://www.gentoo-wiki.info/HOWTO_FS-Cache"

Last modified: Wed, 03 Sep 2008 09:29:00 +1000 Hits: 13,079

Created by NickStallman.net, Luxury Homes Australia
Real estate agents should list their apartments, townhouses and units in Australia.