[讨论]nfsv4_pnfs_using_spnfsd_and_nfsv4_as_back-end

Web、Mail、Ftp、DNS、Proxy、VPN、Samba、LDAP 等基础网络服务
回复
yuzl2005
帖子: 1
注册时间: 2008-07-25 14:31
送出感谢: 0
接收感谢: 0

[讨论]nfsv4_pnfs_using_spnfsd_and_nfsv4_as_back-end

#1

帖子 yuzl2005 » 2008-07-25 14:39

有没有人试试?NFSv4.1中的pNFS中的一种简单实现。
https://www.os3.nl/2007-2008/students/thijs_stuurman/nfsv4_pnfs_using_spnfsd_and_nfsv4_as_back-end

spNFS (Simple Parallel Network File System, by NetApp) is a Linux pNFS server implementation from NetApp who use it to develop their pNFS client. Currently it is the only working Linux implementation using the latest code. More information can be found in their presentation: dmuntz-spnfs-cthon08.pdf

The setup in a virtualized environment (Xen) looks as follows:

The client uses pNFS with a file layout driver to mount an export from the MDS (Metadata server). The MDS uses a single physical connection to mount NFSv4 exports on the DSs (Data servers). When data from the client is written to the export, pNFS is used to stripe the data over the DSs without tranferring data through the MDS. Retrieving data works the same way. For example of a file of 230M, each DS in this setup stores 115M. Retrieving data uses both DSs network connections as well to retrieve the data.

The picture is just an abstract view of the situation, it could be that the pNFS client has for example a single very fast physical network connection such as 10G Ethernet which a regular NFS setup is unable to fully utilize. With pNFS the data comes from multiple servers at once.

Some personal notes from setting up spNFS:

% spNFS:

代码: 全选

      apt-get install cogito git-core
      git clone git://linux-nfs.org/~dmuntz/spnfs.git
      git clone git://linux-nfs.org/~dmuntz/nfs-utils.git


spNFS:

代码: 全选

      make menuconfig
      -.config
              CONFIG_NETWORK_FILESYSTEMS=y
              CONFIG_NFS_FS=y
              CONFIG_NFSD=y
              CONFIG_NFS_V4_1=y
              CONFIG_NFSD_V4_1=y
              CONFIG_PNFS=y
              CONFIG_PNFSD=y
              CONFIG_SPNFS=y


代码: 全选

      make
      make modules
      make modules_install
      make install


1.
→ /boot/

mkinitrd -o mkinitrd -o initrd.img-2.6.25-pnfs 2.6.25-pnfs

代码: 全选

name    /lib/modules/2.6.25-pnfs ...
    edit /boot/grub/menu.lst


*
GRUB is installed. To automatically switch to new kernels, point your
*
default entry in menu.lst to /boot/vmlinuz-2.6.25-pnfs

%nfs-utils:

代码: 全选

apt-get install automake libtool pkg-config
export PKG_CONFIG_PATH=/usr/lib/pkgconfig
sh autogen.sh
./configure --disable-gss (avoid dependency problems, not required for this setup anyway)
make

*do NOT install

%%% Setting things up:

On MDS:

代码: 全选

cp /root/sPNFS/nfs-utils/utils/spnfsd/spnfsd.conf /etc/


Reconfigure for two DSs:

代码: 全选

"""
[General]
Verbosity = 1
Stripe-size = 8192
Dense-striping = 0
Pipefs-Directory = /var/lib/nfs/rpc_pipefs
DS-Mount-Directory = /spnfs
[DataServers]
NumDS = 2
DS1_IP = 145.100.106.107
DS1_PORT = 2049
DS1_ROOT = /pnfs
DS1_ID = 1
DS2_IP = 145.100.106.108
DS2_PORT = 2049
DS2_ROOT = /pnfs
DS2_ID = 2
"""

代码: 全选

MD:  mkdir -m 1777 /spnfs
DSs: mkdir -m 1777 /pnfs
     mkdir /pnfs/145.100.106.107
     mkdir /pnfs/145.100.106.108


On the MD, the /spnfs will contain mount points (IP address as dir) to all DSs. The root is /pnfs thus on the DSs we export the /:

etc/exports

代码: 全选

 / *(rw,fsid=0,insecure,no_root_squash,sync,no_subtree_check)


DSs mounts on MDS:

etc/fstab

代码: 全选

 # spNFS DS mounts
       145.100.106.107:/pnfs /spnfs/145.100.106.107 nfs    defaults        1 2
       145.100.106.108:/pnfs /spnfs/145.100.106.108 nfs    defaults        1 2

/etc/exports

代码: 全选

 / *(rw,fsid=0,insecure,no_root_squash,sync,no_subtree_check)


On MDS and DSs:

代码: 全选

/usr/sbin/rpc.statd
/usr/sbin/rpc.mountd
/usr/sbin/rpc.nfsd 8
exportfs -vr


on MDS, start spnfsd:

代码: 全选

"""
debian02:/usr/sbin# spnfsd
Verbosity: 1
Stripe size: 8192
Dense striping: 0
Number of data servers: 2
DS1 IP: 145.100.106.107
DS1 PORT: 2049
DS1 ROOT: /pnfs
DS1 DSID: 1
DS2 IP: 145.100.106.108
DS2 PORT: 2049
DS2 ROOT: /pnfs
DS2 DSID: 2
"""


** be sure to mount the /pnfs/145….. 's on the MDS

on client, load layout driver:

代码: 全选

modprobe nfslayoutdriver


mount:

代码: 全选

mount -t nfs4 145.100.106.106:/ /mnt
附件
spnfs_diagram.jpg
a virtualized environment (Xen)
回复

回到 “服务器基础应用”