Tags:
create new tag
view all tags

Installing Lustre and Xen on Scientific Linux 5

In order to have the advantages of both Xen and Lustre on the same box it necessary to do little bit of work at the moment. Since both projects require kernel patches you need to have a kernel with both. There are several different approaches for doing this. I present only the one I followed that worked for me.

We are targeting Scientific Linux 5.X kernels with the latest release of Lustre and the version of Xen present in SL5.

The basic steps are as follows:

  1. Download the Luster patched RHEL 5 kenel from Oracle or (sun) in the form of src rpm.
  2. Configure and build the source RPM into regular rpms containing both the Lustre and Xen patches.
  3. Install the Luste Xen Kernel from rpm.
  4. Download the source tarball for Lustre that matches the kernel you have just built.
  5. Run the configure script for the lustre sources pointing at the kernel you have just installed
  6. Build the Lustre kernel modules, ldiskfs kernel modules, and lustre rpms.
  7. Install the rpms.

Step by Step

Most of the follow was derived from reading this: http://wiki.centos.org/HowTos/Custom_Kernel

Create a place to build rpms not as the root user and install the downloaded source rpms:

$ cd ~
$ mkdir -p rpmbuild/{BUILD,RPMS,SOURCES,SPECS,SRPMS}
$ echo '%_topdir /hepuser/igable/rpmbuild' >./.rpmmacros
$ rpm -i kernel-lustre-2.6.18-128.7.1.el5_lustre.1.8.1.1.src.rpm

Build the source RPMS into the complete set of kernel RPMS:

$ rpmbuild -bb --without kabichk --target=x86_64 kernel-2.6.spec

As root install the RPMS:

# rpm -ivh kernel-lustre-xen-2.6.18-128.7.1.el5_lustre.1.8.1.1.x86_64.rpm
# rpm -ivh kernel-lustre-xen-devel-2.6.18-128.7.1.el5_lustre.1.8.1.1.x86_64.rpm

Clean up you build are in preparation for building RPMS for lustre from the Lustre Source:

$ mv rpmbuild rpmbuild-2.6.18.128_lustre_xen
$ mkdir -p rpmbuild/{BUILD,RPMS,SOURCES,SPECS,SRPMS}

Untar luster and configure it pointing to the correct kernel source:

$ cd lustre-1.8.1.1
$ ./configure --with-linux=/usr/src/kernels/2.6.18-128.7.1.el5_lustre.1.8.1.1-xen-x86_64

Make and install the lustre RPMS:

$ make rpms
$ rpm -ivh lustre-modules-1.8.1.1-2.6.18_128.7.1.el5_lustre.1.8.1.1xen_200912081130.x86_64.rpm
$ rpm -ivh lustre-ldiskfs-3.0.9-2.6.18_128.7.1.el5_lustre.1.8.1.1xen_200912081131.x86_64.rpm
$ rpm -ivh lustre-1.8.1.1-2.6.18_128.7.1.el5_lustre.1.8.1.1xen_200912081130.x86_64.rpm

Original History Dump

$ cd ~
$ mkdir -p rpmbuild/{BUILD,RPMS,SOURCES,SPECS,SRPMS}
$ rpm -i kernel-lustre-2.6.18-128.7.1.el5_lustre.1.8.1.1.src.rpm
$ rpmbuild -bb --without kabichk --target=x86_64 kernel-2.6.spec
# rpm -ivh kernel-lustre-xen-devel-2.6.18-128.7.1.el5_lustre.1.8.1.1.x86_64.rpm
# rpm -ivh kernel-lustre-xen-devel-2.6.18-128.7.1.el5_lustre.1.8.1.1.x86_64.rpm

$ mv rpmbuild rpmbuild-2.6.18.128_lustre_xen
$ mkdir -p rpmbuild/{BUILD,RPMS,SOURCES,SPECS,SRPMS}
$ cd lustre-1.8.1.1
$ ./configure --with-linux=/usr/src/kernels/2.6.18-128.7.1.el5_lustre.1.8.1.1-xen-x86_64
$ make rpms
$ rpm -ivh lustre-modules-1.8.1.1-2.6.18_128.7.1.el5_lustre.1.8.1.1xen_200912081130.x86_64.rpm
$ rpm -ivh lustre-ldiskfs-3.0.9-2.6.18_128.7.1.el5_lustre.1.8.1.1xen_200912081131.x86_64.rpm
$ rpm -ivh lustre-1.8.1.1-2.6.18_128.7.1.el5_lustre.1.8.1.1xen_200912081130.x86_64.rpm
$ cat /proc/fs/lustre/devices

Garbage

solution is to add

http://ftp1.scientificlinux.org/linux/scientific/5x/SRPMS/vendor/

e2fsprogs-1.39-23.el5.x86_64

warning about ldiskfs stuff WARNING: /lib/modules/2.6.18-128.7.1.el5_lustre.1.8.1.1xen/kernel/fs/lustre/fsfilt_ldiskfs.ko needs unknown symbol ldiskfs_bread

Edit | Attach | Watch | Print version | History: r10 < r9 < r8 < r7 < r6 | Backlinks | Raw View | More topic actions
Topic revision: r10 - 2010-02-03 - crlb
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback