Difference: WiP1 (3 vs. 4)

Revision 42010-02-15 - crlb

Line: 1 to 1
 
META TOPICPARENT name="ColinLeavettBrown"
-- ColinLeavettBrown - 2010-02-11

Nimbus Install with just ws-core

Line: 6 to 6
 

Install Prerequisites

Changed:
<
<
Create the nimbus account on elephant head node (elephant01) and propagate to all nodes in the cluster.
>
>
Create the nimbus account on elephant head node (elephant01) with required sudo privileges.
 
[crlb@elephant01 ~]$ sudo adduser nimbus
Added:
>
>
[crlb@elephant01 ~]$ sudo visudo

Comment out the requiretty directive

#Defaults    requiretty

Add the following privileges

nimbus ALL=(root) NOPASSWD: /opt/nimbus/libexec/workspace-control/mount-alter.sh
nimbus ALL=(root) NOPASSWD: /opt/nimbus/libexec/workspace-control/dhcp-config.sh
nimbus ALL=(root) NOPASSWD: /opt/nimbus/libexec/workspace-control/xen-ebtables-config.sh

Save changes and propagate to every node in the cluster.

 [crlb@elephant01 ~]$ sudo /usr/local/sbin/usync
Line: 32 to 49
 

Install Apache Ant

Changed:
<
<
[nimbus@elephant11 nimbus]$ 
[crlb@elephant11 ant]$ cd /usr/local
>
>
[crlb@elephant11 ~]$ cd /usr/local
 [crlb@elephant11 local]$ sudo tar -xjvf ~nimbus/Downloads/apache-ant-1.8.0-bin.tar.bz2
Line: 234 to 251
 
Changed:
<
<

Installing Nimbus

>
>

Install Nimbus

 Unpack the nimbus package and run the install script.
Changed:
<
<
[nimbus@elephant11 ~]$
>
>
[nimbus@elephant11 ~]$ cd /tmp [nimbus@elephant11 tmp]$ tar -xvf ~/Downloads/nimbus-2.3.tar.gz [nimbus@elephant11 tmp]$ /tmp/nimbus-2.3/bin/all-build-and-install.sh [nimbus@elephant11 tmp]$ rm -rf /tmp/nimbus-2.3/ [nimbus@elephant11 tmp]$ rmdir hsperfdata_nimbus

Setting Up Worker Nodes

Setting up passwordless access to worker nodes

Nimbus needs to be able to ssh without a password from the head node to the worker nodes and vice versa. This is for sending commands back and forth. The following setup assumes you have the nimbus home directory mounted over NFS between the head node and the worker nodes. If you don't you'll just need to copy the .ssh directory on the head node to the nimbus home directory on each worker.

$ ssh-keygen 
Generating public/private rsa key pair.
Enter file in which to save the key (/home/nimbus/.ssh/id_rsa): 
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /home/nimbus/.ssh/id_rsa.
Your public key has been saved in /home/nimbus/.ssh/id_rsa.pub.
The key fingerprint is:
9c:75:52:2f:d9:bd:5a:05:43:ee:3f:b2:83:cc:f2:0b nimbus@canfardev.dao.nrc.ca
$ cd ~/.ssh
$ cp id_rsa.pub authorized_keys
$ chmod 600 authorized_keys

Now test it:

nimbus@canfardev $ ssh gildor
nimbus@ gildor $ ssh canfardev.dao.nrc.ca
nimbus@canfardev $

Great. It works. You may be asked to authorize a new host key. If so, just answer "yes".

Setting up Xen, ebtables and dhcpd

First, make sure Xen is installed. If it is, you should see something like the following when you run these commands:

# which xm
/usr/sbin/xm
# uname -r
2.6.18-128.1.1.el5xen
$ ps aux | grep xen
root        21  0.0  0.0      0     0 ?        S<   16:34   0:00 [xenwatch]
root        22  0.0  0.0      0     0 ?        S<   16:34   0:00 [xenbus]
root      2549  0.0  0.0   2188   956 ?        S    16:35   0:00 xenstored --pid-file /var/run/xenstore.pid
root      2554  0.0  0.1  12176  3924 ?        S    16:35   0:00 python /usr/sbin/xend start
root      2555  0.0  0.1  63484  4836 ?        Sl   16:35   0:00 python /usr/sbin/xend start
root      2557  0.0  0.0  12212   364 ?        Sl   16:35   0:00 xenconsoled --log none --timestamp none --log-dir /var/log/xen/console

If it's not installed, you can do so with:

# yum install xen kernel-xen
# chkconfig xend on

Then reboot.

You'll also need to install ebtables (not currently used) and dhcp. Do this by first enabling the DAG repository, then installing with yum:

# rpm -Uhv http://apt.sw.be/redhat/el5/en/i386/rpmforge/RPMS/rpmforge-release-0.3.6-1.el5.rf.i386.rpm
# yum install ebtables dhcp

Now, edit the dhcpd config file. Make sure it looks something like this:

 
Changed:
<
<
Get Nimbus from the Nimbus website. You'll need the Nimbus package.
>
>
# vim /etc/dhcpd.conf
# dhcpd.conf
#
# Configuration file for ISC dhcpd for workspaces
 
Changed:
<
<
$ wget http://workspace.globus.org/downloads/nimbus-TP2.2.tar.gz
$ tar xzf nimbus-TP2.2.tar.gz
>
>
################# ## GLOBAL OPTS ## #################

# Option definitions common or default to all supported networks

# Keep this: ddns-update-style none;

# Can be overriden in host entry: default-lease-time 120; max-lease-time 240;

############# ## SUBNETS ## #############

# Make an entry like this for each supported subnet. Otherwise, the DHCP # daemon will not listen for requests on the interface of that subnet.

subnet 172.21.0.0 netmask 255.255.0.0 { }

### DO NOT EDIT BELOW, the following entries are added and ### removed programmatically.

### DHCP-CONFIG-AUTOMATIC-BEGINS ###

Setting Up Control Agents

The Nimbus Control Agents are the binaries on the worker node that act on behalf of the head node. They need to be installed on each worker node.

If you've already set up the control agents on one node, you shouldn't need to do the following steps on the other nodes. Just make sure the install directory is NFS mounted.

First, make sure we have the install directory:

# ls /opt/nimbus
/opt/nimbus
 
Changed:
<
<
Installing Nimbus depends on Ant and Ant depends on the xml-commons-api.
>
>
Now do the install:
 
Changed:
<
<
# yum install ant ant-trax ant-apache-regexp ant-nodeps xml-commons-api perl-XML-Parser
>
>
# wget http://workspace.globus.org/downloads/nimbus-controls-TP2.2.tar.gz # tar xzf nimbus-controls-TP2.2.tar.gz # cd nimbus-controls-TP2.2/workspace-control # cp worksp.conf.example /opt/nimbus/worksp.conf # python install.py -i -c /opt/nimbus/worksp.conf -a nimbus -g nimbus
 
Changed:
<
<
Now we're ready to install Nimbus. There is an auto configuration that we can run that will help our installation. See the http://workspace.globus.org/vm/TP2.2/index.html for more details than are listed here.
>
>
The installer will ask you a bunch of questions. Answer them out to the best of your knowledge, and don't worry too much if you're not sure of the answers to some of the questions. Chances are though, you will just answer yes to all of them.

Adding Node to Nimbus Config

This should be done after you've already installed Nimbus on the head node. If you haven't done that yet, come back to this section.

 
Added:
>
>
Edit $GLOBUS_LOCATION/etc/nimbus/workspace-service/vmm-pools/canfardevpool to add the new node. Your file should look something like this:
 
Changed:
<
<
$ cd nimbus-TP2.2 $ sh ./bin/all-build-and-install.sh
>
>
#Some comments up here gildor 3072 guilin 3072
 
Added:
>
>
Your worker node should now be ready!
 Now run the auto-configuration program. Following is a transcript of running this program on canfardev:
$ $GLOBUS_LOCATION/share/nimbus-autoconfig/autoconfig.sh
Line: 796 to 942
 If you encounter an ebtables problem. You can try a patched version of ebtables. See This page for details.
Deleted:
<
<

Setting Up Worker Nodes

Setting up passwordless access to worker nodes

Nimbus needs to be able to ssh without a password from the head node to the worker nodes and vice versa. This is for sending commands back and forth. The following setup assumes you have the nimbus home directory mounted over NFS between the head node and the worker nodes. If you don't you'll just need to copy the .ssh directory on the head node to the nimbus home directory on each worker.

$ ssh-keygen 
Generating public/private rsa key pair.
Enter file in which to save the key (/home/nimbus/.ssh/id_rsa): 
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /home/nimbus/.ssh/id_rsa.
Your public key has been saved in /home/nimbus/.ssh/id_rsa.pub.
The key fingerprint is:
9c:75:52:2f:d9:bd:5a:05:43:ee:3f:b2:83:cc:f2:0b nimbus@canfardev.dao.nrc.ca
$ cd ~/.ssh
$ cp id_rsa.pub authorized_keys
$ chmod 600 authorized_keys

Now test it:

nimbus@canfardev $ ssh gildor
nimbus@ gildor $ ssh canfardev.dao.nrc.ca
nimbus@canfardev $

Great. It works. You may be asked to authorize a new host key. If so, just answer "yes".

Setting up Xen, ebtables and dhcpd

First, make sure Xen is installed. If it is, you should see something like the following when you run these commands:

# which xm
/usr/sbin/xm
# uname -r
2.6.18-128.1.1.el5xen
$ ps aux | grep xen
root        21  0.0  0.0      0     0 ?        S<   16:34   0:00 [xenwatch]
root        22  0.0  0.0      0     0 ?        S<   16:34   0:00 [xenbus]
root      2549  0.0  0.0   2188   956 ?        S    16:35   0:00 xenstored --pid-file /var/run/xenstore.pid
root      2554  0.0  0.1  12176  3924 ?        S    16:35   0:00 python /usr/sbin/xend start
root      2555  0.0  0.1  63484  4836 ?        Sl   16:35   0:00 python /usr/sbin/xend start
root      2557  0.0  0.0  12212   364 ?        Sl   16:35   0:00 xenconsoled --log none --timestamp none --log-dir /var/log/xen/console

If it's not installed, you can do so with:

# yum install xen kernel-xen
# chkconfig xend on

Then reboot.

You'll also need to install ebtables (not currently used) and dhcp. Do this by first enabling the DAG repository, then installing with yum:

# rpm -Uhv http://apt.sw.be/redhat/el5/en/i386/rpmforge/RPMS/rpmforge-release-0.3.6-1.el5.rf.i386.rpm
# yum install ebtables dhcp

Now, edit the dhcpd config file. Make sure it looks something like this:

# vim /etc/dhcpd.conf
# dhcpd.conf
#
# Configuration file for ISC dhcpd for workspaces


#################
## GLOBAL OPTS ##
#################

# Option definitions common or default to all supported networks

# Keep this:
ddns-update-style none;

# Can be overriden in host entry:
default-lease-time 120;
max-lease-time 240;


#############
## SUBNETS ##
#############

# Make an entry like this for each supported subnet.  Otherwise, the DHCP
# daemon will not listen for requests on the interface of that subnet.

subnet 172.21.0.0 netmask 255.255.0.0 {
}

### DO NOT EDIT BELOW, the following entries are added and 
### removed programmatically.

### DHCP-CONFIG-AUTOMATIC-BEGINS ###


Setting up Sudo

You need a few rules for the nimbus user to be able to run the xm scripts it needs:

Add the following rules to sudoers:

nimbus ALL=(root) NOPASSWD: /opt/nimbus/bin/mount-alter.sh
nimbus ALL=(root) NOPASSWD: /opt/nimbus/bin/dhcp-config.sh
nimbus ALL=(root) NOPASSWD: /usr/sbin/xm
nimbus ALL=(root) NOPASSWD: /usr/sbin/xend

And set requiretty to false in sudoers.

Now that we've set up our pre-requisites, we can install the worker node tools.

Setting Up Control Agents

The Nimbus Control Agents are the binaries on the worker node that act on behalf of the head node. They need to be installed on each worker node.

If you've already set up the control agents on one node, you shouldn't need to do the following steps on the other nodes. Just make sure the install directory is NFS mounted.

First, make sure we have the install directory:

# ls /opt/nimbus
/opt/nimbus

Now do the install:

# wget http://workspace.globus.org/downloads/nimbus-controls-TP2.2.tar.gz
# tar xzf nimbus-controls-TP2.2.tar.gz
# cd nimbus-controls-TP2.2/workspace-control
# cp worksp.conf.example /opt/nimbus/worksp.conf
# python install.py -i -c /opt/nimbus/worksp.conf -a nimbus -g nimbus

The installer will ask you a bunch of questions. Answer them out to the best of your knowledge, and don't worry too much if you're not sure of the answers to some of the questions. Chances are though, you will just answer yes to all of them.

Adding Node to Nimbus Config

This should be done after you've already installed Nimbus on the head node. If you haven't done that yet, come back to this section.

Edit $GLOBUS_LOCATION/etc/nimbus/workspace-service/vmm-pools/canfardevpool to add the new node. Your file should look something like this:

#Some comments up here
gildor 3072
guilin 3072

Your worker node should now be ready!

 

-- PatrickArmstrong - 16 Jul 2009

 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback