create new tag
view all tags


Steps taken to create the SL6.4 VM image for the clouds in the ATLAS IAAS queue.


Some backgrounds information:
  • Inherited from BuildingATLASCernVM27
  • Using Scientific Linux 6.4, link to the download page.
  • Create a 20GB VM image, some helpful instructions are here


In the remainder of this document, the following formatting convention is used to differentiate terminal commands from file content

This background colour denotes terminal input

This background colour denotes file content

Getting Started

The vanilla Scientific Linux 6.4 VM I've created is available here. Use the following XML on a qemu/kvm hypervisor to run the image:

<domain type='kvm'>
  <memory unit='KiB'>1048576</memory>
  <currentMemory unit='KiB'>1048576</currentMemory>
  <vcpu placement='static'>1</vcpu>
    <type arch='x86_64' machine='rhel6.4.0'>hvm</type>
    <boot dev='hd'/>
  <clock offset='utc'/>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='none'/>
      <source file='/opt/qemu/atlas-node-sl64.img'/>
      <target dev='vda' bus='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    <controller type='usb' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
    <interface type='bridge'>
      <mac address='fa:16:3e:7a:a3:a2'/>
      <source bridge='br0'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    <serial type='pty'>
      <target port='0'/>
    <console type='pty'>
      <target type='serial' port='0'/>
    <input type='tablet' bus='usb'/>
    <input type='mouse' bus='ps2'/>
    <graphics type='vnc' port='-1' autoport='yes'/>
      <model type='cirrus' vram='9216' heads='1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    <memballoon model='virtio'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>

Edit the line appropriately. Then launch the VM via:

sudo virsh create atlas-node-sl64.xml

Log into the VM and follow the preperation steps. Probably a good idea to update the VM before proceeding.

Set up contexthelper

The contexthelper marries openstack-style user data in the metadata server with nimbus style XML. Set up the places context helper places files

mkdir -p /etc/grid-security/certificates
touch /etc/grid-security/hostkey.pem
chmod 600 /etc/grid-security/hostkey.pem
touch /var/lib/cloud_type
mkdir /etc/condor
touch /etc/condor/central_manager

Install context helper and context service:

yum install wget
wget https://raw.github.com/hep-gc/cloud-scheduler/dev/scripts/ec2contexthelper/context \
          https://raw.github.com/hep-gc/cloud-scheduler/dev/scripts/ec2contexthelper/contexthelper \
python setup.py install
chmod +x /etc/init.d/context
chkconfig context on

Set up puppet

Install puppet as outlined by puppetlabs. Ensure that puppet is configured to connect your puppet master. The puppet.conf is here:

    # The Puppet log directory.
    # The default value is '$vardir/log'.
    logdir = /var/log/puppet

    # Where Puppet PID files are kept.
    # The default value is '$vardir/run'.
    rundir = /var/run/puppet

    # Where SSL certificates are kept.
    # The default value is '$confdir/ssl'.
    ssldir = $vardir/ssl

    # The file in which puppetd stores a list of the classes
    # associated with the retrieved configuratiion.  Can be loaded in
    # the separate ``puppet`` executable using the ``--loadclasses``
    # option.
    # The default value is '$confdir/classes.txt'.
    classfile = $vardir/classes.txt

    # Where puppetd caches the local configuration.  An
    # extension indicating the cache format is added automatically.
    # The default value is '$confdir/localconfig'.
    localconfig = $vardir/localconfig

    server = puppet-master.heprc.uvic.ca

    certname = elephant214.heprc.uvic.ca

The certname ensures that a specific certificate generated once will be used to authenticate all worker nodes to the puppet master, rather than creating a new cert for each node. Creating a new cert for every node has the issue that if a new nodes is booted to replace one of the same FQDN (as is bound to happen on niumbus) the certificate authentication would fail. So use a single cert for all workers.

Generate a cert or use an existing one and ensure it works by running

puppet agent --test --noop

Let cron run puppet once on boot instead of using it's agent and connecting every 30 minutes. Turn off the puppet agent

chkconfig puppet off

Use the crontab:

@reboot puppet agent --onetime


LVM setup

It would be very nice to have a separate filesystem for /var/lib/cvmfs and possibly other utilities in the future. LVM could be very useful here, try setting up the next image on LVM.

-- FrankBerghaus - 2014-01-29

Edit | Attach | Watch | Print version | History: r4 < r3 < r2 < r1 | Backlinks | Raw View | More topic actions
Topic revision: r4 - 2014-01-30 - frank
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback