(r24) CloudHowToForCC < HEPrc

Tags: view all tags
-- Main.ColinLeavettBrown - 2015-06-22

---+ Cloud howto: running batch jobs with HTCondor/Cloud Scheduler

---+++ Obtain images - three methods:
   1 Use cloud provided default images (installed as public images by sysadmins) - easiest method.
   1 Download an existing image (see [[http://docs.openstack.org/image-guide/content/ch_obtaining_images.html][OpenStack image guide, section 2]]):
      * Select and image and copy its' URL
      * Use the horizon "Create Image" dialog and copied URL to upload the required image to your cloud account.
   1 Create image with virt-install (and cobbler):
      1 [[%ATTACHURL%/CobblerInstallVM.png][Hardware confguration]]
      1 On hypervisor node:
         * create and run [[%ATTACHURL%/mk-cc-demo][/root/qemu/mk-cc-demo]]. This will create a virtual machine with a vnc console that will fail to boot. At this point, the vnc console can be closed.
         * Issue "virsh edit cc-demo" and copy the "MAC" address.
         * Modify  /root/qemu/mk-cc-demo to include the copied MAC address.
         * "virsh destroy cc-demo"
         * "virsh undefine cc-demo"
      1 On cobbler server:
         * Create a public/private ssh key: *ssh-keygen -f /root/.ssh/id_rsa_cc-demo*
         * Create a [[%ATTACHURL%/cc-demo.cfg][/var/lib/cobbler/kickstarts/cc-demo.cfg]] suitable to your application needs. 
            * Create an encrypted root password (make sure cipher matches passalgo) and insert in kickstart: mkpasswd -m sha-512 following [[][these instructions]].
            * Copy your new public key to the end of your kickstart file.
         * Create [[%ATTACHURL%/add-cc-demo][/root/cobbler/add-cc-demo]] specifying the kickstart file and the copied MAC address. Run the shell script and then "cobbler sync"
      1 On the hypervisor node:
         * Re-run the "mk-cc-demo" shell script. This will recreate the VM with a vnc console, but this time it will boot/install. *** ( 5 minutes) ***
         * After the reboot, prepare the VM for other network connections:
            * ssh login from the cobbler server: *ssh -i ~/.ssh/id_rsa_cc-demo 192.168.1.11*
            * /etc/sysconfig/network (no host or gateway, nozeroconf=yes)
            * /etc/sysconfig/network-scripts/ifcfg-eth0
            * /etc/inittab
            * halt
            * *virsh edit cc-demo* and change the network bridge to br3 (public network)
            * virsh start cc-demo (*virsh start cc-demo; virt-viewer cc-demo*)
         * Which public IP did it get?
            * reboot in single user mode
            * ifup eth0 and ifconfig
            * reboot  
         * Log in through ssh and make any required customizations (see, [[http://docs.openstack.org/image-guide/content/ch_creating_images_manually.html][OpenStack image guide, section 5]]):
            * *yum -y update* *** (5 minutes) ***
            * *yum -y install epel-release*
            * *yum -y install cloud-init* (may complain about not having [[ftp://ftp.pbone.net/mirror/ftp.pramberger.at/systems/linux/contrib/rhel6/i386/python-six-1.9.0-1.el6.pp.noarch.rpm][python-six]])
            * Batch jobs? Need to install HTCtcondor:
               * *(cd /etc/yum.repos.d; wget http://research.cs.wisc.edu/htcondor/yum/repo.d/htcondor-stable-rhel6.repo)*
               * *yum -y install condor-8.2.7**
               * Modify the daemon list in /etc/condor/condor_config to include only the MASTER and STARTD daemons.
            * Authenticating condor server/workers with GSI?:
               * *(cd /etc/yum.repos.d; wget %ATTACHURL%/igtf.repo)*
               * *yum -y install ca_policy_igtf-classic*
            * rm -fv /etc/udev/rules.d/70-persistent-net.rules
         * shutdown the VM
         *  Use the horizon "Create Image" dialog to upload the image to your cloud account.

---+++ Customizing images interactively on the cloud:
   * Interractive instantiation:
      * Images->Actions->Launch
      * Provide name, flavor, key pair, network - then Launch
      * Instances->actions->associate floating IP
   * Modifying an image:
      * Login as root
      * Make changes
      * Snapshot:
         * Instances->Actions->Snapshot, provide snapshot name.

---+++ Running batch jobs:
   * !OpenStack security groups: 9614, 9618, 20000-25000, and 40000-50000, TCP ports open.
   * On CC (litai08) demonstrate the necessity for grid credentials:
      * condor_q
      * ~/.globus
      * grid-proxy-ini
      * condor_q
   * Submit jobs: 
      * Prepare [[%ATTACHURL%/batch-demo.tar.gz][batch control files]]:
      * cc-demo.sh - executable
      * cc-demo.job - Job Description Language (JDL) file
      * condor_submit
   * Monitoring jobs: 
      * bin/watch: cloud_status -m, condor_status -m, condor_q
   * Retrieving job output & removing jobs:
      * condor_transfer
      * condor_q & condor_rm 
      * check our spool directory
   * Custom VM contextualization:
      * yaml/custom.yaml
      * scp yaml/custom.yaml unix.uvic.ca:www/
      * +VMAMIGONFIG.
      * condor_submit
      * condor_release
      * The job log.
Attachments
Topic attachments
I	Attachment	History	Action	Size	Date	Who
png	CobblerInstallVM.png	r2 r1	manage	85.8 K	2015-07-06 - 19:10	UnknownUser
EXT	add-cc-demo	r3 r2 r1	manage	0.4 K	2015-10-15 - 16:11	UnknownUser
gz	batch-demo.tar.gz	r10 r9 r8 r7 r6	manage	1.8 K	2015-10-15 - 16:43	UnknownUser
cfg	cc-demo.cfg	r4 r3 r2 r1	manage	2.5 K	2015-10-15 - 16:07	UnknownUser
repo	igtf.repo	r2 r1	manage	0.2 K	2015-07-29 - 20:35	UnknownUser
EXT	mk-cc-demo	r2 r1	manage	0.5 K	2015-07-06 - 19:12	UnknownUser
Topic revision: r24 - 2015-10-16 - crlb
~~Edit~~
~~Attach~~