-- ColinLeavettBrown - 2015-06-22

Cloud howto: running batch jobs with HTCondor/Cloud Scheduler

Obtain images - three methods:

  1. Use cloud provided default images (installed as public images by sysadmins) - easiest method.
  2. Download an existing image (see OpenStack image guide, section 2):
    • Select and image and copy its' URL
    • Use the horizon "Create Image" dialog and copied URL to upload the required image to your cloud account.
  3. Create image with virt-install (and cobbler):
    1. Hardware confguration
    2. On hypervisor node:
      • create and run /root/qemu/mk-cc-demo. This will create a virtual machine with a vnc console that will fail to boot. At this point, the vnc console can be closed.
      • Issue "virsh edit cc-demo" and copy the "MAC" address.
      • Modify /root/qemu/mk-cc-demo to include the copied MAC address.
      • "virsh destroy cc-demo"
      • "virsh undefine cc-demo"
    3. On cobbler server:
      • Create a public/private ssh key: ssh-keygen -f /root/.ssh/id_rsa_cc-demo
      • Create a /var/lib/cobbler/kickstarts/cc-demo.cfg suitable to your application needs.
        • Create an encrypted root password (make sure cipher matches passalgo) and insert in kickstart: mkpasswd -m sha-512 following [[][these instructions]].
        • Copy your new public key to the end of your kickstart file.
      • Create /root/cobbler/add-cc-demo specifying the kickstart file and the copied MAC address. Run the shell script and then "cobbler sync"
    4. On the hypervisor node:
      • Re-run the "mk-cc-demo" shell script. This will recreate the VM with a vnc console, but this time it will boot/install. * ( 5 minutes) *
      • After the reboot, prepare the VM for other network connections:
        • ssh login from the cobbler server: ssh -i ~/.ssh/id_rsa_cc-demo 192.168.1.11
        • /etc/sysconfig/network (no host or gateway, nozeroconf=yes)
        • /etc/sysconfig/network-scripts/ifcfg-eth0
        • /etc/inittab
        • halt
        • virsh edit cc-demo and change the network bridge to br3 (public network)
        • virsh start cc-demo (virsh start cc-demo; virt-viewer cc-demo)
      • Which public IP did it get?
        • reboot in single user mode
        • ifup eth0 and ifconfig
        • reboot
      • Log in through ssh and make any required customizations (see, OpenStack image guide, section 5):
        • yum -y update * (10 minutes) *
        • yum -y install epel-release
        • yum -y install cloud-init (may complain about not having python-six)
        • Batch jobs? Need to install HTCtcondor:
        • Authenticating condor server/workers with GSI?:
        • rm -fv /etc/udev/rules.d/70-persistent-net.rules
      • shutdown the VM
      • Use the horizon "Create Image" dialog to upload the image to your cloud account.

Customizing images interactively on the cloud:

  • Interractive instantiation:
    • Images->Actions->Launch
    • Provide name, flavor, key pair, network - then Launch
    • Instances->actions->associate floating IP
  • Modifying an image:
    • Login as root
    • Make changes
    • Snapshot:
      • Instances->Actions->Snapshot, provide snapshot name.

Running batch jobs:

  • On CC (litai08) demonstrate the necessity for grid credentials:
    • condor_q
    • ~/.globus
    • grid-proxy-ini
    • condor_q
  • Submit jobs:
    • Prepare batch control files:
    • cc-demo.sh - executable
    • cc-demo.job - Job Description Language (JDL) file
    • condor_submit
  • Monitoring jobs:
    • bin/watch: cloud_status -m, condor_status -m, condor_q
  • Retrieving job output & removing jobs:
    • condor_transfer
    • condor_q & condor_rm
    • check our spool directory
  • Custom VM contextualization:
    • yaml/custom.yaml
    • scp yaml/custom.yaml unix.uvic.ca:www/
    • +VMAMIGONFIG.
    • condor_submit
    • condor_release
    • The job log.
Topic attachments
I AttachmentSorted ascending History Action Size Date Who Comment
Unknown file formatEXT add-cc-demo r2 r1 manage 0.4 K 2015-09-29 - 18:30 UnknownUser  
Unknown file formatgz batch-demo.tar.gz r7 r6 r5 r4 r3 manage 1.8 K 2015-10-08 - 17:01 UnknownUser  
Unknown file formatcfg cc-demo.cfg r3 r2 r1 manage 2.5 K 2015-09-14 - 21:50 UnknownUser  
PNGpng CobblerInstallVM.png r2 r1 manage 85.8 K 2015-07-06 - 19:10 UnknownUser  
Unknown file formatrepo igtf.repo r2 r1 manage 0.2 K 2015-07-29 - 20:35 UnknownUser  
Unknown file formatEXT mk-cc-demo r2 r1 manage 0.5 K 2015-07-06 - 19:12 UnknownUser  
Edit | Attach | Watch | Print version | History: r27 | r23 < r22 < r21 < r20 | Backlinks | Raw View | More topic actions...
Topic revision: r21 - 2015-10-08 - crlb
 
  • Edit
  • Attach
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback