-- ColinLeavettBrown - 2015-06-22

Cloud howto: running batch jobs with HTCondor/Cloud Scheduler

  • Obtain images - three methods:
    1. Use cloud provided default images (installed as public images by sysadmins) - easiest method.
    2. Download an existing image (see OpenStack image guide, section 2):
      • Select and image and copy its' URL
      • Use the horizon "Create Image" dialog and copied URL to upload the required image to your cloud account.
    3. Create image with virt-install (and cobbler):
      1. Hardware confguration
      2. On hypervisor node, create and run /root/qemu/mk-cc-demo. This will create a virtual machine with a vnc console that will fail to boot. At this point, the vnc console can be closed.
      3. Issue "virsh edit cc-demo" and copy the "MAC" address.
      4. On cobbler server:
      5. On the hupervisor node:
        • Modify /root/qemu/mk-cc-demo to include the copied MAC address.
        • "virsh destroy cc-demo"
        • "virsh undefine cc-demo"
        • Run the modified "mk-cc-demo" shell script. This will recreate the VM with a vnc console, but this time it will boot/install.
        • After the reboot, log in and make any required customizations (see, OpenStack image guide, section 5).
        • Remove any UDEV network rules that exist (ie. "rm -f /etc/udev/rules.d/70-persistent-net.rules) and halt the VM.
        • Convert the image to qcow2 format: "qemu-img convert -O qcow2 cc-demo.img cc-demo.qcow2".
        • Use the horizon "Create Image" dialog to upload the image to your cloud account.

  • Customizing images:
    • Interractive instantiation:
    • login as root and modify image
    • Snapshot

  • Running batch jobs:
    • Prepare executable
    • Prepare JDL
    • Prepare data souces and repositories
    • Submit jobs
    • Monitoring jobs
    • Retrieving job output
Topic attachments
I Attachment History Action Size Date Who Comment
Unknown file formatcfg 0-cc-demo.cfg r2 r1 manage 3.5 K 2015-06-25 - 15:54 UnknownUser  
Unknown file formatEXT 0-mk-cc-demo r2 r1 manage 0.5 K 2015-06-25 - 15:45 UnknownUser  
Unknown file formatEXT 1-mk-cc-demo r2 r1 manage 0.5 K 2015-06-25 - 15:45 UnknownUser  
PNGpng CobblerInstallVM.png r1 manage 84.6 K 2015-06-24 - 21:45 UnknownUser  
Unix shell scriptsh add-cc-demo.sh r1 manage 0.4 K 2015-06-25 - 15:42 UnknownUser  
Edit | Attach | Watch | Print version | History: r27 | r6 < r5 < r4 < r3 | Backlinks | Raw View | More topic actions...
Topic revision: r4 - 2015-06-25 - crlb
  • Edit
  • Attach
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2021 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback