Difference: CCworkPlan (1 vs. 2)

Revision 22015-01-26 - crlb

Line: 1 to 1
 
META TOPICPARENT name="ComputeCanadaCloudProject"
-- ColinLeavettBrown - 2015-01-26

Compute Canada Cloud Project Work Plan

  • Cloud Scheduler
Changed:
<
<
    • Develop new Virtual Machine, sl7.0, latest HTCondor and CS releases. One or more instances to be run on CC East or West.
>
>
    • Batch through HTCondor/CS, all resources dedicatd to CS to achieve 90% (see FS below).
    • Interractive through Horizon on deicated resources.
    • Systems to develop new Virtual Machine with latest HTCondor and CS releases. One or more instances to be run on CC East or West.
 
    • CS/HTCondor VM must run with public/external NIC.
Added:
>
>
    • End user doc for VM preparation, JDL creation, Job submission, general work flow, etc.
 
    • Achieve 90%+ utilization on cloud:
Changed:
<
<
      • Develop script to monitor utilization on attached clouds and adjust CS resource allocation:
        • suggestion: leave 5% margin for interractive VMs, ie. allocate 95% resources to CS.
        • Script to have configurable thresholds, eg leave 5% (max) free, when less than 2% (min) free up another 4% (adjustment). Different values for each cloud?
        • Effect by adjusting /etc/cloudscheduler/cloud_resources.conf + quickrestart
        • May require some enhancements to CS to prioritize VM shutdowns and avoid ATLAS blacklisting.
        • Percentage utilization of what? (cores, memory, slots, etc.)
        • How do we get the utilization numbers? Suggest Nova API.
>
>
    • Fairshare scheduling algorithm (2 weeks history).
 
    • Time line:
      • 27 January 2015 - agree on strategy.

Revision 12015-01-26 - crlb

Line: 1 to 1
Added:
>
>
META TOPICPARENT name="ComputeCanadaCloudProject"
-- ColinLeavettBrown - 2015-01-26

Compute Canada Cloud Project Work Plan

  • Cloud Scheduler
    • Develop new Virtual Machine, sl7.0, latest HTCondor and CS releases. One or more instances to be run on CC East or West.
    • CS/HTCondor VM must run with public/external NIC.
    • Achieve 90%+ utilization on cloud:
      • Develop script to monitor utilization on attached clouds and adjust CS resource allocation:
        • suggestion: leave 5% margin for interractive VMs, ie. allocate 95% resources to CS.
        • Script to have configurable thresholds, eg leave 5% (max) free, when less than 2% (min) free up another 4% (adjustment). Different values for each cloud?
        • Effect by adjusting /etc/cloudscheduler/cloud_resources.conf + quickrestart
        • May require some enhancements to CS to prioritize VM shutdowns and avoid ATLAS blacklisting.
        • Percentage utilization of what? (cores, memory, slots, etc.)
        • How do we get the utilization numbers? Suggest Nova API.
    • Time line:
      • 27 January 2015 - agree on strategy.

  • Glint
    • Glint server deployments in CC East and West, each using the local OpenStack services (keystone, glance).
    • Parallel deployments in both regions, ie. both modified and standard horizon dashboards available from both CC regions.
    • Modified dashboards to run in python virtual environments.
    • Initial glint dashboard will be based on Havana.
    • Support for later versions of the dashboard to be aded.
    • Time line:
      • 27 January 2015 - agree on strategy.

  • Shoal
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback