Difference: CloudScheduler (1 vs. 20)

Revision 202015-12-16 - rsobie

Line: 1 to 1
 
META TOPICPARENT name="RandallSobie"
Line: 112 to 112
 condor_release -const 'JobStatus==1'
Changed:
<
<
To edit the RAM size requirement for Belle2 jobs:
>
>
To edit the RAM size or disk space requirement for Belle2 jobs:
 
vi /etc/condor/config.d/partition
Deleted:
<
<
# Default job memory request in Mbytes
 JOB_DEFAULT_REQUESTMEMORY = 2000
Added:
>
>
JOB_DEFAULT_REQUESTDISK = 4000000
 

Also check the logs in

Revision 192015-12-14 - rsobie

Line: 1 to 1
 
META TOPICPARENT name="RandallSobie"
Line: 112 to 112
 condor_release -const 'JobStatus==1'
Added:
>
>
To edit the RAM size requirement for Belle2 jobs:
vi /etc/condor/config.d/partition
# Default job memory request in Mbytes
JOB_DEFAULT_REQUESTMEMORY = 2000
  Also check the logs in

Revision 182015-11-30 - rsobie

Line: 1 to 1
 
META TOPICPARENT name="RandallSobie"
Line: 39 to 39
 cloud_admin -k -c cernopenstack -n i-0001ab95
Added:
>
>
Adjusting the quota using the following command will cause the extra VM to be moved into a separate list and retired.
cloud_admin -c mouse -v 1

 List the cloud aliases and the active clouds within those aliases (for condor.heprc.uvic.ca useful for the IAAS queue)
cloud_admin -y

Revision 172015-11-10 - rsobie

Line: 1 to 1
 
META TOPICPARENT name="RandallSobie"
Line: 113 to 113
 /opt/dirac/runit/WorkloadManagement/SiteDirectorUVic/log/current
Added:
>
>
To find out where the job is running
grep "match (" /var/log/condor/SchedLog
 To get the plot of the EC2 spot price (c3.4xlarge)
https://us-west-2.console.aws.amazon.com/ec2/v2/home?region=us-west-2#SpotInstances:

Revision 162015-11-02 - rsobie

Line: 1 to 1
 
META TOPICPARENT name="RandallSobie"
Line: 58 to 58
 vi /etc/cloudscheduler/cloud_scheduler.conf
Added:
>
>
When adding or removing a cloud, one needs to edit the following file:
vi /etc/cloudscheduler/cloud_aliases.json
  To retire all the VMs in a cloud (they must be registered with HTCondor)

Revision 152015-06-18 - rptaylor

Line: 1 to 1
 
META TOPICPARENT name="RandallSobie"
Added:
>
>

See also CloudSchedulerAdminGuide.

 

CloudScheduler commands

List the status of the Belle2 Cloud Schdeduler (bellecs.heprc.uvic.ca and condor.heprc.uvic.ca)

Revision 142015-03-16 - rsobie

Line: 1 to 1
 
META TOPICPARENT name="RandallSobie"

CloudScheduler commands

Line: 77 to 77
  JobPoller Thread(45): 21 MachinePoller Thread(45): 35
Changed:
<
<
If the numbers are large, then one has to
>
>
If the numbers are large, then one has to kill the processes:
ps aux | grep cloud_scheduler
kill -9 <CS process #>
service cloud_scheduler start
  To see what VMs have attached to condor

Revision 132015-03-16 - rsobie

Line: 1 to 1
 
META TOPICPARENT name="RandallSobie"

CloudScheduler commands

Line: 66 to 66
 cloud_admin -k -c -a
Added:
>
>
List the status of the CS threads
cloud_status -x

Thread Heart beat times:
   Scheduler Thread(45): 23
   Cleanup Thread(90): 74
   VMPoller Thread(155): 49
   JobPoller Thread(45): 21
   MachinePoller Thread(45): 35
If the numbers are large, then one has to
 To see what VMs have attached to condor
condor_status -m

Revision 122015-02-25 - rsobie

Line: 1 to 1
 
META TOPICPARENT name="RandallSobie"

CloudScheduler commands

Line: 82 to 82
 
/var/log/cloudscheduler.log 
/var/log/condor/MatchLog
Added:
>
>
/var/log/condor/ShadowLog
 /opt/dirac/runit/WorkloadManagement/SiteDirectorUVic/log/current

Revision 112015-02-23 - rsobie

Line: 1 to 1
 
META TOPICPARENT name="RandallSobie"

CloudScheduler commands

Line: 35 to 35
 cloud_admin -k -c cernopenstack -n i-0001ab95
Added:
>
>
List the cloud aliases and the active clouds within those aliases (for condor.heprc.uvic.ca useful for the IAAS queue)
cloud_admin -y
cat /etc/cloudscheduler/cloud_alias.json
 To change the number of VM slots, edit the CLOUD entry in /etc/cloudscheduler/cloud_resources.conf and then execute (this will charge the cloud statuses back to the default setting)
Line: 84 to 90
 https://us-west-2.console.aws.amazon.com/ec2/v2/home?region=us-west-2#SpotInstances:
Added:
>
>
To list all the image names and their ami values on CERN (on Belle-CS)
/root/cern_ec2_ami.py
  -- RandallSobie - 2014-03-18 \ No newline at end of file

Revision 102014-12-16 - rsobie

Line: 1 to 1
 
META TOPICPARENT name="RandallSobie"

CloudScheduler commands

Line: 43 to 43
 service cloud_scheduler quickrestart
Added:
>
>
To modify the VM-type or VM-flavour, edit the file and restart Cloudscheduler
vi /etc/cloudscheduler/cloud_scheduler.conf
 To retire all the VMs in a cloud (they must be registered with HTCondor)
cloud_admin -d <cloudname>

Revision 92014-10-22 - rsobie

Line: 1 to 1
 
META TOPICPARENT name="RandallSobie"

CloudScheduler commands

Changed:
<
<
List the status of the Belle2 Cloud Schdeduler (bellecs.heprc.uvic.ca)
>
>
List the status of the Belle2 Cloud Schdeduler (bellecs.heprc.uvic.ca and condor.heprc.uvic.ca)
 
cloud_status -a

Revision 82014-10-15 - rsobie

Line: 1 to 1
 
META TOPICPARENT name="RandallSobie"

CloudScheduler commands

Line: 73 to 73
 /opt/dirac/runit/WorkloadManagement/SiteDirectorUVic/log/current
Added:
>
>
To get the plot of the EC2 spot price (c3.4xlarge)
https://us-west-2.console.aws.amazon.com/ec2/v2/home?region=us-west-2#SpotInstances:
  -- RandallSobie - 2014-03-18 \ No newline at end of file

Revision 72014-10-14 - rsobie

Line: 1 to 1
 
META TOPICPARENT name="RandallSobie"

CloudScheduler commands

Line: 45 to 45
  To retire all the VMs in a cloud (they must be registered with HTCondor)
Added:
>
>
cloud_admin -d
 cloud_admin -o -c -a

Revision 62014-10-09 - rsobie

Line: 1 to 1
 
META TOPICPARENT name="RandallSobie"

CloudScheduler commands

Line: 39 to 39
 /etc/cloudscheduler/cloud_resources.conf and then execute (this will charge the cloud statuses back to the default setting)
Added:
>
>
vi /etc/cloudscheduler/cloud_resources.conf
 service cloud_scheduler quickrestart
Line: 68 to 69
 
/var/log/cloudscheduler.log 
/var/log/condor/MatchLog
Added:
>
>
/opt/dirac/runit/WorkloadManagement/SiteDirectorUVic/log/current
 

Revision 52014-09-18 - rsobie

Line: 1 to 1
 
META TOPICPARENT name="RandallSobie"

CloudScheduler commands

Line: 42 to 42
 service cloud_scheduler quickrestart
Added:
>
>
To retire all the VMs in a cloud (they must be registered with HTCondor)
cloud_admin -o -c <cloudname> -a

To kill all the VMs in a cloud (get rid of VMs not registered with HTCondor)

cloud_admin -k -c <cloudname> -a

To see what VMs have attached to condor

condor_status -m
 To hold all idle jobs and then release the idle jobs in the condor queue:
condor_hold -const 'JobStatus==1'

Revision 42014-04-09 - rsobie

Line: 1 to 1
 
META TOPICPARENT name="RandallSobie"

CloudScheduler commands

Line: 39 to 39
 /etc/cloudscheduler/cloud_resources.conf and then execute (this will charge the cloud statuses back to the default setting)
Changed:
<
<
service cloud_scheduler quickrestart'
>
>
service cloud_scheduler quickrestart
 

To hold all idle jobs and then release the idle jobs in the condor queue:

Revision 32014-03-25 - rsobie

Line: 1 to 1
 
META TOPICPARENT name="RandallSobie"

CloudScheduler commands

Line: 35 to 35
 cloud_admin -k -c cernopenstack -n i-0001ab95
Added:
>
>
To change the number of VM slots, edit the CLOUD entry in /etc/cloudscheduler/cloud_resources.conf and then execute (this will charge the cloud statuses back to the default setting)
service cloud_scheduler quickrestart'

To hold all idle jobs and then release the idle jobs in the condor queue:

condor_hold -const 'JobStatus==1'
condor_release -const 'JobStatus==1'
 Also check the logs in
/var/log/cloudscheduler.log 

Revision 22014-03-21 - rsobie

Line: 1 to 1
 
META TOPICPARENT name="RandallSobie"

CloudScheduler commands

Line: 25 to 25
 condor_q -analy [job.id]
Added:
>
>
To retire VM i-0001ab95 running on cernopenstack and is already in condor use:
cloud_admin -o -c cernopenstack -n i-0001ab95

To kill a VM which does not show up in condor yet use -k instead of -o, for example:

cloud_admin -k -c cernopenstack -n i-0001ab95
 Also check the logs in
/var/log/cloudscheduler.log 

Revision 12014-03-18 - rsobie

Line: 1 to 1
Added:
>
>
META TOPICPARENT name="RandallSobie"

CloudScheduler commands

List the status of the Belle2 Cloud Schdeduler (bellecs.heprc.uvic.ca)

cloud_status -a

Clusters in resource pool:
Cluster: alto
ADDRESS                    CLOUD TYPE       VM SLOTS    MEMORY     STORAGE    HYPERVISOR ENABLED
alto.cloud.nrc.ca          OpenStack        10          [224000]   2000                  False
(and more clouds)

To enable (-e) or disable (-d) clouds

cloud_admin -d e alto

Check the status of a single job

condor_q -analy [job.id]

Also check the logs in

/var/log/cloudscheduler.log 
/var/log/condor/MatchLog

-- RandallSobie - 2014-03-18

 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback