This page contains information on GSI support in the cloud scheduler.

GSI support in the cloud scheduler allows a user to authenticate using his/her grid certificate when submitting a job to the cloud scheduler. The cloud scheduler will then use these credentials for authenticating the Nimbus workspace creation and workspace deletion.

Enabling GSI support in the cloud scheduler will also put some restrictions on the VM which will only allow jobs from the owner of that VM to be started on it. In other words, jobs owned by user B will not be started on a VM owned by user A. The rationale behind this is to prevent access to a delegated proxy on a VM to other users.

The GSI support is still in prototype mode and is available under the myproxy-integration-dev branch.

Requirements:

  • A cloud scheduler codebase with GSI support
    • merged into dev branch on Sep 13, 2010
  • A working CA is required to sign the dummy VM host certificate.
    • Done. Running on alto.cloud.nrc.ca
  • A working Globus Toolkit is required on the host running the cloud scheduler.
  • The user requires a valid grid certificate (x509)
  • The VM images must have a recent version of the condor startup scripts (with generic local condor config support)

Install NEP-52 root CA package

On the cloud scheduler host:
$ wget --no-check-certificate https://wiki.heprc.uvic.ca/twiki/pub/Restricted/CsGsiSupport/globus_simple_ca_ebd3459c_setup-0.20.tar.gz
$ gpt-build ./globus_simple_ca_ebd3459c_setup-0.20.tar.gz <flavor>
# gpt-postinstall
# $GLOBUS_LOCATION/setup/globus_simple_ca_ebd3459c_setup/setup-gsi

Request dummy host certificate for VM instances

On the cloud scheduler host:
$ mkdir VM-host-cert
$ cd VM-host-cert
$ grid-cert-request -dir . -host NEP-52_VM_instance -ca
For the time being, the NEP-52 CA is hosted on alto.cloud.nrc.ca. Send the above certificate request to Andre.Charbonneau@nrc-cnrc.gc.ca Install the signed certificate in the VM-host-cert directory created above.

Configure GSI Authentication in Condor

In the condor config on the cloud scheduler host:
SEC_CLIENT_AUTHENTICATION = REQUIRED
SEC_CLIENT_AUTHENTICATION_METHODS = GSI
SEC_CLIENT_ENCRYPTION = REQUIRED
SEC_CLIENT_ENCRYPTION_METHODS = 3DES
GRIDMAP = /etc/grid-security/grid-mapfile.condor
This will enable both authentication (GSI) and encryption for clients connecting to this condor server. Do not forget to create the grid mapfile specified in the above configuration.

For some unknown reason, my grid mapfile must contain an entry for the host cert of cloud scheduler (vm129 in my case), such as:

"/C=CA/O=Grid/CN=host/vm129.cloud.nrc.ca" condor@vm129.cloud.nrc.ca
(Replace vm129.cloud.nrc.ca with the hostname of your cloud scheduler host.)

Note that if uses are not listed in the grid mapfile, these users will sill be authenticated and authorized to look at the condor info (READ operations), but will not be allowed to submit any jobs.

In order to the changes to take effect, you need to restart condor:

# /etc/init.d/condor stop
# /etc/init.d/condor start

To test that GSI authentication is actually enabled, try to run the condor_q command without a user proxy. You should get an error message, such as shown below:

$ grid-proxy-destroy
$ condor_q

-- Failed to fetch ads from: <132.246.148.29:8080> : vm129.cloud.nrc.ca
AUTHENTICATE:1003:Failed to authenticate with any method
AUTHENTICATE:1004:Failed to authenticate using GSI
GSI:5003:Failed to authenticate.  Globus is reporting error (851968:28).  There is probably a problem with your credentials.  (Did you run grid-proxy-init?)

Then create a proxy and try the command again. The condor_q command should work now.

Configure CA roots in cloud scheduler

We need to specify which CA root certificates and signing policy we need on our VMs. This is done by adding the following to the cloud scheduler config file:
ca_root_certs: /etc/grid-security/bffbd7d0.0,/etc/grid-security/certificates/ebd3459c.0
ca_signing_policies: /etc/grid-security/bffbd7d0.signing_policy,/etc/grid-security/certificates/ebd3459c.signing_policy

Configure VM dummy host certificate in cloud scheduler

cert_file: /home/andre/work/cloud-scheduler/VM_host_cert/hostcert.pem
cert_file_on_vm: /etc/grid-security/hostcert.pem

key_file: /home/andre/work/cloud-scheduler/VM_host_cert/hostkey.pem
key_file_on_vm: /etc/grid-security/hostkey.pem

Configure the nimbus grid-mapfile on the cloud servers

Make sure that the authorized users DN are added to the Nimbus grid mapfiles on the cloud server that this user is allowed to use. For example, on alto.cloud.nrc.ca, this is in the following file:
/usr/local/nimbus/services/etc/nimbus/nimbus-grid-mapfile

Testing

Restart the cloud scheduler

Create a user proxy (full legacy). Make sure it's lifetime will cover the duration of the job.

$ grid-proxy-init -old [-valid HH:MM]

Add x509 proxy info in your job description

In order to use GSI authentication, you need to specify your user proxy in your job description. This is done using the x509userproxy classad attribute. For example:
x509userproxy = /tmp/x509up_u20200

Submit the job

$ condor_submit <job-description-file>

Credential renewal

The cloud scheduler implements job credential renewal via a MyProxy server. The idea is simple: the user first puts a long lived proxy a MyProxy server prior to submitting a job and then puts the proxy information in the job description. Periodically, the cloud scheduler will scan all the jobs proxy certificates and attempt to renew those which are about to expire.

To use automatic credential renewal, follow the instructions below:

Configure cloud scheduler to enable credential renewal

Edit the following cloud scheduler configuration parameters in your cloud_scheduler.conf file:
# job_proxy_refresher_interval specifies the amount of time, in seconds, between each job proxy
# credential expiry checks.  To disable proxy refreshing altogether, simply set this
# value to -1
#
# The default value is -1
#job_proxy_refresher_interval: -1

# job_proxy_renewal_threshold determines the amount of time, in seconds, 
# prior to proxy expiry date at which a proxy will be refreshed
#
# The default value is 900 (15 minutes)
#job_proxy_renewal_threshold: 900

Put a long-lived proxy to a MyProxy server

Prior to submitted one or more long lived job, the user should run a command like the following:
myproxy-init -R 'host/<cloud_scheduler_host>' -k <creds_name> -s alto.cloud.nrc.ca -d
In the above command, replace with the FQHN of your cloud scheduler and with unique name for your credentials in the MyProxy server. Also, if needed, change alto.cloud.nrc.ca in the above command to the name of the MyProxy server for your cloud scheduler (contact your system administrator if you are not sure what value to use for the MyProxy server).

The default lifetime of the delegated credentials on the MyProxy server is one week. If you want a different lifetime, specify it using the -c command line argument to the myproxy-init command shown above.

Add MyProxy info to job description

The user must put the following information in his/her job description:
+CSMyProxyServer     = alto.cloud.nrc.ca
+CSMyProxyCredsName  = <creds_name>
In the above job description attributes, replace with the unique name for your credentials in the MyProxy server. Also, if needed, change alto.cloud.nrc.ca in the above command to the name of the MyProxy server for your cloud scheduler.

-- AndreCharbonneau - 2010-08-30

Topic attachments
I Attachment History Action Size Date Who Comment
Unknown file formatgz globus_simple_ca_ebd3459c_setup-0.20.tar.gz r1 manage 209.7 K 2010-08-30 - 20:38 UnknownUser NEP-52 CA package (root cert, signing policy, etc...)
Edit | Attach | Watch | Print version | History: r24 | r8 < r7 < r6 < r5 | Backlinks | Raw View | More topic actions...
Topic revision: r6 - 2010-10-04 - andrec
 
  • Edit
  • Attach
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback