-- ColinLeavettBrown - 2013-08-28

NEP52 Shared Software Repository


At the core of the Shared Software Repository service is the CERN Virtual Machine File System (CVMFS) server. This server provides a read-only, hierarchical storage management, POSIX compliant file system, mountable through HTTP. That's a lot of buzz words, but basically it means that you can install your software once in a CVMFS server and share it efficiently across the web with many other machines, both physical and vittual.

* **Much more needed here, plus diagram** *

Shared Software Repository Test Drive

You can use the software provided in this RPI project to easily distribute software on demand to virtual machines. The following examples will allow you to test drive this functionality quickly. In summary:

  • "Running the sample application" will have you launch an instance of the CVMFS server image (NEP52-cvmfs-server) and an instance of an interactive client (NEP52-interactive-cvmfs-client). You will then log into the client VM, point it at your CVMFS instance and run the sample application.
  • " Customizing the software repository" will have you create and publish a second sample application within the software repository. Subsequently, you will execute the new application from the interactive client.

In order to try the Shared Software Repository Test Drive, you will need the following:

  • A DAIR login ID with a large enough quota to run the three concurrent demonstration instances.
  • To create your own keypair and save the pem file locally (see the Openstack dashboard/documentation).
  • To retrieve your EC2_ACCESS_KEY and EC2_SECRET_KEY from the Openstack dashboard.

Running the sample application

Step 1: Log into DAIR and boot a Shared Software Repository instance

Login into the DAIR OpenStack Dashboard: https://nova-ab.dair-atir.canarie.ca . Select the alberta region. Refer to the OpenStack docs for all the details of booting and managing VMs via the dashboard.

Go to the 'Images and Snapshot' tab on the left of the page then click the button that says 'Launch' next to the NEP52-cvmfs-server image.

Fill in the form to look the same as the screen shot below substituting your username wherever you see the string "hepnet".


Next, you need to select your SSH key to associate with the instance so that you can login to the image. Click the access and security tab, pick your key, click "launch" (see screen shot below) and wait for the instance to become active.


Now associate a floating IP to the machine. Click on the instances tab on the left. From the "Actions" beside your newly started CVMFS server instance, choose "Associate Floating IP", complete the dialog and click "Associate" (we will use the IP address "" for the CVMFS server).

Step 2: Launch an interactive client

Using the same procedure as for the CVMFS server image, launch an instance of NEP52-interactive-cvmfs-client. You may set the instance name to anything you like, but you must assign your keypair and associate a floating IP to the instance (we will use the IP address "" for the interactive client).

Step 3: Log into the interactive client and run the sample application

Log into the interactive client and point it to the CVMFS server (again, substitute your username wherever you see the string "hepnet". ):

%STARTCONSOLE% ssh -i ~/.ssh/MyKey.pem root@ PointToCVMFS hepnet %ENDCONSOLE%

Now switch to the guest account, list and run the application:

%STARTCONSOLE% su - guest ls -l /cvmfs/dair.cvmfs.server/* /cvmfs/dair.cvmfs.server/Hello %ENDCONSOLE%

Adding an application to the CVMFS server

In this section we will show you how to modify the CVMFS server to distribute your own software.

Step 1: Log into the CVMFS server and create a second demonstration application

Log into the CVMS server, switch to the distributed software directory, and copy the "Hello" bash script to the file "Goodbye". Then edit the "Goodbye" script to echo a different message.

%STARTCONSOLE% ssh -i ~/.ssh/MyKey.pem root@ cd /cvmfs/dair.cvmfs.server cp Hello Goodbye vi Goodbye %ENDCONSOLE%

When you have saved your changes to the Goodbye script, publish your newly changed script to the world via CVMFS:

%STARTCONSOLE% chown -R cvmfs.cvmfs /cvmfs/dair.cvmfs.server cvmfs-sync cvmfs_server publish %ENDCONSOLE%

Step 2: Use the interactive client to run the new application

If you are still logged into the interactive client as "guest", use the "PointToCVMFS" command to restart then CVMFS client and then list and run the new application (again, make sure you substitute your username wherever you see the string "hepnet". ):

%STARTCONSOLE% sudo PointToCVMFS hepnet ls -l /cvmfs/dair.cvmfs.server/* /cvmfs/dair.cvmfs.server/Goodbye %ENDCONSOLE%

Otherwise, if you are not still logged into the interactive client, log in, restart the CVMFS client, switch to the guest account, and run the newly created application:

%STARTCONSOLE% ssh -i ~/.ssh/MyKey.pem root@ PointToCVMFS hepnet su - guest ls -l /cvmfs/dair.cvmfs.server/* /cvmfs/dair.cvmfs.server/Goodbye %ENDCONSOLE%

Take snapshots of your customized images

If you followed all the steps above you have customized versions of the CVMFS server and the Interactve CVMFS client running. You can now use the OpenStack dashboard to snapshot these servers to save yourself the work of customizing them again.

Topic attachments
I Attachment History Action Size Date Who Comment
PNGpng launch.png r1 manage 60.4 K 2013-08-28 - 21:37 UnknownUser  
Edit | Attach | Watch | Print version | History: r8 | r6 < r5 < r4 < r3 | Backlinks | Raw View | More topic actions...
Topic revision: r4 - 2013-08-29 - crlb
  • Edit
  • Attach
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2021 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback