Overall Architecture

Key Features

• independent of the Cloud Scheduler
• no need to archive Condor log files and grep through megabytes of logs
• live stats (DB entry added as soon as a job completes)
• easy data replication (if needed; provided by MySQL)
• flexible query system from any host (using MySQL client)
• rabbitmq takes care of message delivery (data collector can go offline without losing any data)
• can select what goes into the accounting DB and what doesn't (i.e., keep out all the benchmark runs and tests)

Online Accounting Stats

Accounting stats are available online via the babar.cloud.nrc.ca web portal at:

• https://babar.cloud.nrc.ca/?q=node/20

VM Setup

1. Install the following python libraries using your favorite method:
   • pika
   • json
2. Install the accounting reporting tool
   1. Download the code from github at https://github.com/hep-gc/nep52accounting

      git clone git@github.com:hep-gc/nep52accounting.git
      cd nep52accounting/client
            

   2. Open the nep52accountant file and make sure the python executable on the first line of the script is OK. Edit as needed.
   3. Run make install to install the data reporting component and follow instructions

      sudo make install
            

Job Setup

1. Add the following line to jobs you want to report accounting data:

   +HookKeyword = "NEP52ACNT"
      

That's it! From now on, these jobs will report back the classad of completed jobs to a data collector on babar.cloud.nrc.ca.

Collected Data

Currently, the following data is collected from every job that completes and has the NEP52ACNT Condor hook set:

GlobalJobId
User
Owner
x509userproxysubject
QDate
JobStartDate
JobCurrentStartDate
JobDuration
StageInStart
StageInFinish
CommittedTime
CumulativeSuspensionTime
RemoteSysCpu
RemoteUserCpu
RemoteWallClockTime
LocalSysCpu
LocalUserCpu
BytesSent
ExitCode
DiskUsage
RemoteHost
OrigCmd

This is just a preliminary set of Condor classad attributes to get things started. More attributes can easily be added at a later date.

Running Queries Against the Accounting Database

To query the accounting database, simply use a recent mysql client and connect to babar.cloud.nrc.ca as user nep52acntRO. (Contact Andre.Charbonneau@nrc-cnrc.gc.ca for password.) Note that the nep52acntRO user only has SELECT privileges on the nep52.completed_jobs table.

For example:

mysql -h babar.cloud.nrc.ca -u nep52acntRO -p nep52 -e "<mysql query goes here...>"

If you want the output to be stored in a text file to be included in other applications for processing, simply redirect the output of the mysql command to a file. For example:

mysql -h babar.cloud.nrc.ca -u nep52acntRO -p nep52 -e "SELECT QDate, JobStartDate FROM completed_jobs ORDER BY QDate" > query.dat

Query Examples

Here you will find a list of commonly used queries. Add your favorite queries here!

# Get total number of completed jobs
SELECT COUNT(*) FROM completed_jobs;

-- AndreCharbonneau - 2011-08-29