Tags:
create new tag
view all tags

Recent Changes in HEPrc Web retrieved at 08:09 (GMT)

Computing documentation MonitoringPages Private page General services CloudScheduler OpenstackCommandLine FindDefunctVMs...
CSV2 journal paper Introduction review of cloud provisioning systems (vcycle, vmdirac, htcondor, commercial systems, ..) opportunistic computing CS first...
Hugo heprc.phys.uvic.ca and rjs.heprc.uvic.ca web sites How to use Development is done on my desktop in the Desktop/Htest directory. Files are also archived...
Benchmarks from elastic search using Bmk V2.0 MonitoringPages RandallSobie Only showing data from 2019
Summary of monitoring pages (27 links and counting) RandallSobie Cloud scheduler systems (monitor.heprc and monitor dev.heprc) CSv1 production monitor...
Monthly summary of cloud accounting from elastic search using Acc V2.0 MonitoringPages RandallSobie Only showing data from 2019 Delivered kilo CPU hours and efficiency...
Monthly summary of cloud accounting from elastic search MonitoringPages RandallSobie Note that the elastic search data started in mid January 2018 Delivered kilo...
bellecs.heprc VM benchmarks from elastic search MonitoringPages RandallSobie Csv2Benchmarks, BellecsBenchmarks, Aiatlas009Benchmarks, CondorBenchmarks Amazon Compute...
aiatlas009 VM benchmarks from elastic search MonitoringPages RandallSobie Csv2Benchmarks, BellecsBenchmarks, Aiatlas009Benchmarks, CondorBenchmarks Cern plots...
Monthly summary for 2018 of cloud accounting from elastic search MonitoringPages RandallSobie Note that these tables are no longer updated Delivered kilo CPU hours...
Daily summary of Amazon EC2 usage MonitoringPages RandallSobie Current month (note first bin is the cost deducted from the AWS credit):
Monthly summary of Amazon EC2 usage MonitoringPages RandallSobie Note that the Totals do not include the period before June 2018 (start of current grant)
RandallSobie Submitting Belle2 analysis jobs on the grid using gbasf2 from KEK Running grid jobs (3.0.0) Initialization of Belle2 environment at KEK source /sw/belle...
RandallSobie Submitting Belle2 jobs from KEK and UVIC Login into KEK (login and computer systems) or the belleuser VM on beaver ssh sshcc1.kek.jp ssh login.cc.kek...
RandallSobie Get the free space on the UVIC Belle II storage element See the ES web page UVIC SE From the belleuser machine 206.12.154.170 (as user rsobie) $ source...
RandallSobie Login into KEK (login and computer systems) ssh sshcc1.kek.jp ssh login.cc.kek.jp Set up a new release (3.0.0) Need KEK ssh passphrase and DESY password...
KEK GHI file/tape system RandallSobie KEC CC documentation How to tell if a file is on disk or tape (Note that a slash (/) is required at the end when specifying...
condor.heprc VM benchmarks from elastic search (only good for 2018) MonitoringPages RandallSobie Csv2Benchmarks, BellecsBenchmarks, Aiatlas009Benchmarks, CondorBenchmarks...
csv2.heprc VM benchmarks from elastic search MonitoringPages RandallSobie Csv2Benchmarks, BellecsBenchmarks, Aiatlas009Benchmarks, CondorBenchmarks
Monthly summary of LRZ accounting from elastic search MonitoringPages RandallSobie
Monthly summary of Innsbruck accounting from elastic search MonitoringPages RandallSobie
Monthly summary of Google accounting from elastic search MonitoringPages RandallSobie
Monthly summary of Amazon (west and east) accounting from elastic search MonitoringPages RandallSobie
Monthly summary of ECDF accounting from elastic search MonitoringPages RandallSobie
HEP support icon:searchtopic Web search advanced search icon:index Web topic list all topics icon:mail Web notify subscribe to e mail alerts...
VM benchmarks from elastic search MonitoringPages RandallSobie Fast benchmark plots for each cloud scheduler system. Points (or green histograms) are for this month...
Cloud accounting from elastic search on aiatlas009.cern.ch (CERN CS) MonitoringPages RandallSobie
Cloud accounting from elastic search on bellecs.heprc.uvic.ca MonitoringPages RandallSobie
Cloud accounting from elastic search on condor.heprc.uvic.ca MonitoringPages RandallSobie
Atlas Job Monitoring We are running ATLAS payloads on many clouds. We have only little influence what type of jobs are submitted to our worker nodes. We can select...
General accounting information MonitoringPages RandallSobie Note that February 2018 had a 5 day outage (Feb 16 21) in the log accounting system
Cloud accounting from LOG files on CERN CS MonitoringPages RandallSobie Note that February 2018 had a 5 day outage (Feb 16 21) in the log accounting system
Cloud accounting from LOG files on Belle CS MonitoringPages RandallSobie Note that February 2018 had a 5 day outage (Feb 16 21) in the log accounting system
Cloud accounting on condor.heprc MonitoringPages RandallSobie Note that February 2018 had a 5 day outage (Feb 16 21) in the log accounting system
Jobs on Sim@P1 system MonitoringPages RandallSobie
Tips when running and using elasticsearch, logstack and kibana et al. Note that below I am running the curl commands on the private network interface on the normal...
Rolfs Random Collection of Stuff ElasticSearchTips
Belle2 DIRAC Site Director (SD) configuration The DIRAC software is located the following directories (bellecs.heprc.uvic.ca) /opt/dirac/pro /home/dirac Frank`s documentation...
Definition of VM states Orphaned VM: VM is not registered in HTCondor and is also not registered with Cloudscheduler Note one must check both condor...
Adding your Squid to Shoal This page describes how to configure a Squid server with shoal agent. Squid Installation If you already have a squid installed, skip to...
Epoch time date date `Nov 1 2016 00:00:00` %s date d @1477983600
Set up CVMFS Modify the site.conf and default.local in /etc/cvmfs directory cat site.config CVMFS HTTP PROXY `auto;http://kraken01.westgrid.ca:3128;DIRECT` CVMFS...
Network Logging Network monitoring from bellecs.heprc.uvic.ca or condor.heprc.uvic.ca sudo /usr/local/sbin/monitor chameleon network summary less /var/log/monitor...
Dynafed installation and user authentication copy tools tested with dynafed can be found here manual installation start with minimal CentOS 7 enable...
Test of copy tools with dynafed Version of tools needed to have it working on the source site (at least if DPM) latest certificates for the sites involved...
s3cmd Get pip curl O https://bootstrap.pypa.io/get pip.py python get pip.py Install s3cmd (currently installed on rjs1 VM at CERN) sudo pip install s3cmd s3cmd...
davix CERN (these both work) They also work from the belleuser VM 206.12.154.170 davix ls s3accesskey xx s3secretkey xx https://rjsBucket.cs3.cern.ch davix ls s3accesskey...
Set up root Set up CVMFS Modify the site.conf and default.local in /etc/cvmfs directory cat site.config CVMFS HTTP PROXY `auto;http://kraken01.westgrid.ca:3128;DIRECT...
RJS Dynafed documentation Installation on beaver.heprc VM (using cernvm3 micro 2.7 7.hdd): yum install epel release yum install dynafed yum install dynafed dmlite...
How to login to an Azure VM In the Azure portal, click on the VM. Go to `All Settings` Go to `Properties` Copy address and port number from `SSH` ssh p root@ Usually...
How to retire Azure VMs that are not registered in CS Two step process: Step 1: Find running VMs on condor (but not registered in CS) condor status wide grep...
Log into OpenStack VM with nova commands On macsob2.phys, cd Desktop/Nova, and issue nova set up script for required cloud and tenant nova floating ip associate...
Find VM`s with defunct processes (java) Hourly reporting of VM`s with defunct process in /var/log/vm defunct.log with entry (`defunct`: `1`). List all VM`s with a...
AmazonWebServices The instructions on the CernVM website use the 2009 java based release of the Amazon EC2 documented here. The current release of the java based...
VM Benchmarks CERN documentation http://bmkwg.web.cern.ch/bmkwg/docs/HowToRun.html Installation: bellecs.heprc.uvic.ca in /root/HEPbm.yaml condor.heprc.uvic...
To change the date requirement use echo $((`date %s` 86400)) Note that the query requires JSON time (multiply unix time by 1000) { `size`: 50000, `sort`:...
Documentation lcgdm dav lcgdm dav is the front end module for the federator (also used as a front end for DPM). It passes queries and responses to the federator through...
Coop WorkTerm Report Template Notes on style Co op workterm reports should take the form of a scientific report on a specific problem and a solution. Narrative style...
HTCondor Mips HTCondor makes a measurement of the MIPS when a VM is booted and puts the value into the class ad of the VM. There is an entry per VM and the entry for...
CPU efficiency of each running job Show VM image size, load average, wall clock time and CPU efficiency for each job. (need to manually change the cloud name) cd...
Fast LHCb HS06 Benchmark List files in specified day with non zero file size (all files later than April 1st 2016): ( the output redirection only works from /root...
Cloud Scheduler REST API The APIs provided by admin server.py and info server.py and consumed by cloud admin and cloud status have been transitioned from XML...
Setting up the Elliott 219 Kyocera printer on your Mac OSX workstation Download and save one of the following PPD files (corresponding to the project you are...
See also CloudSchedulerAdminGuide . CloudScheduler commands List the status of the Belle2 Cloud Schdeduler (bellecs.heprc.uvic.ca and condor.heprc.uvic.ca) cloud...
This documentation describes a simple recommended frontier squid configuration, for use with Shoal. It is a simplified subset of information from https://twiki.comp...
ColinLeavettBrown 2015 01 23 HEP Support Check out the topics below for answers to your query. If you still have issues with Linus, Mac OS X or the wiki, send...
testing RandallSobie 2015 11 30
HEPrc Web Preferences The following settings are web preferences of the HEPrc web. These preferences override the site level preferences in . and...
ColinLeavettBrown 2013 08 28 NEP52 Shared Software Repository Overview If you just want to use Shared Software Repository Service skip to the Shared Software Repository...
Setting up a new Shoal Server Setup and Configure a target VM First, create a Centos 6.x VM. Log into the VM and replace the default root `authorized keys` so that...
ColinLeavettBrown 2015 10 29 NEP 101: Final Technical Review (Sustaining) Date: 30 October 2015, 10:30 11:30 Location: University of Victoria, Elliott 105...
function Toggle(div) { if (div.style.display `block`) { div.style.display `none`; } else { div.style.display `block`; } } ColinLeavettBrown...
KevinCasteels 2015 06 02 CloudInit yaml merging The following describes how to merge multiple .yaml configuration files together when submitting as user data to CloudInit...
Glint Login Glint Server: ssh centos #64;glint.heprc.uvic.ca Login Glint Web Portal User Interface: https://glint.heprc.uvic.ca:8483/project/ Processes: Glint...
ColinLeavettBrown 2015 09 14 NEP 101: Fifth Technical Review (Sustaining) Date: 17 September 2015, 10:30 11:00 Location: University of Victoria, Elliott...
ColinLeavettBrown 2015 07 22 NEP 101: Fourth Technical Review (Sustaining) Date: 24 July 2015, 10:00 10:30 Location: University of Victoria, Elliott 105 Agenda...
ColinLeavettBrown 2015 06 11 NEP 101: Third Technical Review (Sustaining) Date: 12 June 2015, 10:00 10:30 Location: University of Victoria, Elliott 105 Agenda...
Cloud Scheduler Admin Guide Note on the cloud scheduler init script usage To avoid having VMs shutdown, always use quickrestart or quickstop instead of the...
ColinLeavettBrown 2015 04 22 NEP 101: Second Technical Review (Sustaining) Date: 24 April 2015, 10:00 10:30 Location: University of Victoria, Elliott 105...
ColinLeavettBrown 2015 02 03 NEP 101: First Technical Review (Sustaining) Date: 10 March 2015, 10:00 10:30 Location: University of Victoria, Elliott 105 Agenda...
ColinLeavettBrown 2015 03 09 IBM IaaS Cloud in Canada Date: 10 March 2015, 11:00 13:00 Location: University of Victoria, Elliott 105 Agenda (Randy...
ColinLeavettBrown 2015 02 03 NEP 101: Final Technical Review Date: 24 February 2015, 15:00 16:30 Location: University of Victoria, Elliott 105 Agenda...
ColinLeavettBrown 2015 02 06 Linux workstation user authentication using central LDAP service This procedure has been tested on Scientific Linux 6.6 and 7.0. Prerequisites...
Compute Canada Cloud Project Meetings 27 January 2015, 10:30 PST, Elliott 105 VC. (Randy)
ColinLeavettBrown 2015 01 27 NEP 101: Sixth Technical Review Date: 27 January 2015, 09:00 10:00 Location: University of Victoria, Elliott 105 Agenda...
ColinLeavettBrown 2015 01 26 Compute Canada Cloud Project Work Plan Cloud Scheduler For all but current users (ATLAS, Belle, CANFAR). Propose...
ColinLeavettBrown 2015 01 21 CANARIE Project: NEP 101 HEP Data Intensive Distributed Cloud Computing Researchers at the University of Victoria, National Research...
ColinLeavettBrown 2015 01 22 Links to Glint source code source code source code installation scripts
How to acquire a new computer in the UVic HEP group We can assist you in choosing an appropriate computer for your needs or the needs of your students. There are a...
NEP 101: Fifth Technical Review Date: 16 December 2014, 09:00 10:00 Location: University of Victoria, Elliott 105 Agenda (Randy) Complete Project...
ColinLeavettBrown 2014 11 04 Temporary Paris Presentation Page.
SC 2014 100G Server Development Stand Alone chef for R920 ssh A root #64;yourR920.uni.edu cd ~ curl L https://www.opscode.com/chef/install.sh bash git clone git...
Proxy expiration on condor.heprc.uvic.ca (should run in a cron job) sudo /usr/sbin/fetch crl Location of the DIRAC code /opt/dirac/pro/DIRAC/Resources/Computing...
ColinLeavettBrown 2014 10 10 NEP 101: Fourth Technical Review Date: 15 October 2014, 11:00 12:00 Location: University of Victoria, Elliott 105 Agenda...
Running Belle2 in a VM (sobie belle) image Login into the VM (Image Checksum ef895936a9e82f48a946bd1608312cfd) source /cvmfs/belle.cern.ch/tools/setup belle2.sh...
Boot openstack image and log in interactively Create key pair on Openstack cloud (Security and Access) Move it to the .ssh directory and chmod 600 Run the VM ssh...
ColinLeavettBrown 2014 08 28 NEP 101: Third Technical Review Date: 4 September 2014, 11:00 12:00 Location: University of Victoria, Elliott 105 Agenda...
NEP 101: Second Technical Review Date: 14 July 2014, 10:00 11:00 Location: University of Victoria, Elliott 105 Agenda Percentage Complete Project...
This page described the installation and upgrade procedures for a DIRAC server operating as a slave to the server at KEK for BelleII References The DIRAC Interweave...
Managing VMs on the UVIC Mouse OpenStack Cloud There is documentation at CERN http://information technology.web.cern.ch/book/cern private cloud user guide/openstack...
NEP 101: First Technical Review/Report to CANARIE Date: 26 March 2014, 13:00 14:45 Location: University of Victoria, Elliott 105 Agenda Introduction and...
Renewing my grid certificate Go to the Grid Canada web site https://cert.gridcanada.ca/pki/pub And `Request a Certificate`, use the same name and organization (uvic...
Running on the CERN OpenStack Cloud Login into lxplus and source initialization script . cern openrc.sh List running instances nova list Add credential (only needed...
NEP52 Cloud Computing Batch Service Overview If you already are familiar with batch computing and just want to get to running your jobs on VMs skip to the CloudScheduler...
BuildingATLASCernVM27 Step by step walk through on how to build the cloud image for ATLAS based on CernVM 2.7. This will be for a KVM hypervisor only for either Nimbus...
BuildingATLASScientificLinux6 Steps taken to create the SL6.4 VM image for the clouds in the ATLAS IAAS queue. Links Some backgrounds information: Inherited...
This page contains information on GSI support in the cloud scheduler. GSI support in the cloud scheduler allows a user to authenticate using his/her grid certificate...
ColinLeavettBrown 2013 12 19 NEP 101 HEP Data Intensive Distributed Cloud Computing Public Documents https://wiki.heprc.uvic.ca/twiki/pub/HEPrc/Nep101DataIntensiveClouds...
Amazon EC2 peering from UVic From the traceroutes below it appears that there is no research network peering from UVic to Amazon. We are transiting Bell commercial...
Smartstack Airbnb http://nerds.airbnb.com/smartstack service discovery cloud/ Smart stack is a tool for providing service discovery and routing to services dynamically...
RabbitMQTests Source code available on GitHub. This page has information and instructions on how to setup a rabbitMQ server using SSL. As of this writing, a test certificate...
to the project page. four solutions version of this page (rev 11). Problem description Cloud Scheduler development began in 2009. At that time the most...
Statistics for HEPrc Web Month: Topic views: Topic saves: File uploads: Most popular topic views: Top viewers: Top contributors...
ColinLeavettBrown 2013 09 09 NEP 3 Objectives ATLAS production on the cloud is currently restricted to workloads requiring modest amounts of input and output...
BuildingBelle2CernVM27 Step by step walk through on how to build the cloud image for Belle2 based on CernVM 2.7. This will be for a KVM hypervisor only for either...
FGHotel currently enabled and allowed to boot new VMs: Disable FGHotel to prevent new VMs from booting: FGHotel is now Disabled:...
If clouds would follow this standard it would make it easier to contextualize images: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AESDG chapter instancedata...
ColinLeavettBrown 2013 06 13 All In One Puppet install of Ubuntu 13.04/OpenStack on mouse01.heprc.uvic.ca On root #64;elephant.heprc.uvic.ca, Cobbler install...
CERN OpenStack Ibex Cloud Testing Outline We are adding some CERN Open Stack resources to our currently operating IaaS. We boot the VMs using Scheduler, and contextualize...
ColinLeavettBrown 2013 03 06 Installing Ubuntu 12.04 via Cobbler Using the cobbler server on elephant.heprc.uvic.ca, Ubuntu 12.04.2 was installed via the private...
Documentation https://rc.coepp.org.au/cernvm nectar Recipe used by the NeCTAR group to customize UVic images for OpenStack Dan`s recipe for CloudSigma may be a useful...
This is how to configure a CernVM image to run as a squid server. Refer also to ConfigureCernVM . CernVM v2.6 Batch Node Configuration Xen Image Configuration...
http://www.gizmodo.com.au/2012/11/42gb per second researchers bust not one but two data transfer speed records/ http://www.hpcwire.com/hpcwire/2012 11 26/high energy...
Supercomputing 2012 Documentation. MegaRaid Upgrade The following IBM provided software was saved on our repository for easy distribution: ibm fw sraidmr...
SuperComputing 2012 Memory to Memory No problems with memoty to memory: 37.806 Gb/s root@sc02 ~ # fdtClient P 7 c 192.168.100.4 /dev/zero d /dev/null Oct...
ATLAS IaaS Draft In collaboration with the ATLAS Distributed Computing Cloud R D group we have developed a method for running regular ATLAS jobs from Panda workload...
Mainstream: http://www.bbc.co.uk/news/technology 16174972 http://www.npr.org/blogs/thetwo way/2011/12/14/143711104/scientists set new internet speed record...
What is Jenkins? `Jenkins provides continuous integration services for software development, primarily in the Java programming language. It is a server based system...
100G Notes Links HundredGigabitTestingLog HundredGigabitMedia Diagrams 100G network Diagram (pdf) Rack Layout (png) Four Node to Four Node...
Overall Architecture Key Features independent of the Cloud Scheduler nothing special needed on the user`s VMs `watches` the condor history file; more...
November 14, 2011 http://sc repo.uslhcnet.org http://supercomputing.uvic.ca/ Memory to memory started the afternoon of the 13th. The key to improving the performance...
First off, you will want to build images on a KVM capable machine. If not, you will be missing out on the hardware acceleration and your install will take forever...
Top Menu of HEPrc Web This topic defines the menu structure of the HEPrc web, used by the TopMenuSkin. 1 Web` Create New Topic...
ColinLeavettBrown 2010 07 20 NEP 52 Lustre file system notes: Overview The elephant cluster was used in developing these notes. Much of the information comes from...
ColinLeavettBrown 2011 06 20 PCI, PCI X, and PCIe Comparisons BUS Clock Speed Type Width Max MB/s PCI 33.33 MHz Parallel...
Coding Guidelines What do we want for the coding outlines/layout? These are open to discussion General Do not use abbreviations For Java Use javaDoc comments Keep...
Useful Openssl Commands Certificates Create a self signed certificate with no password This command will create a certificate and key with no password. This is...
Bash autocompletion There is a good tutorial for this here. This is documented in man bash but here is a quick overview The following table lists useful bash variables...
Web Notification This is a subscription service to be automatically notified by e mail when topics change in this 1 web. This is a convenient service, so you do...
VM Parameters for Job Submission This document will describe the parameters that can be set in job description files in order to define and characterize the VMs that...
Installing Lustre and Xen on Scientific Linux 5 In order to have the advantages of both Xen and Lustre on the same box it necessary to do little bit of work at the...
Using git Repositories This is a very quick guide to using git repositories. It assumes that git is installed and running on your machine. For git information and...
Using Grid Proxy Certs with Https This page describes how to use grid proxy certificates with Apache and OpenSSL over https. First configure your server httpd and...
Informal HEP Git Repository This is a guide to use the informal git repository at HEP GC Git on fate 2 Creating a Shared Repository To create a shared repository...
Using github Online Repositories This is a very brief introduction to using github to manage your git repositories. It assumes that you have git installed and know...
Cloud Scheduler Requirements This document contains all potential and realized requirements for the cloud scheduler that is being designed as of May 2009. Though the...
Cloud Scheduler Use Cases The following use cases represent the basic functionality of the cloud scheduler that is, the following tasks are those that will be regularly...
Cloud Scheduler: Introduction and Design Concepts The cloud scheduler is software that sits within a cloud and is responsible for creating and manipulating cloud resources...
1 Web Create New Topic Index Search Changes Notifications RSS Feed Statistics Preferences
TWiki`s HEPrc web The 1 web of TWiki. TWiki is a Web Based Collaboration Platform for the Enterprise.
/HEPrc The 1 web of TWiki. TWiki is a Web Based Collaboration Platform for the Enterprise.

 

Show 10, 20, 50, 100, 500, 1000 results per page, or show all.

Related topics: RSS feed, rounded corners RSS feed, ATOM feed, WebNotify, site changes, site map

Edit | Attach | Watch | Print version | History: r1 | Backlinks | Raw View | More topic actions
Topic revision: r1 - 2006-11-15 - TWikiContributor
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback