create new tag
view all tags
CSV2 journal paper


  • review of cloud provisioning systems (vcycle, vmdirac, htcondor, commercial systems, ..)
  • opportunistic computing

CS first version

  • motivation for original CS system and its design
    • Take advantage of opportunistic virtual resources.
    • Monolithic, multi-threaded python 2 module with in-memory data structures.
  • performance
    • use for HEP (atlas and belle2) and canfar
  • limitations and issues
    • constrained to one node.
    • Single job scheduler.
    • CLI user interface only.
    • Scaling.

CS second version

  • Use cases
    • Multiple Projects/Users/Condors
    • Simplified/Remote management (CLI/GUI)
    • Layered management of the resources
    • Built in status monitoring (as opposed to external tracking)
    • More effective use of resources (auto scaling)
    • Improved diagnostics
    • Run on any cloud type
    • Integrated image & SSH key management (glint)

  • Design goal
    • Useability & maintainability.
    • Latest software packages
    • Use standard based open source software, develop only when needed
    • Database centric (maintains state, easily extensible, crash recovery)
    • Improved maintenance options & state recovery by reducing number of critical components
    • Modular components
    • General purpose (not HEP specific)
    • Allow user complete control of VM contextualization
    • Abstraction of cloud type

  • architecture
    • Based on Openstack database centric model (Groups/Projects, Users assigned to groups)
    • Check for Openstack papers on architecture

  • implementation
    • Retain condor as job scheduler because of it's dynamic resource management
    • Mariadb, apache, python3, amqp, influxDB, celery, Jinja2, html5, css, javascript, centos7

  • monitoring
    • timeseries
    • no alerting at the moment

  • accounting
    • currently this is a user responsiblity



  • Installation and configuration


Edit | Attach | Watch | Print version | History: r7 < r6 < r5 < r4 < r3 | Backlinks | Raw View | More topic actions
Topic revision: r7 - 2019-07-02 - rsobie
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2022 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback