Difference: CloudschedulerPaperOutline (1 vs. 7)

Revision 72019-07-02 - rsobie

Line: 1 to 1
 
META TOPICPARENT name="RandallSobie"
CSV2 journal paper
Line: 11 to 11
 
  • motivation for original CS system and its design
    • Take advantage of opportunistic virtual resources.
    • Monolithic, multi-threaded python 2 module with in-memory data structures.
Deleted:
<
<
 
  • performance
Added:
>
>
    • use for HEP (atlas and belle2) and canfar
 
  • limitations and issues
    • constrained to one node.
    • Single job scheduler.
Line: 49 to 49
 
    • Retain condor as job scheduler because of it's dynamic resource management
    • Mariadb, apache, python3, amqp, influxDB, celery, Jinja2, html5, css, javascript, centos7
Changed:
<
<
  • monitoring, accounting and alerting
>
>
  • monitoring
  * timeseries
Changed:
<
<
* ???
>
>
    • no alerting at the moment

  • accounting
    • currently this is a user responsiblity
 

Revision 62019-07-02 - crlb

Line: 1 to 1
 
META TOPICPARENT name="RandallSobie"
CSV2 journal paper
Line: 9 to 9
  CS first version
  • motivation for original CS system and its design
Added:
>
>
    • Take advantage of opportunistic virtual resources.
 
    • Monolithic, multi-threaded python 2 module with in-memory data structures.

  • performance
  • limitations and issues
Added:
>
>
    • constrained to one node.
    • Single job scheduler.
 
    • CLI user interface only.
    • Scaling.

CS second version

Changed:
<
<
  • design goal and new use cases
>
>
  • Use cases
    • Multiple Projects/Users/Condors
    • Simplified/Remote management (CLI/GUI)
    • Layered management of the resources
    • Built in status monitoring (as opposed to external tracking)
    • More effective use of resources (auto scaling)
    • Improved diagnostics
    • Run on any cloud type
    • Integrated image & SSH key management (glint)

  • Design goal
    • Useability & maintainability.
    • Latest software packages
    • Use standard based open source software, develop only when needed
    • Database centric (maintains state, easily extensible, crash recovery)
    • Improved maintenance options & state recovery by reducing number of critical components
    • Modular components
    • General purpose (not HEP specific)
    • Allow user complete control of VM contextualization
    • Abstraction of cloud type
 
  • architecture
Added:
>
>
    • Based on Openstack database centric model (Groups/Projects, Users assigned to groups)
    • Check for Openstack papers on architecture
 
  • implementation
Added:
>
>
    • Retain condor as job scheduler because of it's dynamic resource management
    • Mariadb, apache, python3, amqp, influxDB, celery, Jinja2, html5, css, javascript, centos7
 
  • monitoring, accounting and alerting
Added:
>
>
* timeseries * ???
 

Revision 52019-07-02 - crlb

Line: 1 to 1
 
META TOPICPARENT name="RandallSobie"
CSV2 journal paper
Line: 9 to 9
  CS first version
  • motivation for original CS system and its design
Added:
>
>
    • Monolithic, multi-threaded python 2 module with in-memory data structures.
 
  • performance
  • limitations and issues
Added:
>
>
    • CLI user interface only.
    • Scaling.
  CS second version
  • design goal and new use cases

Revision 42019-07-02 - rsobie

Line: 1 to 1
 
META TOPICPARENT name="RandallSobie"
CSV2 journal paper
Line: 6 to 6
 
  • review of cloud provisioning systems (vcycle, vmdirac, htcondor, commercial systems, ..)
  • opportunistic computing
Deleted:
<
<
https://cloud.google.com/solutions/using-clusters-for-large-scale-technical-computing

https://azure.microsoft.com/en-ca/services/batch/

https://www.gridpp.ac.uk/vcycle/

vcycle CHEP paper

vmdirac CHEP paper

  CS first version
  • motivation for original CS system and its design
Line: 27 to 18
 
  • implementation
  • monitoring, accounting and alerting
  • performance
Added:
>
>
  Summary

Appendix

  • Installation and configuration
Added:
>
>
References

Revision 32019-07-01 - rsobie

Line: 1 to 1
 
META TOPICPARENT name="RandallSobie"
CSV2 journal paper
Line: 6 to 6
 
  • review of cloud provisioning systems (vcycle, vmdirac, htcondor, commercial systems, ..)
  • opportunistic computing
Added:
>
>
https://cloud.google.com/solutions/using-clusters-for-large-scale-technical-computing

https://azure.microsoft.com/en-ca/services/batch/

https://www.gridpp.ac.uk/vcycle/

vcycle CHEP paper

vmdirac CHEP paper

 CS first version
  • motivation for original CS system and its design
  • performance

Revision 22019-06-30 - rsobie

Line: 1 to 1
 
META TOPICPARENT name="RandallSobie"
CSV2 journal paper
Line: 15 to 15
 
  • design goal and new use cases
  • architecture
  • implementation
Added:
>
>
  • monitoring, accounting and alerting
 
  • performance

Summary

Added:
>
>
Appendix
  • Installation and configuration
 \ No newline at end of file

Revision 12019-06-30 - rsobie

Line: 1 to 1
Added:
>
>
META TOPICPARENT name="RandallSobie"
CSV2 journal paper

Introduction

  • review of cloud provisioning systems (vcycle, vmdirac, htcondor, commercial systems, ..)
  • opportunistic computing

CS first version

  • motivation for original CS system and its design
  • performance
  • limitations and issues

CS second version

  • design goal and new use cases
  • architecture
  • implementation
  • performance

Summary

 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback