Cloud Scheduler REST API

The APIs provided by admin_server.py and info_server.py and consumed by cloud_admin and cloud_status have been transitioned from XML-RPC to REST. SimpleXMLRPCServer and xmlrpclib have been replaced with web.py and requests. This page provides information regarding this transition.

API Design

After a review of the server and client, the following URL schema was proposed. The HTTP method, URL and parameters, and replaced XML-RPC functions are listed. The API follows a RESTful approach, where URLs represent resources and HTTP methods represent verbs that apply to the resources. GET requests are safe, PUT requests are unsafe but idempotent, and POST and DELETE are neither.

cloud_admin

PUT      /?log_level=LOG_LEVEL
         change_log_level
POST   /?action=reconfig|quick_shutdown
         cloud_resources_reconfig
         perform_quick_shutdown
PUT      /clouds/CLOUD?action=(enable|disable)
         enable_cloud
         disable_cloud
PUT      /clouds/CLOUD?allocations=NUMBER
         adjust_cloud_allocation
DELETE   /clouds/CLOUD/vms
         delete_all_vm_entry_cloud
PUT      /clouds/CLOUD/vms?action=shutdown[&count=all|NUMBER]
         shutdown_cluster_count
         shutdown_cluster_all
POST   /clouds/CLOUD/vms?action=force_retire[&count=all|NUMBER]
         force_retire_count_vm
         force_retire_all_vm
DELETE   /clouds/CLOUD/vms/VMID
         delete_vm_entry
PUT      /clouds/CLOUD/vms/VMID?action=(reset_override_state|shutdown)
         reset_override_state
         shutdown_vm
POST   /clouds/CLOUD/vms/VMID?action=force_retire
         force_retire_vm
POST   /users/USER?refresh=(job_proxy|vm_proxy)
         refresh_job_proxy_user
         refresh_vm_proxy_user
GET      /cloud-aliases
         list_cloud_alias
POST   /cloud-aliases
         cloud_alias_reload
GET      /user-limits
         list_user_limits
POST   /user-limits
         user_limit_reload

cloud_status

GET      /
         get_version
GET      /cloud
         get_cloud_resources
GET      /cloud/config
         get_cloud_config_values
GET      /clusters
         get_cluster_resources
GET    /clusters.json
         get_json_resource
GET      /clusters/CLUSTER
         get_cluster_info
GET      /clusters/CLUSTER.json
         get_json_cluster
GET      /clusters/CLUSTER/vms
         get_vm_stats
         get_total_vms_cloud
GET      /clusters/CLUSTER/vms/VMID
         get_vm_info
GET      /clusters/CLUSTER/vms/VMID.json
         get_json_vm
GET      /jobs?state=(new|sched|high|idle|running|complete|held)
         get_newjobs
         get_schedjobs
         get_highjobs
         get_idlejobs
         get_runningjobs
         get_completejobs
         get_heldjobs
GET      /jobs/JOBID
         get_job
GET      /jobs/JOBID.json
         get_json_job
GET      /vms[?metric=total|missing|startup_time|job_run_times]
         get_vm_stats
         get_total_vms
         get_missing_vm_list
         get_vm_startup_time
         get_vm_job_run_times
GET      /vms/resources
         get_cluster_vm_resources
GET      /job_pools.json
         get_json_jobpool
GET      /developer_info
         get_developer_information
GET      /ips
         get_ips_munin
GET      /diff-types
         get_diff_types
GET      /failures?type=(boot|image)
         get_job_failure_reasons
         get_image_failures
GET      /shared-objs
         check_shared_objs
GET      /thread-heart-beats
         get_thread_heart_beats

Server-side

On the server-side, web.py provides the HTTP server and request handling. web.py is a minimalist web framework that maps a set of URLs to view classes, and HTTP methods to view class instance methods. Replacing the XML-RPC server required modifying the imports, creating view classes and methods, moving the XML-RPC functions into the corresponding views, and replacing the code that started the SimpleXMLRPCServer with code that instantiates and starts a web.py server.

Typically a web.py application is served using

self.app = web.application(self.urls, globals())
self.app.run()

but this provides very little configurability. For instance, the port on which the server listens is taken from the first command line argument and cannot be specified otherwise. web.py embeds the WSGI server from CherryPy, and app.run() simply provides a convenient wrapper. Instead, CherryPyWSGIServer is used directly:

self.app = web.application(self.urls, globals())
self.server = web.wsgiserver.CherryPyWSGIServer(self.listen, self.app.wsgifunc(), server_name="localhost")
self.server.start()

Note that this does require using web.py's internal API and it is possible that this API could change. A possible alternative would be to embed CherryPyWSGIServer or add some other WSGI server as an additional dependency.

Client-side

On the client-side, requests provides the HTTP client and allows the simple replacement of XML-RPC calls with HTTP requests. The requests library is significantly simpler than Python's built-in HTTP library, urrlib. Making HTTP requests and handling the responses can be done in a single line of code. As an example, an XML-RPC request such as

print s.get_cluster_info(cli_options.cluster_name)

is replaced with

print requests.get(base_url + 'clusters/' + cli_options.cluster_name).text

where base_url gives the hostname and port (http://localhost:8111). The client code is otherwise unchanged, except that requests throws requests.exceptions.ConnectionError instead of socket.error.

Finally, requests automatically quotes/encodes URLs, parameters, and data, but since user input is being used directly in the URLs, additional quoting is performed. This allows any character to be used. For example, in the GET request above, if cli_options.cluster_name contains a /, as in test/name the following URL would be used:

Quoted:
http://localost:8111/clusters/test%2Fname

Unquoted:
http://localost:8111/clusters/test/name

The unquoted request would cause a 404 Not Found error, since no URL pattern matches this input. The quoted request is matched by /clusters/([\w\%-]+) Other problem characters include ? and #, both of which are handled correctly.

Testing

Manual testing was performed locally and on a development server. In addition, a simple test script, client_test was written that runs each possible client command and logs the output.

-- ringda - 2016-01-08

Comments