Getting started with the elasticslice client

We’ve created a simple yet powerful client (bin/esclient.py) that can create a slice for you, keep it renewed, and add/delete nodes from it based on some simple policy knobs and configuration. We’ll show you how to do that below.

Overview

esclient.py was originally designed as a daemon process that would run forever, managing the needs of an elastic slice — creating it, renewing it periodically, and adding and removing nodes according to a policy set by the user (which is ideally a combination of the slice’s resource demands and currently-available resources). This daemon process spawns multiple, synchronized threads to do its work, and leaves the python console in interactive mode so that an administrator can control the daemon trivially. To run in daemon/interactive mode, do something like

$ path/to/bin/esclient.py [OPTIONS] interactive

(Recall that esclient.py and the elasticslice library use configuration options first from the command line, then falling back to the same options if present in your ~/.protogeni/elasticslice.conf file.)

However, esclient.py also supports individual commands covering nearly all of its functionality, in a subcommand style. Below we show a partial help listing (obtained via esclient.py -h — global option details excluded for brevity — make sure to run it yourself to see the most current information):

usage: esclient.py [-h] [-V] [-d] [--config CONFIG_FILE] [-e ENDPOINT]
                 [-s SERVER] [-p PORT] [-c CERT]
                 [--passphrasefile PASSPHRASEFILE] [-C CACERT]
                 [--ext_addr EXT_ADDR] [--ext_port EXT_PORT] [-U USERNAME]
                 [-P PASSWORD] [-A] [-M]
                 [--am-percent-available-minimum PERCENT_AVAILABLE_MINIMUM]
                 [--am-min-threshold MINTHRESHOLD]
                 [--am-max-threshold MAXTHRESHOLD] [--am-nodetype NODETYPE]
                 [--am-enable-delete] [-D DEVUSER] [-S SLICENAME]
                 [--default-cm DEFAULT_CM] [-H] [--cmuri CMURI]
                 [--sauri SAURI] [--fscache] [--fscachedir FSCACHEDIR]
                 [--image_urn IMAGE_URN] [--manager_class MANAGER_CLASS]
                 [--helper_class HELPER_CLASS]
                 [--clientserver_endpoint_class CLIENTSERVER_ENDPOINT_CLASS]
                 [--node_delete_wait_time NODE_DELETE_WAIT_TIME]

             {interactive,add_nodes,create_slice,create_sliver,delete_nodes,
              delete_slice,delete_sliver,get_hostname_and_port,get_keys,
              get_manifest,get_resources,get_self_credential,
              get_slice_credential,get_slice_expiration,get_sliver_credential,
              get_sliver_urn,get_user_email,get_user_uid,get_user_urn,
              managed_add_nodes,managed_create_slice,managed_create_sliver,
              managed_ensure_slice,managed_ensure_sliver,
              managed_should_add_nodes,managed_should_delete_nodes,
              renew_slice,renew_sliver,resolve_slice,resolve_sliver,
              resolve_user,sliver_status,sliver_status_details}
                  ...
positional arguments:
  {interactive,add_nodes,create_slice,create_sliver,delete_nodes,delete_slice,
   delete_sliver,get_hostname_and_port,get_keys,get_manifest,get_resources,
   get_self_credential,get_slice_credential,get_slice_expiration,
   get_sliver_credential,get_sliver_urn,get_user_email,get_user_uid,get_user_urn,
   managed_add_nodes,managed_create_slice,managed_create_sliver,
   managed_ensure_slice,managed_ensure_sliver,managed_should_add_nodes,
   managed_should_delete_nodes,renew_slice,renew_sliver,resolve_slice,
   resolve_sliver,resolve_user,sliver_status,sliver_status_details}
                      Subcommands
  interactive         Run in daemon/interactive mode
  add_nodes           Add nodes to a sliver at a CM.
  create_slice        Create a slice.
  create_sliver       Create a sliver at a CM.
  delete_nodes        Delete nodes from a sliver at a CM.
  delete_slice        Delete a slice.
  delete_sliver       Delete a sliver at a CM.
  get_hostname_and_port
                      Get a hostname and SSH port for the given `client_id`.
  get_keys            Get user's keys from the SA (SA:GetKeys).
  get_manifest        Get a sliver manifest from a CM
  get_resources       Get a resource advertisement from a CM.
  get_self_credential
                      Gets a self credential (SA::GetCredential).
  get_slice_credential
                      Get a slice credential (SA::GetCredential).
  get_slice_expiration
                      Get expiration time for slice
  get_sliver_credential
                      Get a sliver credential (CM::GetSliver()).
  get_sliver_urn      Get a sliver URN at a CM.
  get_user_email      Get your user email.
  get_user_uid        Get your user uid.
  get_user_urn        Get your user URN.
  managed_add_nodes   This function manages the addition of nodes. This
                      manager will call it to add nodes to this slice, as
                      necessary.
  managed_create_slice
                      Create the slice from within the manager, using
                      information from the ElasticSliceHelper associated
                      with this ElasticSliceManager.
  managed_create_sliver
                      Create the sliver from within the manager, using
                      information from the ElasticSliceHelper associated
                      with this ElasticSliceManager.
  managed_ensure_slice
                      Check to see if the slice exists; if not, create it.
  managed_ensure_sliver
                      Check to see if the sliver exists; if not, create it
                      using information from the ElasticSliceHelper
                      associated with this ElasticSliceManager.
  managed_should_add_nodes
                      By default, add_nodes calls this method to find out if
                      it should add any nodes. If this function returns an
                      integer, add_nodes interprets that value as the number
                      of new nodes to add, and it calls get_add_args() to
                      get the arguments to pass to _add_nodes. If the
                      function returns a dict, it passes that directly to
                      _add_nodes. If the function returns None, False,
                      nothing is added. This gives subclasses enough
                      flexibility, hopefully. If you pass a nonzero,
                      positive integer via `count`, this method must check
                      if it should add exactly `count` nodes, or not, and
                      return accordingly.
  managed_should_delete_nodes
                      Should we delete a node? If so, this function must
                      return a list of nodes to delete. This list is exactly
                      the arguments that CM::DeleteNodes expects. If there
                      is nothing to do, it can return None or False.
  renew_slice         Renew a slice (and possibly its slivers).
  renew_sliver        Renew sliver at a CM using the current slice
                      expiration time.
  resolve_slice       Resolve a slice (SA::Resolve(type=slice)).
  resolve_sliver      Resolve a sliver at a CM.
  resolve_user        Resolve a GENI user (SA::Resolve(type=user).
  sliver_status       Get sliver status from a CM.
  sliver_status_details
                      Get status for all resources in a sliver from a CM.

Examples

First, test that things are working by getting a self credential (for a primer on credentials in GENI, see http://www.protogeni.net/ProtoGeni/wiki/ReferenceImplementationPrivileges, http://www.protogeni.net/wiki/SliceAuthorityAPI, and http://www.protogeni.net/ProtoGeni/wiki/Credentials):

$ esclient.py get_self_credential

If things work, you’ll see an XML blob that is the credential. If not, hopefully you’ll be able to make progress based on the error message (and you may find debug mode to be useful — supply the global -d argument and make sure to dump the stdout/stderr to a file):

$ esclient.py -d get_self_credential

Second, if you don’t already have a slice, create one. If you didn’t set the slicename option in your config file, you’ll need to pass -s <slice_URN> on the command line (an example URN is urn:publicid:IDN+emulab.net+slice+djdynslice2 — we’re creating the slice at the emulab.net SA; with a name of dydynslice2). You really want to stick it into the elasticslice.conf config file; if you don’t, you’ll have continue to supply it as an argument to nearly every command.

$ esclient.py create_slice

You’ll see a large blob of XML (a slice credential) returned on stdout if all goes well.

Once you have a slice (or if you already had one), make sure you can resolve it (and use the -j argument to pretty-print the result as a JSON blob):

$ esclient.py resolve_slice -j

Once you’ve created a slice, you have the “container” to allocate resources to, but you have no resources. You have two options with esclient.py to add resources. You can use the create_sliver subcommand, but that requires an RSpec (http://groups.geni.net/geni/wiki/GENIExperimenter/RSpecs, http://www.protogeni.net/wiki/RSpec).

$ esclient.py create_sliver -r etc/one-node-rspec.xml

(etc/one-node-rspec.xml is in the source tree, or online at https://gitlab.flux.utah.edu/elasticslice/elasticslice/tree/master/one-node-rspec.xml)

You can also use the managed_create_sliver subcommand, which uses an elasticslice.managers.core.ElasticSliceManager class, and a elasticslice.managers.core.ElasticSliceHelper class to obtain the initial RSpec for creating the sliver. Those classes are primarily abstract interfaces with some helper code; the defaults when using esclient.py are elasticslice.managers.core.SimpleElasticSliceManager and elasticslice.managers.core.SimpleElasticSliceHelper. These classes respect the num_pcs, num_lans, image_urn, multiplex_lans, tarballs, startup_command, nodetype, node_prefix, and lan_prefix config file options to control your RSpec generation (and to generate per-node arguments for subsequent managed_add_nodes commands). Anyway, you should do something like:

$ esclient.py managed_create_sliver

This may take awhile, but if it works, you’ll see a blob of XML on stdout — the sliver manifest (a manifest is a type of RSpec — see http://groups.geni.net/geni/wiki/GENIExperimenter/RSpecs). However, your nodes will not have finished booting for a few minutes. Check their status via

$ esclient.py sliver_status_details -l

There will be a one-line dictionary for each node in your slice. To pretty-print status a little better, you could do

$ esclient.py sliver_status_details -l -k status

which selects only the status key (in addition to the primary key, which is the component_urn (the Emulab node_id). All nodes should show a status of ready before you proceed.