Getting started with the elasticslice client ============================================ We've created a simple yet powerful client (``bin/esclient.py``) that can create a slice for you, keep it renewed, and add/delete nodes from it based on some simple policy knobs and configuration. We'll show you how to do that below. Overview -------- ``esclient.py`` was originally designed as a daemon process that would run forever, managing the needs of an elastic slice --- creating it, renewing it periodically, and adding and removing nodes according to a policy set by the user (which is ideally a combination of the slice's resource demands and currently-available resources). This daemon process spawns multiple, synchronized threads to do its work, and leaves the python console in interactive mode so that an administrator can control the daemon trivially. To run in daemon/interactive mode, do something like ``$ path/to/bin/esclient.py [OPTIONS] interactive`` (Recall that esclient.py and the elasticslice library use configuration options first from the command line, then falling back to the same options if present in your ``~/.protogeni/elasticslice.conf`` file.) However, ``esclient.py`` also supports individual commands covering nearly all of its functionality, in a subcommand style. Below we show a partial help listing (obtained via ``esclient.py -h`` --- global option details excluded for brevity --- make sure to run it yourself to see the most current information):: usage: esclient.py [-h] [-V] [-d] [--config CONFIG_FILE] [-e ENDPOINT] [-s SERVER] [-p PORT] [-c CERT] [--passphrasefile PASSPHRASEFILE] [-C CACERT] [--ext_addr EXT_ADDR] [--ext_port EXT_PORT] [-U USERNAME] [-P PASSWORD] [-A] [-M] [--am-percent-available-minimum PERCENT_AVAILABLE_MINIMUM] [--am-min-threshold MINTHRESHOLD] [--am-max-threshold MAXTHRESHOLD] [--am-nodetype NODETYPE] [--am-enable-delete] [-D DEVUSER] [-S SLICENAME] [--default-cm DEFAULT_CM] [-H] [--cmuri CMURI] [--sauri SAURI] [--fscache] [--fscachedir FSCACHEDIR] [--image_urn IMAGE_URN] [--manager_class MANAGER_CLASS] [--helper_class HELPER_CLASS] [--clientserver_endpoint_class CLIENTSERVER_ENDPOINT_CLASS] [--node_delete_wait_time NODE_DELETE_WAIT_TIME] {interactive,add_nodes,create_slice,create_sliver,delete_nodes, delete_slice,delete_sliver,get_hostname_and_port,get_keys, get_manifest,get_resources,get_self_credential, get_slice_credential,get_slice_expiration,get_sliver_credential, get_sliver_urn,get_user_email,get_user_uid,get_user_urn, managed_add_nodes,managed_create_slice,managed_create_sliver, managed_ensure_slice,managed_ensure_sliver, managed_should_add_nodes,managed_should_delete_nodes, renew_slice,renew_sliver,resolve_slice,resolve_sliver, resolve_user,sliver_status,sliver_status_details} ... positional arguments: {interactive,add_nodes,create_slice,create_sliver,delete_nodes,delete_slice, delete_sliver,get_hostname_and_port,get_keys,get_manifest,get_resources, get_self_credential,get_slice_credential,get_slice_expiration, get_sliver_credential,get_sliver_urn,get_user_email,get_user_uid,get_user_urn, managed_add_nodes,managed_create_slice,managed_create_sliver, managed_ensure_slice,managed_ensure_sliver,managed_should_add_nodes, managed_should_delete_nodes,renew_slice,renew_sliver,resolve_slice, resolve_sliver,resolve_user,sliver_status,sliver_status_details} Subcommands interactive Run in daemon/interactive mode add_nodes Add nodes to a sliver at a CM. create_slice Create a slice. create_sliver Create a sliver at a CM. delete_nodes Delete nodes from a sliver at a CM. delete_slice Delete a slice. delete_sliver Delete a sliver at a CM. get_hostname_and_port Get a hostname and SSH port for the given `client_id`. get_keys Get user's keys from the SA (SA:GetKeys). get_manifest Get a sliver manifest from a CM get_resources Get a resource advertisement from a CM. get_self_credential Gets a self credential (SA::GetCredential). get_slice_credential Get a slice credential (SA::GetCredential). get_slice_expiration Get expiration time for slice get_sliver_credential Get a sliver credential (CM::GetSliver()). get_sliver_urn Get a sliver URN at a CM. get_user_email Get your user email. get_user_uid Get your user uid. get_user_urn Get your user URN. managed_add_nodes This function manages the addition of nodes. This manager will call it to add nodes to this slice, as necessary. managed_create_slice Create the slice from within the manager, using information from the ElasticSliceHelper associated with this ElasticSliceManager. managed_create_sliver Create the sliver from within the manager, using information from the ElasticSliceHelper associated with this ElasticSliceManager. managed_ensure_slice Check to see if the slice exists; if not, create it. managed_ensure_sliver Check to see if the sliver exists; if not, create it using information from the ElasticSliceHelper associated with this ElasticSliceManager. managed_should_add_nodes By default, add_nodes calls this method to find out if it should add any nodes. If this function returns an integer, add_nodes interprets that value as the number of new nodes to add, and it calls get_add_args() to get the arguments to pass to _add_nodes. If the function returns a dict, it passes that directly to _add_nodes. If the function returns None, False, nothing is added. This gives subclasses enough flexibility, hopefully. If you pass a nonzero, positive integer via `count`, this method must check if it should add exactly `count` nodes, or not, and return accordingly. managed_should_delete_nodes Should we delete a node? If so, this function must return a list of nodes to delete. This list is exactly the arguments that CM::DeleteNodes expects. If there is nothing to do, it can return None or False. renew_slice Renew a slice (and possibly its slivers). renew_sliver Renew sliver at a CM using the current slice expiration time. resolve_slice Resolve a slice (SA::Resolve(type=slice)). resolve_sliver Resolve a sliver at a CM. resolve_user Resolve a GENI user (SA::Resolve(type=user). sliver_status Get sliver status from a CM. sliver_status_details Get status for all resources in a sliver from a CM. Examples -------- First, test that things are working by getting a self credential (for a primer on credentials in GENI, see http://www.protogeni.net/ProtoGeni/wiki/ReferenceImplementationPrivileges, http://www.protogeni.net/wiki/SliceAuthorityAPI, and http://www.protogeni.net/ProtoGeni/wiki/Credentials): ``$ esclient.py get_self_credential`` If things work, you'll see an XML blob that is the credential. If not, hopefully you'll be able to make progress based on the error message (and you may find debug mode to be useful --- supply the global ``-d`` argument and make sure to dump the stdout/stderr to a file): ``$ esclient.py -d get_self_credential`` Second, if you don't already have a slice, create one. If you didn't set the ``slicename`` option in your config file, you'll need to pass ``-s `` on the command line (an example URN is ``urn:publicid:IDN+emulab.net+slice+djdynslice2`` --- we're creating the slice at the emulab.net SA; with a name of ``dydynslice2``). You really want to stick it into the ``elasticslice.conf`` config file; if you don't, you'll have continue to supply it as an argument to nearly every command. ``$ esclient.py create_slice`` You'll see a large blob of XML (a slice credential) returned on stdout if all goes well. Once you have a slice (or if you already had one), make sure you can resolve it (and use the ``-j`` argument to pretty-print the result as a JSON blob): ``$ esclient.py resolve_slice -j`` Once you've created a slice, you have the "container" to allocate resources to, but you have no resources. You have two options with ``esclient.py`` to add resources. You can use the ``create_sliver`` subcommand, but that requires an RSpec (http://groups.geni.net/geni/wiki/GENIExperimenter/RSpecs, http://www.protogeni.net/wiki/RSpec). ``$ esclient.py create_sliver -r etc/one-node-rspec.xml`` (``etc/one-node-rspec.xml`` is in the source tree, or online at https://gitlab.flux.utah.edu/elasticslice/elasticslice/tree/master/one-node-rspec.xml) You can also use the ``managed_create_sliver`` subcommand, which uses an :class:`elasticslice.managers.core.ElasticSliceManager` class, and a :class:`elasticslice.managers.core.ElasticSliceHelper` class to obtain the initial RSpec for creating the sliver. Those classes are primarily abstract interfaces with some helper code; the defaults when using ``esclient.py`` are :class:`elasticslice.managers.core.SimpleElasticSliceManager` and :class:`elasticslice.managers.core.SimpleElasticSliceHelper`. These classes respect the `num_pcs`, `num_lans`, `image_urn`, `multiplex_lans`, `tarballs`, `startup_command`, `nodetype`, `node_prefix`, and `lan_prefix` config file options to control your RSpec generation (and to generate per-node arguments for subsequent ``managed_add_nodes`` commands). Anyway, you should do something like: ``$ esclient.py managed_create_sliver`` This may take awhile, but if it works, you'll see a blob of XML on stdout --- the sliver manifest (a manifest is a type of RSpec --- see http://groups.geni.net/geni/wiki/GENIExperimenter/RSpecs). However, your nodes will not have finished booting for a few minutes. Check their status via ``$ esclient.py sliver_status_details -l`` There will be a one-line dictionary for each node in your slice. To pretty-print status a little better, you could do ``$ esclient.py sliver_status_details -l -k status`` which selects only the `status` key (in addition to the primary key, which is the `component_urn` (the Emulab `node_id`). All nodes should show a status of ``ready`` before you proceed.