BonFIRE logo and link to main BonFIRE site

Table Of Contents

Previous topic

OCCI/API via HTTP(cURL)

Next topic

Restfully

This Page

Command Line Interface Tools

The BonFIRE CLI tools are meant to provide users with a way to interact with the BonFIRE broker API from a command line interface. This usage might happen either manually or programatically (i.e. scripted).

Introduction

The structure of these tools is inspired by the command line interface offered by the OpenNebula toolkit, in the sense that individual tools exists to interact with resources of a particular type. These individual tools are:

  • bfexperiment - to interact with experiments
  • bfcompute - to interact with compute resources
  • bfstorage - to interact with storage resources
  • bfnetwork - to interact with network resources
  • bfsitelink - to interact with Autobahn sitelinks
  • bfrouter - to interact with Federica routers
  • bffednet - to interact with Federica networks
  • bfphysicalrouter - to retrieve information on physical routers available through Federica
  • bfreservation - to create or retrieve reservations
  • bflocation - to retrieve information on BonFIRE sites

The general usage pattern is the same for all these tools:

# <clitool> <command> <arguments ...>  [<options ...>]

where <clitool> is one of the tools mentioned above, <command> is one of {info|list|show|create|update|delete}, <arguments ...> are command specific arguments and [<options ...>] are additional options. Both generic that apply to all command, as well as command specific options exist. Options are - as the name implies - always optional.

The meaning of the individual commands is as follows:

  • info - display configuration settings and some system information for diagnostic purposes. The output of this command is the same for all tools and should be included when reporting any issues.
  • list - list the items in question. (translated to a HTTP GET request on the broker)
  • show - display information on a particular item (also corresponds to a GET)
  • create - create an item. (corresponds to a POST)
  • update- change one or mor parameters of an item (corresponds to PUT)
  • delete - delete a particular item (corresponds to DELETE)
  • ssh - Open an SSH connection to a BonFIRE compute resource. Only applicable to the bfcompute tool.

Download

The BonFIRE CLI Tools and related libraries can be downloaded from here.

Installation

Installation from source

After obtaining a copy of the source distribution, unpack the downloaded archive (unless you got a copy from SVN), change to its directory and, as the root user, say:

# python setup.py install

Note that when installing from source you will also need the BonFIRE Python Libraries package. It is installed in the same manner.

RPM Installation

Add the BonFIRE RPM repository to your system’s configuration and say:

# yum install bonfire-cli

Windows

Prerequisites

  • Python 2.6 or higher
  • Make sure that both the python interpreter executable (python.exe) and the python scripts directory (e.g. c:\python27\scripts) are on %PATH%

Installation

Follow the link above and download the relevant package (bonfire-cli-<version>-win32.exe). After the download finishes, run the downloaded file and follow the instructions on the screen.

Configuration

Upon startup, each tool will try to subsequently read configuration data from a number of locations. Any configuration key encountered during a later stage of this process will override any previously encountered instance of the same key. Any configuration parameters specified on the command line as options (see below) will override the respective configuration parameter specified in configuration files.

On UNIX like systems (including Linux and Apple), the following locations will be read (in that order):

  • /etc/default/bonfire (note that this file will automatically exist on every compute resource created through the broker API, meaning that the BonFIRE CLI tools will work on BonFIRE computes without further configuration)
  • /etc/bonfire
  • ~/.config/bonfire
  • ~/.bonfire

On MS Windows systems, the following locations will be read (in that order):

  • %APPDATA%\\bonfire
  • %USERHOME%\\.bonfire
  • %USERHOME%\\bonfire

Configuration parameters are specified on a single line in the form <KEY>="<VALUE>". Any line starting with the hash sign (#) is ignored. The following configuration keys will be recognized:

  • BONFIRE_URI - the URI of the BonFIRE broker
  • BONFIRE_CREDENTIALS - credentials to use when accessing the broker API in the form <username>:<password>

General Usage Notes

Generic Options

Note that the short form of all generic options consists of a lower-case letter, whereas the short form of tool specific options consists of an upper-case letter. The following is a list of options that are applicable to all tools:

  • --version - display version information and exit successfully.
  • -h, --help - display a brief usage summary and exit successfully.
  • -b <uri>, --broker-uri <uri> - URI of the BonFIRE broker API.
  • -u <username>, --username <username> - BonFIRE username.
  • -p <password>, --password <password> - BonFIRE password. Will be prompted for if unavailable. Note that specifying the password in this manner is strongly discouraged since it creates a security problem. You should rather consider to specify BONFIRE_CREDENTIALS in a configuration file.
  • -f <file>, --configfile <file> - additional config file location to read. Will be read after all builtin locations have been read
  • -q, --quiet - suppress any diagnostic output on stderr.
  • -s, --show - print requests to stdout before sending them to the broker.
  • -d, --dry-run - don’t actually send requests to the broker. Implies -s unless -q is given.

Abbreviations

When specifying relations with BonFIRE entities, a number of abbreviations can be used to make specifying IDs more convenient. To be exact, the following applies:

  • Location IDs can be specified either as their full ID (/locations/uk-epcc) or their short name (uk-epcc).
  • Experiments can be specified using either their full ID (/experiments/42) or their short ID (42).
  • Network and storage resources can be specified using either their full ID (/locations/uk-epcc/networks/42), their short ID (42) or their name (BonFIRE-WAN). Note that referring to a resource by name will be significantly slower than all other methods.
  • When specifying the URI to the Bonfire RM (through --broker-uri or -u), in addition to give the full URI (e.g. https://api.bonfire-project.eu) one can refer to the magic values %production, %qualification and %integration which point to the well known URIs of these particular BonFIRE infrastructures. These values can be abbreviated as %p, %q, and %i respectively.

Boolean Values

On some occasions, boolean values must/can be entered. Boolean value interpretation is not case sensitive. In Addition, the following applies:

  • The following values will be interpreted as true: true, t, yes, y, 1
  • The following values will be interpreted as false: false, f. no, n, 0

The info Command

The info command does not execute any options on the BonFIRE RM. Its purpose is to print information about the envorinment the BonFIRE CLI tools run in as well as its current configuration. It behaves exactly the same for all tools. It is therefore irrelevant which tool is used to invoke it.

Note

The output of bfexperiment info should always be included when reporting bugs in the CLI tools.

The following options apply to the info command:

  • -t <format>, --format <format> - Choose to write output either in a human readable way (plain) or in a way that is suitable for use as config file (config). Defaults to config format when outputting to a known config location and to plain format in all other cases.
  • -o <outfile>, --outfile <outfile> - Write output to this file. When outputting in config file format (--format config), the magic values %globalconfig and %userconfig can be used to refer to the well known config file location. These values can be abbreviated as %g and $u.
  • -n, --no-purge - Show plaintext passwords in output. Note that plaintext passwords will always be written when writing config format output to a file.
  • -r, --force - Force execution even when trying to overwrite a file or trying to write plain output to a known config file.

Tip

A combination of the options above allows us to conveniently write a configuration file without having to use a text editor or other peasantly means:

$ bfexperiment info --user cliuser --outfile %userconfig --broker-uri %p --verbose
BonFIRE password:
[bonfire.cli] INFO: Writing output in config format to /home/demouser/.config/bonfire

Or in an equivalent but shorter way:

$ bfexperiment info -u cliuser -o %u -b %p
BonFIRE password:

Both these commands configure the CLI tools to use the RM on BonFIRE’s production infrastructure and the username cliuser. Since no password is specified, it is prompted for. The resulting confiugration is written to the default user configuration file. Subsequently, the CLI tools can be used without having to specify this information again.

Output

All BonFIRE CLI tools write any data that is the result of the specified operation to stdout. All additional (diagnostic and error) information is wrtten to stderr. It is therefore safe (and recommended) to redirect stdout as appropriate when data should be stored in a file and/or further processing by another program should take place.

Controlling Output

Information that is produced by the show and list commands can be controlled in various ways. The following options are always applicable to these commands, regardless of which tool is invoked:

  • -t <format>, --format <format> - Format in which to show output. Possible values are plain (a human readable representation), csv (comma separated values) and xml (The raw XML representation coming from the RM). Defaults to plain.
  • -n <num>, --num <num> - Number of times to retrieve information. Set to 0 to poll indefinitely. Defaults to 1.
  • -d <delay>, --delay <delay - Time to sleep between polling attempts in seconds. Obviously only meaningful when --delay is not 1.
  • -k, --keep-going - Keep going in face of errors encountered during polling.
  • -a, --no-labels - Do not show field labels in csv and plain output.
  • -r, --no-decorations - Suppress decorations in plain output.
  • -s <separator>, --separator <separator> - Separator to use in csv and plain output. Defaults to , for csv output and | for plain output.

In addition, the following option is applicable to the list command:

  • -e, --details - By default the output of the list command will only contain the ID and name of listed entitied. Specifying this option will cause additional information to included in the output. This option has no impact for xml output. Note that this may cause the operation to take significantly longer to complete.

Tip

In combination with any readily available xpath tool, we can use the XML output option to easily apply xpath expressions to filter results, simply by piping the output of the CLI tool into an xpath tool. For example, the following expression will select all compute resources that implement BonFIRE services (e.g. monitoring aggregator or elasticity engine):

$ bfcompute list -t xml | xpath '/items/compute[starts-with(@name,"BonFIRE")]'

Note that common xpath expressions tend to contain chracters that have special meanings in a UNIX shell (like parenthesis or quotation marks). Thus, it is necessary to either escape them with a backslash or surround the whole expression in single ticks (') as seen above.

Many xpath implementations exist, a particularly comprehensive specimen is contained in perl’s XML-XPath package: http://search.cpan.org/dist/XML-XPath/

Working with Experiments (bfexperiment)

The bfexperiment tool can be used to query and manipulate existing experiments as well as to create new ones. Its usage is as follows:

Creating Experiments

# bfexperiment create <name> [-D <description>] [-W <walltime>] [-G <group> ...]

The following positional argument must be specified when creating experiments:

  • <name> - The name for this experiment.

The following options are applicable when creating experiments:

  • -D <description>, --description <description> - description of the experiment. Defaults to “<no description>”.
  • -W <walltime>, --walltime <walltime> - lifetime of the experiment in seconds. Defaults to one day.
  • -G <group>, --group <group> - A user group this experiment will be accessible by. This option can be specified multiple times.

Updating Experiments

# bfexperiment update <id> -S {running|stopped}

The status of an experiment can be updated through the -S option:

  • -S {running|stopped}, --status {running|stopped} - Set the status of an existing experiment to either running or stopped.

Working with Sites (bflocation)

The bflocation tool can be used to query existing BonFIRE locations. Only the show and list commands are available for locations.

Working with Storages (bfstorage)

The bfstorage tool can be used to query and manipulate existing storage resources as well as to create new ones. Its usage is as follows:

Creating Storages

# bfstorage create <name> <location> [<experiment>] [-D <description>] [-S <size>] [-T {datablock|shared}] [-F <fstype>] [-G <group> ...] [-P] [-R]

The following positional arguments must be specified when creating storages (in that order):

  • <name> - The name for this resource.
  • <location> - The BonFIRE site where this resource will be created.

Optionally, an experiment this resource will be part of can be specified as a third positional argument. The following options are applicable when creating storages:

  • -D <description>, --description - The description of this storage resource. Defaults to “<no description>”.
  • -S <size>, --size - Size of this storage resource in MiB. This is only applicable to datablock resources. Defaults to 1024 MiB.
  • -T {datablock|shared} - The type of this storage resource. Note that “shared” storages are only available on be-ibbt. Defaults to “datablock”.
  • -F <filesystem>, --fstype <filesystem> - The filesystem the storage resource will be formatted with. This is only applicable to datablock resources. Defaults to “ext3”.
  • -G <group>, --group <group> - A user group this resource will be accessible by. This option can be specified multiple times.
  • -P, --public - Indicates that this storage resource should be publicly available. Defaults to <false>.
  • -R, --persistent - Indicates that a persistent storage resource should be created. Defaults to <false>.

Updating Storages

# bfstorage update <id> -P {true,false,yes,no,1,0,t,f} -R {true,false,yes,no,1,0,t,f}

The following options are applicable when updating storages:

  • -P {true,false,yes,no,1,0,t,f}, --public {true,false,yes,no,1,0,t,f} - Changes the “public” flag of this storage resource.
  • -R {true,false,yes,no,1,0,t,f}, --persistent {true,false,yes,no,1,0,t,f} - Changes the “persistent” flag of this storage resource.

Working with Networks (bfnetwork)

The bfnetwork tool can be used to query and manipulate existing network resources as well as to create new ones. Its usage is as follows:

Creating Networks

# usage: /usr/bin/bfnetwork create <name> <location> <experiment> [-A <address>] [-S {A,B,C}] [-X <prefix>] [-N <netmask>] [-V <vlan>] [-P] [-L <latency<] [-R <lossrate>] [-B <bandwidth>] [-C {TCP,UDP}] [-K <packetsize>] [-U <throughput>] [-T {managed,active}] [-G <group> ...]

The following positional arguments must be specified when creating storages (in that order):

  • <name> - The name for this resource
  • <location> - The BonFIRE site where this resource will be created.
  • <experiment> - The experiment this resource will be part of.

The following options are applicable when creating network resources. Note that at present, networks can only be created on be-ibbt and that all options except -G are only applicable to networks on be-ibbt.

  • -A <address>, --address <address> - Network address for this network resource. Applicable to networks on ONE and VW sites. (default: 192.168.0.0)
  • -X <prefix>, --prefix <prefix> - Network size specified as CIDR prefix. (default: 24)
  • -S {A|B|C}, --size {A|B|C} - Network size specified as network class. Only applicable to VW sites. (default: C)
  • -N <netmask>, --netmask <netmask> - Network size specified as bit mask. Applicable to ONE sites. (default: 255.255.255.0)
  • -L <latency>, --latency <latency> - Specifies the network latency in ms with the minimum non-zero value being 2ms.
  • -R <lossrate>, --lossrate <lossrate> - Specifies the lossrate for this network as a floating point number in the range 0..1.
  • -B <bandwidth>, --bandwidth <bandwidth> - Specifies the network bandwidth in Mbit/s.
  • -C {TCP,UDP}, --protocol {TCP,UDP} - Type of generated background traffic. Specifying this implies the creation of an active network and thus requires –packetsize and –throughput as well. Only applicable to networks on VW. (default: TCP)
  • -K <packetsize>, --packetsize <packetsize> - Packetsize of generated background traffic in bytes. Specifying this implies the creation of an active network and thus requires --protocol and --throughput as well. Only applicable to networks on VW.
  • -U <throughput>, --throughput <throughput> - Number of generated background traffic packets per second. Specifying this implies the creation of an active network and thus requires --packetsize and --protocol as well. Only applicable to networks on VW.
  • -T {managed,active}, --type {managed,active} - Force the type of this network to be either managed or active. If this option is missing the type will be derived from other options. Only applicable to networks on VW.
  • -P, --public - Indicates that this network resource should be publicly available.
  • -G <group>, --group <group> - A user group this resource will be accessible by. This option can be specified multiple times.

Updating Networks

# bfnetwork update <id> [-L <latency>] [-R <lossrate>] [-B <bandwidth>]

The following options are applicable when updating network resources. Note that at present, only networks form be-ibbt can be updated.

  • -L <latency>, --latency <latency> - Specifies the network latency in ms with the minimum non-zero value being 2ms.
  • -R <lossrate>, --lossrate <lossrate> - Specifies the lossrate for this network as a floating point number in the range 0..1.
  • -B <bandwidth>, --bandwidth <bandwidth> - Specifies the network bandwidth in Mbit/s.

Working with Computes (bfcompute)

The bfcompute tool can be used to query and manipulate existing compute resources as well as to create new ones. Its usage is as follows:

Creating Computes

# bfcompute create <name> <os_image> <experiment> [-I <instance type>] [-V <vcpu>] [-P <cpu>] [-M <memory>] [-S <storages>...] [-N <networks>...] [-R <cluster>] [-H <host>] [-C <context_name>:<context_value>] [-L <location>] [-G <group> ...]

Note that contrary to all other resources, compute resources can be created without explicitly specifying their location. That is because their location will derived from either the OS image, any other involved storage resources and any involved network resources (in that order). Only when this is not possible (because all involved networks and storages were specified in an abbreviated form), specifying the location (through the -L option) is neccessary.

The following positional arguments must be specified when creating computes (in that order):

  • <name> - The name for this resource.
  • <os_image> - Storage resource to use as OS image.
  • <experiment> - The experiment this resource will be part of.

The following options are applicable when creating compute resources:

  • -I <instance type>, --instance_type <instance type - Instance type to use when creating the compute resource. (default: “small”)
  • -V <vcpu>, --vcpu <vcpu> - Number of virtual CPUs present within the compute resource. Only applicable for the “custom” instance type. (default: 1)
  • -P <cpu>, --cpu <cpu> - The amount of CPU resources to allocate. Must be a non-negative, non-zero floating point value. Only applicable for the “custom” instance type. (default: 1)
  • -M <memory>, --memory <memory> - The amount of memory to allocate in MiB. Only applicable for the “custom” instance type. (default: 512MiB).
  • -S <storage>, --storage <storage> - An additional storage resource to attach. Can be specified either as a full id (/locations/uk-epcc/storages/42), short id (42) or name (My-Storage). This option can be specified multiple times.
  • -N <network>, --network <network> - A network resource to attach. Can be specified either as a full id (/locations/uk-epcc/networks/42), short id (42) or name (BonFIRE-WAN). This option can be specified multiple times.
  • -R <cluster>, --cluster <cluster> - The cluster to deploy this compute resource on.
  • -H <host>, --host <host> - The host to deploy this compute resource on.
  • -C <context>, --context <context> - Specifies a contextualization parameter in the form “KEY=VALUE”. This option can be specified multiple times.
  • -L <location>, --location <location> - Location where this resource will be deployed. If this is missing, the location will be derived from other onvolved resources.
  • -G <group>, --group <group> - A user group this resource will be accessible by. This option can be specified multiple times.

Updating Computes

# bfcompute update <id> [-S {resume,suspended,shutdown}] [-A <save_as target>]

The following options are applicable when updating compute resources:

  • -S {resume,suspended,shutdown}, --state {resume,suspended,shutdown} - Changes the state of the compute resources.
  • -A <save_as target>, --save_as <save_as target - Sets the “save_as” target for this compute resource.

Working with Federica Routers (bfrouter)

# bfrouter create <name> <host> <location> <experiment> <interface> [<interface>...] [-C <config>] [-G <group>...]

Working with Federica Networks (bffednet)

# bffednet create <name> <location> <experiment> <network_link> [<network_link> ...] [-G <group>...]