Boscli Oss

Boscli is a framework to create interactive command line interfaces. Our mission is to help developers write interactive shells for domain specific task. It provides an easy way to wrap and join together command line apps and utilities and create specific shells with security levels and modes, history, auto-completion and so on.

Using this framework any developer can build an interactive command line interface for configuration or monitor an appliance in a few minutes.

The Boscli can be used as a great "glue code" for appliance's interfaces or to create a "homogeneous" administration interface for different systems and applications. For example if you need to integrate different services and applications (asterisk, linux conf, monit, apache, etc...) but you want to offer to your users or to the systems administrator an integrated line oriented interface, you can use Boscli as a framework to create thin wrappers around the apps/services you want to integrate and create one or more boscli shells. All the commands are defined as a Boscli plugin that can be reused for different shells in different configurations, so with the same code you can create different shells, for different users or for different task (initial conf, admon, etc....)

The main features are:

  • For developers:
    • Integrated plugin system
    • Easy new plugin creation
      • Easy new command definition
      • Easy new types definition
    • Auto generated help
    • Basic types support
    • Function access control support
    • Function mode support
    • Command execution Log
  • for final shell users
    • Command Autocompletion
    • Interactive line edition
    • Contextual Help
    • Pagination
    • Filters
    • Command history

Version History

boscli-oss-core 0.2.2

  • support autocompletion / interactive help, even if the user use abbreviations
  • Restructured svn dirs

boscli-oss-core 0.2.1

  • Bugfix
  • Normal parameters for bcli functions
  • Improved docs

boscli-oss-core 0.2.0

  • Debian package rewrite (using python-support)
  • Cleanups and code reestucture
  • Added tutorial
  • Improved docs

boscli-oss-core 0.1.9

  • Initial release

Predefined boscli-oss-core commands

  • Commands for define alias.
    • alias define <STRING> <STRING> Define/Overwrite an alias.
    • alias list Show defined aliases and the corresponding values.
    • alias remove <ALIAS> Remove the assigned value for an alias.
  • Commands for interact with the BosCli itself.
    • cli exec <FILE> Execute a BosCli file.
    • cli history Show comand history.
    • cli history clear Clear/Reset command history.
    • cli pager <STATE> Activate or Deactivate the command output pager.
    • cli pager Show if the pager is active or not.
    • cli quit Exit from BosCli.
    • cli reload Reload all the functions registered at CLI.
  • Commands related to privilege level or boscli mode (normal/configure)
    • configure Change mode to configuremode
    • configure terminal Change mode to configuremode
    • disable Change privileges to 0 (initial level)
    • enable Change privileges to enable level (level 1)
    • manufacturer Change privileges to manufacturer level (level 2)
    • passwd Change login password.
    • passwd priv enable Change configure enable privilege level password.
    • passwd priv manufacturer Change configure manufacturer privilege level password.
    • privilege initial enable Set enable level as initial privilege level for actual user
    • privilege initial manufacturer Set manufacturer level as initial privilege level for actual user
    • privilege initial normal Set normal level as initial privilege level for actual user
  • Commands for interactive help.
    • help basic Show basic help. Same as '?<tab>' at command lines
    • help command <CMD> Show short help of the (active) functions corresponding to the command specified.
    • help extended command <CMD> Show extended help of the (active) functions corresponding to the command specified.
    • help show allcommands Show a list of all available commands (only for active mode).
    • help show allcommands full Show basic help for all available commands (only for active mode).
    • help types Show help for type/vars used in CLI commands.
    • remotehelp act Activate remote help mode.
    • remotehelp connect Connect to an active remote help session.
  • exit Exit from configure mode or from CLI (if we are at NORMAL mode).

Predefined boscli-oss-core types

The creation of new types is very easy. Anyway, we have a few basic type already defined at boscli-oss-core that we can use for our commands.

The types already defined are:

  • ALIAS => Doc to be defined
  • BOOL => Conditional value. Accepted values 'true', 'false', '1', '0'
  • CIDR => Address in Classless Inter-Domain Routing notation: Ex: 192.168.1.100/24
  • CMD => Cli available functions
  • FILE => File name with absolute path
  • FLOAT => Decimal number
  • HOST => Host name. Ex: www.google.com, bif-shp1
  • INT => Integer value. Ex: 42
  • IP => IP Address in dot-decimal notation: Ex: 192.168.1.100
  • IPHOST => Host name or IP Address. Ex: 192.168.10.200, www.google.com
  • KERNELVAR => Kernel variable. Ex: net.unix.max_dgram_qlen or fs.mqueue.msgsize_max
  • MAC => MAC or Ethernet Address in hex notation. Ex: 00:01:02:03:04:08
  • NET => Doc to be defined
  • NETIF => Ethernet interface. Ex: eth0, eth1, eth0:0
  • NETMASK => Netmask in Dot-decimal Address. Ex: 255.255.255.0
  • STATE => State value. Accepted values 'on', 'off', 'up', 'down'
  • STRING => Sequence of letters, digits or '_'. Ex: Xema
  • WEBURL => Web URL, Ex: http://www.debian.org or https://www-nasa.gov

Integrated plugin system

The core of the Boscli is a little interpreter that has the responsibility to load and import the user plugins that have the real implementation for the commands we want to expose to the user. Even the predefined bunch of commands (See Already defined cli functs) are loaded from a plugin (exact like the rest of the user plugins)...

A Boscli plugin is only a directory that contains one ore more Boscli python files or boscli file extensions. The base directory for a user plugin is: /usr/lib/boscli/plugins/ The predefined commands and types are defined at the system plugin dir /usr/lib/boscli/lib/.

To start a boscli shell we must pass all the plugins paths (in order) that the Boscli core app should read and import, so we in fact can use any readable path in the system to use as plugin dir, but we recommend to follow the /usr/lib/boscli/plugins/ convention.

As we referred previously, a plugin can contains python files and boscliutils files. When the boscli initializes a plugin it reads all the *.py and *.boscli files in the plugin directory and imports the files. For python files, the file is evaluated using the boscli directory as base python path, so the plugin python file can directly import all the boscli stuff and can define new types and new boscli commands.

For an example on how develop a plugin see the tutorial at BoscliOssTutorial

Defining new Boscli commands

When the boscli load/import a python file from a plugin directory, it automatically register all the functions detected that begins with 'bcli_' string as a boscli command. The boscli splits the function name with the following rules:

  • The initial 'bcli_' string is remove (is only a tag to indicate that this function is a boscli command).
  • The separator is the '_' character.
  • If the word is in lower case, it is considered a literal word for the command.
  • If the word is in upper case, it is considered a parameter of the type indicated by the word.

For example:

def def bcli_cli_history():
    pass

def bcli_help_command_CMD(param_cmd):
    pass

def bcli_cli_pager_STATE(param_state):
    pass

def bcli_alias_define_STRING_STRING(param_string1, param_string2):
    pass

This four functions define the the following commands at the boscli:

  • cli history
  • help command_<CMD>
  • cli pager <STATE>
  • alias define <STRING> <STRING>

To implement the boscli commands and the boscli types the plugin python file can use the following boscli helpers and exported functions:

  • From boscliutils:
    • BatchCommand Execute a shell command connecting the output to the boscli filter system
    • InteractiveCommand Execute a shell command as a interactive command with interfering in the input/output of the command. To be used to execute shell commands that takes control of the terminal (interactive libncurses apps, full terminal editors, etc...)
    • Log related functions:
      • Log.debug log a debug message. This message only will be logged if the boscli is started with -d option.
      • Log.error log a error message. Include the traceback.
      • Log.info log a info message.

Use examples:....

import boscliutils


def bcli_util_tracepath_IPHOST(host):
    """Show tracepath to the host
    """
    print "Press Ctrl-C to quit"
    print ""
    boscliutils.BatchCommand("tracepath %s" % host)


def bcli_util_edit_FILE(file):
    """Edit file"""
    boscliutils.InteractiveCommand("nano %s" % file)

During the execution, there is always a unique instance of the boscli interpreter (object of BiferShell class). This instance has all the boscli commands registered, with the access level and other interesting stuff. For the implementation of some commands we need to access this instance using the boscli.get_cli function. We can use the following methods of the boscli interpreter instance:

  • register_exit_funct Register a function to be executed when the boscli exits.
  • get_functions Return the list of registered functions at boscli.
  • get_active_functions Return the list of active functions at boscli in this moment (in the actual mode and with the actual privilege level).
  • get_extension_paths Return the list of active paths used for loading plugins.
  • set_privilege Set the minimum privilege level and necessary mode to execute a function.
import boscliutils

def example_plugin_exit():
    print "exiting"

get_cli().register_exit_funct(example_plugin_exit)

get_cli().set_privilege(privileges.ENABLE, privileges.NORMAL, 'bcli_explugin_enable_cmd')

def bcli_explugin_enable_cmd():
    """Example command for enable mode"""
    pass

Defining new Boscli types

To create a new type definition in our extension, we need to define a new class derivate from bostypes.BiferShellType, and implement validate_value and values method.

The following example defines a type that returns the names of the sites defined at apache2 configuration (each site definition is a file at '/etc/apache2/sites-enabled/' dir.

The constructor __init__ must define the type help string, and pass it to the bostypes.BiferShellType constructor. The values method must return a string list with all the correct values that can be used for a parameter of this type. The values method can use the imcomplete_word parameter to decide which values should be returned. When the boscli engine must complete a value for this type, it calls the method values passing the actual word text as incomplete_word. Even if the method doesn't check the incomplete_word parameter, the boscli engine will filter the list returned, and it will only use the values returned that start with the incomplete_word.

In this case, the values method return all the file names (without the base path) found at '/etc/apache2/sites-enabled/' dir.

validate_value method must return if a value is valid. Usually we accept a value if it is in the list of values returned by the values method. Some times, even if a value is not returned by values method, we can consider it valid. For example we can have a boscli type that return some IP addresses, but we want to accept any IP. The bostypes.BiferShellType validate_value implementation always return True. So usually we should implement this method.

class BiferShellSiteEnabled(bostypes.BiferShellType):
    def __init__(self): 
        bostypes.BiferShellType.__init__(self, "Enabled sites name")
    def validate_value(self, value):
        return value in self.values('')    
    def values(self, incomplete_word):
        sites = [os.path.basename(d) for d in glob.glob('/etc/apache2/sites-enabled/*')]
        return sites

The new type must be registered at the boscli engine, so we call add_type, with the name type for use at command definitions, and an object of the new type.

get_cli().add_type('SITEENABLED', BiferShellSiteEnabled())

Now, we can use this new type for new command definition. For example:

def bcli_apache_disable_SITEENABLED(site):
    """Disable apache2 site (and reload conf)
    """
    print "Disabling apache2 site %s" % site
    boscliutils.apache_disable_site(site)

More options for create a new types

If the new type we want to create, only need a fixed list of choices or options, is better to use the predefined BiferOptionsType. We can register the new type with all the available options directly.

Example:

get_cli().add_type('TRANSPORT', BiferOptionsType(['tcp', 'udp'], "Transport protocols supported"))

The previous line register a new type TRANSPORT that can have the values tcp and udp. The BiferOptionsType contructor parameters are:

  • The list of available options for the type.
  • The short doc used for interactive help.

If we can define a function for return the available options and we don't want to change the default behaviour of the other methods BiferShellType we can use the predefined class bostypes.BiferGenericType that accept as constructor parameters:

  • The function to generate the valid values for the type. Can be a lambda function.
  • The short doc used for interactive help. If we don't use this parameters, the function doc would be used as short doc for the type.

Example:

def lang_selector(incomplete_word):
    langs = ['english', 'spanish', 'Dutch', 'Danish'] 
    return [l for l in langs if l.find(incomplete_word) != -1]

get_cli().add_type('LANG', BiferGenericType(lang_selector, "Supported languages"))

Example with a lambda function:

get_cli().add_type('LOWERWORDS', BiferGenericType(lambda incomplete_word: incomplete_word.lower(), "Lower case words"))

Defining new Boscli Extension Files

In order to define boscli commands that only use normal shell commands but with concrete parameter or using boscli defined types, we can use boscli files instead of defining a boscli python extension.

This filenames must end with the boscli extension, for example cmdsystem.boscli. It contains one or more command definitions in a .ini format

Example:

[cmd]
cmd=sys show so version
shell=cat /etc/lsb_release
doc=Show SO release 

[cmd]
cmd=sys show kernel version
shell=uname -r
doc=Show kernel version

[cmd]
cmd=sys htop
shell=htop
doc=Interactive process viewer
interactive=yes
priv=ENABLE

Each command is defined in its own command section ([cmd]). In the command section you always must define the following keywords:

  • cmd The command name used at the boscli
  • shell The line to execute at shell prompt (it can use pipes and redirections)
  • doc The short doc showed when help is requested for this command at boscli

There are optional keywords like:

  • priv
  • interactive

priv define the minimal privileges level required for use the command. The accepted values are NONE, ENABLE, MANUF. The default value is NONE.

interactive is a boolean var that defines how the shell command should be executed. The values are:

  • 'interactive = yes' The shell command is executed using boscliutils.InteractiveCommand, so the boscli execute the command without interfering in the standard input/output of the subprocess. This mode is interesting for interactive commands as top, or full term editors, and so on
  • 'interactive = no' The shell command is executed using boscliutils.BatchCommand. In this case the subprocess is supposed to work by line so the boscli will interfere on the output, for example for filter (using begin, include, exclude boscli filters). This is the default

Commands with parameters

The commands defined in boscli files can use parameters using the boscli defined types. For example if we have defined a IPSHAPER type for the boscli (using a python boscli extension) we can use this type in our commands definition:

[cmd]
cmd=shaper dumptraff host:IPSHAPER
doc=Show host traffic real time
shell=tshark -i any -f \"host ${host}\"

The parameter is defined using a lower case parameter name and the upper case type name separated by ':'. In the example we define a parameter 'host' using the type 'IPSHAPER' that is defined in an extension. We can use the parameter value, using ${<paramname>} at the shell line. So in the above example the ${host} will be substituted by the host value introduced by the user.

User input:

shaper dumptraff laptop

Shell command executed:

tshark -i any -f "host laptop"

Of course, we can use all the parameters we need, using our type definitions, or the boscli included type definitions (as INT, IP, HOST, etc)

[cmd]
cmd=shaper dumptraff host:IPSHAPER port:INT
doc=Show host/port traffic real time
shell=tshark -i any -f \"host ${host} and port ${port}\"
priv = MANUF

[cmd]
cmd=shaper tcview iface:NETIF
doc=Show tc simple tc throughtput
shell=/opt/bos/bin/tc-viewer --zero --iface ${iface} --unit=kbit
priv = MANUF

Creating a Boscli Shell

At this point we know how to define plugins with boscli extensions, with new types and commands. We only need a final step, create our boscli shell joining all the stuff and creating a shell script for starting our shell.

The shell script should call boscli app with parameter for the boscli shell name, the config file (See BoscliOss#Deactivatealreadydefinedclifuncts) and all the plugins paths. If we want the predefined functions we must include /usr/lib/boscli/lib/ as a plugin path. This standard plugin include all the code for help support, internal cli commands, and privileges control (See BoscliOss#Alreadydefinedclifuncts)

The boscli app have the following command line arguments (boscli -h):

Use mode: boscli [OPTIONS] [extdir1 extdir2 ...]
Open a interactive Command Line Interface/Interpreter 

The valid options are:
(The mandatory argument for "long format" options are also 
mandatory for the correspondent sort option)
      
  -h, --help 
      show this help
  -a, --name
      boscli instance name. Used for separate conf/history files,
      and to identify log messages.
  -c, --conf = path
      load the specified file as configuration. The configuration
      file is used basically to deactivate already defined boscli
      functions
  -n, --nopasswd
      don't ask for the password when change to a "mode" if we are root.
  -f, --file = path 
      read and execute the commands in this file before enter 
      in interactive mode.
  -v, --verbose 
      log more info to system log
  -d, --debug 
      log debug messages to system log

For an example of final boscli start script see BoscliOssTutorial#Createascripttostarttheshell

Deactivate already defined cli functs

When we develop a new cli, usually we want a lot of the predefined boscli-oss functions, but not all. For deactivating some predefined boscli-oss functions, we should use the section refusedfunctions configuration file. In this section we can define a regular expression that match the functions we want to deactivate. For example, if we define a cli and we don't want the alias functions, the remote help functions and the function help show allcommands full, the conf file would be like this:

[refusedfunctions]
noalias = alias*
noremotehelp = remotehelp*
nofullhelp = help show allcommands full

In the refusedfunctions section, the keys (in this case noalias, noremotehelp and nofullhelp) have no meaning and are not used by the boscli, but they must be unique (you can't use the same key in two different lines). Another important point, is that the value can be a regular expression (like remotehelp*) or can be a complete funct name.

Installation

Go to the root directory of the source tree and type:

make install

For regenerate the deb package, go to the root directory of the source tree and type:

dpkg-buildpackage -us -uc -rfakeroot

Note: at this time the deb package don't follow the Debian Python Policy, but this will be changed soon...

Tutorial and Examples

For a basic tutorial of how can you create your own interactive shell see BoscliOssTutorial

For a Screencast Demo of BoscliOss extension for Asterisk See http://oss.alea-soluciones.com/files/screencasts/demo-boscli-oss-asterisk.html

download

To download the svn development version from Alea Soluciones anonymous SVN execute the following command:

svn co http://oss.alea-soluciones.com/svn/boscli-oss/boscli-oss-core/trunk boscli-oss-core

To download the latest svn stable version (r0.2.2 at the moment) from Alea Soluciones anonymous SVN execute the following command:

svn co http://oss.alea-soluciones.com/svn/boscli-oss/boscli-oss-core/tags/r0.2.2 boscli-oss-core-stable

Download the latest stable version from http://oss.alea-soluciones.com/download/boscli/releases/boscli-oss_latest.tar.gz

Download the binary deb from http://oss.alea-soluciones.com/download/boscli/releases/deb/boscli-oss-core_0.2.2-1_i386.deb

Download the deb source from http://oss.alea-soluciones.com/download/boscli/releases/deb/