ots-build - bash library to build OTS packages
# load library if ots_build=$(pkg-config --libs ots-build); then . $ots_build else echo >&2 "unable to load ots-build library" exit 1 fi
# override some default variables otsb_set var1 val1 otsb_set var2 val2
# initialize the library otsb_init
# parse the command line otsb_options "$@"
# create build variables otsb_set_build_variables
otsb_run_cmds
Sometimes the Off-The-Shelf (OTS) software you need has a complicated build workflow. If you want to manage that in a scripted (patch)-config-build-test-install environment, that usually means building a script to perform all of the work. If you are working with dozens of OTS package, that can be a lot of redundant work.
ots-build is a bash library which provides support for lots of common tasks in building OTS software. It provides a framework for command line arguments and for running partial or complete builds. It provides a means of patching the package prior to configuration. It provides a rudimentary logging mechanism to keep things tidy.
Because ots-build is a library, you'll still have to write a main build script to perform the build, but it should be much simpler.
The main build script (which uses ots-build) should be invoked with an absoulte path inside the directory containing the extracted package source code. For example, if the build script for package foo is located in /imports/foo/build and the package has been extracted and exists in the directory /tmp/foo-1.23, then the build script should be invoked as
cd /tmp/foo-1.23 /imports/foo/build
In order to simplify the interface, much of the information needed and provided by ots-build is shared via bash variables. Variables associated with particular tasks (for instance parsing the command line) are described in the documentation for those tasks.
These variables may be overridden after the library is loaded (using
the otsb_set
function); their documentation will specify when it is
safe to do so.
The library must be loaded at the very start of the build script. ots-build provides a pkg-config metadata file, so may be loaded via
if ots_build=$(pkg-config --libs ots-build); then . $ots_build else echo >&2 "unable to load ots-build library" exit 1 fi
Certain variables must be customized before invoking otsb_init
;
see the documentation for the variables to determine which are
affected.
ots-build must be initialized by calling the otsb_init
function.
The otsb_options
function is used to parse command line options and
arguments. It takes a list of tokens to parse. Typically this is
just those passed on the command line to the script:
otsb_options "$@"
The quotes are required to maintain proper tokenization. Note that
$@
is set to a function's arguments within a function, so the above
invocation will only work in the main body of the bash script.
Prior to issuing this command, additional command line options may be
added using otsb_add_option
. The following build options are
provided by default:
--exec-prefix --prefix
Only long options (preceded by --
) are recognized. Options which
take values may be separated from their values be a =
character or
white space. For example:
--exec-prefix=a --prefix b --flag
Hyphens in option names may be specified as underscores.
Command line options are either boolean (presence signifies true or
false), scalars (take a single value) or arrays (may be specified
multiple times, appending each value to an array). Flag options may
be specified either as affirmative ( --flag
) or negative
(--no-flag
).
After parsing, the values of the options are stored in bash variables
with the same name as the option (with any hyphens converted to
underscores). Boolean values are represented as the string true
for
true and false
for false. Options which are not specified on
the command line are not set to any default value.
Default values for options created via otsb_add_option
may be set
with otsb_set
before calling otsb_options
.
In addition to the options that you provide, ots-build provides
some of its own. These are stored in variables with a special prefix,
otsbv_
to prevent confusion. To specify defaults, use the
otsbv_set
command prior to calling otsb_init
. For example, the
--logsingle
option value will be stored in otsbv_logsingle
. To
set its default value:
otsbv_set logsingle true
ots-build supports a --help
option.
Options added via calls to otsb_add_option
will use the help string
provided in those calls.
Options provided by ots-build have pre-defined help strings which
can be overridden by calling the otsb_help
function prior to
calling ots_init
.
The only arguments accepted on the command line are the names of
commands. These are specified in the otsb_commands
array, which
defaults to
otsb_commands=( patch configure build test install )
The actual requested commands are made available in the
otsb_req_cmds
array after the command line is parsed. See
Commands for more information on how to interface build
commands to the auto-invocation code.
After invoking otsb_options
and before running commands, build
variables must be set by calling otsb_set_build_variables
.
The prefix
variable must already be defined, which is typically set
via the --prefix
command line option.
The following variables are set; they may be overriden prior
to invoking otsb_set_build_variables
using otsb_set
.
This is the directory which contains the build script. By default this is extracted from the name of the script (hence the requirement to invoke the build script with an absolute path).
This is the name of the package (including version information). This
is by default extracted from the name of the current directory
using the otsb_pkg
function.
The name of the package (without any version information). This is by
default derived from the package
variable using the
otsb_pkg_name
function. An alternative to overriding the variable
is to redefine the otsb_pkg_name
function.
The version of the package. This is by default derived from the
package
variable using the otsb_pkg_version
function. An
alternative to overriding the variable is to redefine the
otsb_pkg_name
function.
The name of the patch file (if any). This is by default set to
$import_root/$package.patch
This will default to the value of prefix
if it not already set.
Defaults to
${exec_prefix}/bin
Defaults to
${exec_prefix}/lib
Defaults to
${prefix}/include/${package}
Defaults to
${prefix}/man
Defaults to
${prefix}/share/doc/$package
The main purpose of ots-build is to run commands. The commands to run are specified on the command line (see Arguments). The commands which will always be available are:
Run all of the commands in the otsb_commands
array, in order specified.
Output the command line arguments.
Output a list of the non-administrative (i.e. actual build workflow) commands to stdout.
ots-build provides the otsb_run_cmds
function which will execute
the commands specified on the command line. A command is
implemented as a bash function of the same name, prefixed with
otsb_cmd_. For example, the install command might be written
as:
otsb_cmd_install () { ... commands to perform the actual install ... }
It is important that these functions either exit or return with non-zero values if an error ocurred (see Errors for other means of signalling an error).
Stub functions are provided for the configure
, build
, test
, and
install
commands. These will print an error message and exit with
a non-zero status (to remind you that you haven't implemented them
properly). You must provide replacement versions of these.
The patch
command is more fully implemented. See Patching for
more information.
ots-build provides a default patch command for patching a source
distribution before it is configure. This can be changed by redefining
the otsb_cmd_patch
function.
The default function requires the applypatch command and a working
version of patch (Solaris' version is by definition broken). The --patch-cmd
option is available to specify an alternative version of patch. You
can override it via otsbv_set
before calling otsb_init
If the patch file specified via the patchfile
global variable (see
Sharing information with the library) exists, it is applied using
applypatch -patch "patch -p0 -N -s" $patchfile
If logging is turned on via the --log
or --logsingle
options,
the output of each command is logged.
--log
logs each command to a separate file, pfxcmd.log, where
pfx is a prefix specified by --logpfx
. This defaults to
otsb_
, so that it does not interfere with any log files created by
the package (such as config.log, etc.).
--logsingle
sends the output from all commands to a single file,
pfx.log, where pfx is the value of the --logpfx
option, with
any trailing underscore removed.
When logging is turned on, by default no output is sent to the terminal.
To log to the terminal as well, specify --logconsole
.
Defaults for the variables associated with these options may set via the
otsbv_set
function before calling otsb_init
.
If a command function returns a non-zero value ots-build will exit with an error. Typically it is up to the writer of that function to ensure that an error will cause a non-zero return. That can be a pain. ots-build provides a simple interface to the bash error trapping system which will ensure that if any command executed within a command function exits with an error it will be caught and ots-build will be notified.
The otsb_trap_err
command is used to manage the trap.
otsb_trap_err functions on otsb_trap_err functions off
This sets the default state when a command function is invoked. If this is invoked within a command function it will change the state immediately.
otsb_trap_err on otsb_trap_err off
These turn the system on and off immediately. If invoked within a command function, the trap state will return to off after the function returns.
otsb_trap_err functions reset
This is only valid within a command function and resets the state to
the default state specified by the last otsb_trap_err functions
setting.
otsb_add_option $type $name "$help"
Add an option of the given type (scalar
, bool
, array
),
name, and help string.
otsb_add_dumpvar $var_name
Add the named variable to the list of variables output by the
dump
command. Option variables are automatically added to the
list.
otsb_append_path $path_var_name $path1 $path2 ...
Append the specified path(s) to the path variable with the given name. A path variable's value is a colon separated list of paths. For example,
foo=a:b:c:d otsb_append_path foo e f g h
results in
foo=a:b:c:d:e:f:g:h
otsb_prepend_path $path_var_name $path1 $path2 ...
Prepend the specified path(s) to the path variable with the given name. A path variable's value is a colon separated list of paths. For example,
foo=a:b:c:d otsb_prepend_path foo e f g h
results in
foo=e:f:g:h:a:b:c:d
otsb_die $message
Print the message to the standard error stream and exit with a non-zero status code.
otsb_is_true $value otsb_is_false $value
These returns true if the passed value matches what ots-build understands as true or false:
true: the strings 'true' or 'on', or a non-zero number false: the strings 'false' or 'off', or zero
Any other values will return false.
otsb_is_boolean $value
Returns true if the value is a boolean recognized by either otsb_is_true
or otsb_is_false
.
This function is called when the script exits and performs any necessary cleanup. By default it does nothing. You should redefine it if you need it.
otsb_help cmd cmd_name "cmd help string" otsb_help option option_name "option helpstring"
Change the help string of an existing option.
The second argument is either cmd
or c<option> indicating what
element the help string is for. For example, here's how the default
help string for the --exec-prefix
option is specified:
otsb_help option exec-prefix \ "install architecture-dependent files here"
This function determines the name of the package and prints it to
the standard output stream. It is used to generate the value of the
package_name
variable. The default definition is
otsb_pkg_version () { echo ${package%%-*} ; }
It may be overriden (before calling ots_set_build_variables
). When
called, the package
variable will have been set.
This function determines the version of the package and prints it to
the standard output stream. It is used to generate the value of the
package_version
variable. The default definition is
otsb_pkg_version () { echo ${package##*-} ; }
It may be overriden (before calling ots_set_build_variables
). When
called, the package
variable will have been set.
This executes the build commands listed in the otsb_req_cmds
variable, which is filled in by otsb_options
. The commands are run
in the order in which they were specified, not the order listed in
otsb_commands
. Do not invoke this command more than once.
If a command does not return a successful exit status code, the
subsequent commands are not run and otsb_run_cmds
returns the exit
status code of the failing command.
otsb_options "$@" otsb_options "$my_options_from_somewhere_else"
otsb_options
parses the passed tokens for options and commands. It
recognizes options specified via the otsb_options_bools
,
otsb_options_scalars
and otsb_options_scalars
variables and
commands specified via the otsb_cmds_args
variable. It places
values for passed options in variables of the same name and places the
requested commands in the otsb_req_cmds
variable. See Parsing the Command Line for more information.
Default values for options may be specified by setting the appropriate
option variables before calling otsb_options
.
otsb_set variable value
If the named (scalar!) variable has not been set, assign it the specified value:
otsb_set prefix "a prefix is here"
In most cases this is the preferred way of setting a variable's value, as it allows using environment variables to specify default values.
For example, if the build script is run in an environment with the environment variable prefix set:
prefix=/my/prefix /imports/package/build all
then setting the default value of prefix via
[...] prefix=/a/default/value otsb_options "$@"
will ignore the environment, while
[...] otsb_set prefix /a/default/value otsb_options "$@"
will not.
otsb_set variable value1 value2 ...
If the named (array!) variable has not been set, assign it the specified value:
otsb_set config_opts --do-this --do-that
will do the equivalent of
config_opts=( --do-this --do-that )
if config_opts
is empty.
otsbv_set variable value
Similar to otsb_set
, but used only for option variables provided by
ots-build.
otsbv_set log true
This function is called after all invocations of otsb_options
. It
generates important build variables; see Setting build variables.
See the Errors section.
If a number of packages require the same boilerplate initialization for ots-build, it is possible to redefine some of ots-build's core routines to provide customization; this can be done in a library.
The routines which may be overriden,
otsb_init otsb_options otsb_set_build_variables
are simple wrappers around the actual routines (with an underscore in front of them):
otsb_init () { _otsb_init }
For example, if all packages should have the --ots-root
option
which is used to figure out where to install the package, redefine
otsb_init
and otsb_set_build_variables
as follows:
. $(pkg-config --libs ots-build)
otsb_init () {
otsb_options_scalars+=( ots-root platform ) otsb_help option ots-root "top-level directory into which OTS packages will be installed" otsb_help option platform "platform name"
_otsb_init }
otsb_options () {
_otsb_options "$@"
otsb_set ots_root /my/default/ots_root
}
otsb_set_build_variables {
otsb_set prefix $ots_root/$package otsb_set platform $(uname -m) otsb_set exec_prefix "$prefix/$platform"
_otsb_set_build_variables }
There's a small subtlety in the otsb_options
redefinition. In
order for the caller to use the value of ots_root
to override
variables set by otsb_set_build_variables
its value must be set
between calls to _otsb_options
and otsb_set_build_variables
and
before the caller's calls to otsb_set
.
(Written in modern bash which supports appending to arrays with +=
).
Save the code in local-ots-build and load it instead of ots-build and all packages will get the extra functionality.
The following variables are important to ots-build. Be careful that you do not inadvertently use them in a different context.
An array containing the available commands, in the order that they should be invoked.
An array containing the commands requested on the command line.
An array containing a listing of boolean options.
An array containing a listing of options taking a single value.
An array containing a listing of options which may be specified multiple times, with each value appended to an array.
The version of the ots-build library.
The name of the directory containing the build script, used to find the optional patch file.
The name of the package, including the version number, derived from the directory that the build script is run in.
The (optional) file containing the patch to apply to the source directory, derived from the package name and the directory containing the build script.
The package version, determined from the package name.
A boolean indicating whether or not logging is enabled.
The prefix appended to the command name to create a name for the log file.
A boolean indicating that all logs should go to a single file.
A boolean indicating that logging should be sent to the console as well as to the log files.
Copyright (C) 2010 Smithsonian Astrophysical Observatory
This file is part of ots-build.
ots-build is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with this program. If not, see http://www.gnu.org/licenses/.
Diab Jerius <djerius@cfa.harvard.edu>