scsh-install-lib/doc/proposal.txt

In Emacs, read this file in -*- Outline -*- mode.

* Introduction

The aim of the following proposal is to define a standard for the
packaging, distribution, installation, use and removal of libraries
for scsh. Such packaged libraries are called "scsh packages" or simply
"packages" below.

This proposal attempts to cover both libraries containing only Scheme
code and libraries containing additional C code. It does not try to
cover applications written in scsh, which are currently considered to
be outside of its scope.

** Package identification and naming

Packages are identified by a globally-unique name. This name should
start with an ASCII letter (a-z or A-Z) and should consist only of
ASCII letters, digits or underscore characters (_). Package names are
case-sensitive, but there should not be two packages with names which
differ only by their capitalisation.

Rationale:

  This restriction on package names ensures that they can be used to
  name directories on current operating systems.

Several versions of a given package can exist. A version is identified
by a sequence of non-negative integers. Versions are ordered
lexicographically.

A version has a printed representation which is obtained by separating
(the printed representation of) its components by dots. For example,
the printed representation of a version composed of the integer 1
followed by the integer 2 is the string "1.2". Below, versions are
usually represented using their printed representation for simplicity,
but it is important to keep in mind that versions are sequences of
integers, not strings.

A specific version of a package is therefore identified by a name and
a version. The full name of a version of a package is obtained by
concatenating:

  - the name of the package,

  - a hyphen (-),

  - the printed representation of the version.

In what follows, the term "package" is often used to designate a
specific version of a package, but this should be clear from the
context.

* Distribution of packages

Packages are distributed in TAR archives, which can optionally be
compressed by GZIP or BZIP2.

The name of the archive is composed by appending:

  - the full name of the package,

  - the string ".tar" indicating that it's a TAR archive,

  - either the string ".gz" if the archive is compressed using GZIP,
    or the string ".bz2" if the archive is compressed using BZIP2, or
    nothing if the archive is not compressed.

** Archive contents

The archive is organised so that it contains one top-level directory
whose name is the full name of the package. This directory is called
the package unpacking directory. All the files belonging to the
package are stored below it.

The unpacking directory contains at least the following files:

  - install-pkg
      a script performing the installation of the package,

  - README
      a textual file containing a short description of the package,

  - COPYING
      a textual file containing the license of the package.

* Downloading and installation of packages

A package can be installed on a target machine by downloading its
archive, expanding it and finally running the installation script
located in the unpacking directory.

All files belonging to a package will be installed in a single
directory called the "package installation directory". The package
installation directory contains all files, and only files belonging to
the (version of the) package.

Rationale:

Such an organisation makes it easy to uninstall package by just
recursively deleting a single directory. It also makes it trivial to
know to which package a given file belongs.

** Root directory layout

The package installation directory will be an indirect sub-directory
of some "package root directory". A package root directory is a
directory on the target machine which contains all installed packages.
There can be several package roots on a target machine, for example
one for packages installed globally, and one per user for "personal"
packages.

A package root directory contains exactly two sub-directories, and
nothing else. These directories are called "installed" and "active".

The "installed" directory contains exactly one directory per package,
and nothing else (N.B. here "package" really means "package" and not
"package version"). These directories have the same name as the
package they contain. For every installed version of a package, there
is a sub-directory of the package directory. These directories are
named using the printed representation of the version they contain.

The "active" directory contains only symbolic links which point to
package version directories. There is at most one symbolic link per
installed package. These symbolic links identify the active (or
default) version of a package, that is the version will be used if a
script asks for a given package without explicitly which version is
required.

Example:

Let's imagine a system on which the directory
"/usr/local/lib/scsh/modules" serves as the package root directory. On
this system, versions 1.0 and 2.0 of a package called "package_1" are
installed, the latter being active. Further, version 1.5.4 of a
package called "package_2" is installed and active. The contents of
the package root would look as follows:

/usr/local/lib/scsh/modules/
  installed/
    package_1/
      1.0/
        all files belonging to v1.0 of package_1 (and nothing else)
      2.0/
        all files belonging to v2.0 of package_1
    package_2/
      1.5.4/
        all files belonging to v1.5.4 of package_2
  active/
    package_1 -> ../installed/package_1/2.0
    package_2 -> ../installed/package_2/1.5.4

** Package directory layout

Even though the exact contents of the package directory will depend on
the package, all package directories are laid out according to the
rules below. These rules make it easy to examine the contents of a
package directory, and find important data like the package
documentation.

The standard layout is shown below. Some directories are of course
optional because not all packages have something to put in all of
them.

  load.scm

    Scheme file written in Scheme 48's exec language, whose role is to
    define all the structures which belong to a package.

  scheme/

    Directory containing all the Scheme code of the package.

  lib/

    Directory containing one sub-directory per platform for which the
    package was installed. These sub-directories contain the shared
    libraries for the given platform; the name of a given platform is
    the one given by autoconf's "config.guess" script.

    <platform>/

      Directory containing the shared libraries for <platform>.

  doc/

    Directory containing the documentation of the package, possibly in
    different formats.

    html/

      Directory containing the HTML documentation of the package, if
      any; this directory should at least contain one file called
      "index.html" serving as an entry point to the documentation.

    pdf/

      Directory containing the PDF documentation of the package, if
      any; this directory should contain at least one file called
      "<package>.pdf" where "<package>" is the name of the package.

    ps/

      Directory containing the PostScript documentation of the
      package, if any; this directory should contain at least one file
      called "<package>.ps" where "<package>" is the name of the
      package.

** File permissions

TODO

** Installation procedure

Packages are installed using the "install-pkg" script located in the
package archive. This script must be given the name of the root
directory in which to perform installation with the "--root" option.
It also accepts the following options:

  --dry-run or -n

    Print what actions would be performed to install the package, but
    do not perform them.

  --inactive or -i

    Do not activate package after installing it.

** Creating images

TODO (my current idea is to add support to install-lib to easily
      create an image containing the package being installed, and
      maybe some structures opened. Then, at install time, users could
      say that they want an image to be created, and the install
      script would do that).

* Using packages

To use a package, its "loading script" must be loaded in Scheme 48's
exec package. The loading script for a package is a file written in
the Scheme 48 exec language, whose name is "load.scm" and which is
located directly in the package installation directory.

To load this file, one typically uses scsh's "-lel" option along with
a properly defined SCSH_LIB_DIRS environment variable.

Scsh has a list of directories, called the library directories, in
which it looks for files to load when the options -ll or -lel are
used. This list can be given a default value during scsh's
configuration, and this value can be overridden by setting the
environment variable SCSH_LIB_DIRS before running scsh.

In order for scsh to find the package loading scripts, one must make
sure that scsh's library search path contains the names of both the
"installed" and the "active" directories of *every* existing package
root directories.

Example:

On a system where the package root directory is
"/usr/local/lib/scsh/modules", the SCSH_LIB_DIRS environment variable
has to contain at least the following two directories:

  /usr/local/lib/scsh/modules/active
  /usr/local/lib/scsh/modules/installed

The names of these directories should not end with a slash (/), as
this forces scsh to search them recursively. This could *drastically*
slow down scsh when looking for packages.

A package named "foo" can then be used from a script provided that the
following option is added to its command line:

  -lel foo/load.scm

* Writing packages

Once the Scheme and/or C code for a package has been written, the last
step in turning it into a standard package as defined by this proposal
is to write the installation script.

This script could be written fully by the package author, but in order
to simplify this task a small scsh installation framework is provided.
This framework is composed of several files which are meant to be
included in the package archive. These files are:

  - install-pkg

    a trivial sh script which launches scsh on the main function of
    the installation library, passing it all the arguments given by
    the user,

  - install-lib.scm

    the code for the installation library, documented below,

  - install-lib-module.scm

    Scheme 48 interface and structure definitions for the installation
    library.

As explained above, when the install-pkg script is invoked, it launches
scsh on the main function of the installation library, which does the
following:

  - parse the command line arguments (e.g the --root option),

  - load the package definition file, a (Scheme) file called
    "pkg-def.scm", which is supplied by the package author and which
    contains the installation procedure for the package,

  - install the package which was defined in the previous step.

It is actually possible to define several packages in "pkg-def.scm",
and all will be installed. It should not be often useful, though.

The main job of the package author is therefore to write the package
definition file, "pkg-def.scm".

This file is mostly composed of a package definition statement, which
specifies the name, version and installation code for the package. The
package definition statement is expressed using the following syntax
exported from the installation library:

(define-package <name> <version> <body> ...)    (syntax)

  Define a package to be installed. NAME is the package name (a
  string), VERSION its version (a list of integers) and BODY is the
  list of statements to be evaluated in order to install the package.

The installation statements typically use functions of the
installation library in order to install files in their target
location. The following functions are currently exported:

(install-file <file> [<target-dir>] [<perms>])

  Install the given file in TARGET-DIR. TARGET-DIR is specified
  relative to the package directory, and defaults to "." (i.e. the
  package directory itself). If the target directory does not exist,
  it is created along with all its parents, as needed.

  The copied file and all directories created by this command have
  their permissions set to PERMS, an integer which defaults to #o755
  (i.e. read, write and execute for the owner, read and execute for
  the rest).

(install-files <file-list> [<target-dir>] [<perms>])

  Like install-file but for several files, which are specified as a
  list.

(install-directory <dir> [<target-dir>] [<perms>])

  Install the given directory and all its contents, including
  sub-directories, in TARGET-DIR. This is similar to what INSTALL-FILE
  does, but for complete hierarchies.

  Notice that DIR will be installed as a sub-directory of TARGET-DIR.

(install-directories <dir-list> [<target-dir>] [<perms>])

  Install several directories in one go.

Example:

A typical package definition file for a simple package called
"my_package" whose version is 1.2 could look like this:

(define-package "my_package" (1 2)
  (install-file "load.scm")
  (install-directories '("scheme" "doc")))

With such a definition, invoking the installation script with
"/usr/local/lib/scsh/modules" as package root would have the
following effects:

1. The package directory

     /usr/local/lib/scsh/modules/installed/my_package/1.2

   would be created.

2. File "load.scm" would be copied to this directory.

3. All the contents of the directory called "scheme" would be copied
   to directory

     /usr/local/lib/scsh/modules/installed/my_package/1.2/scheme

   The same would happen for the contents of directory "doc".

4. The package would be activated by creating a symbolic link with
   name

     /usr/local/lib/scsh/modules/active/my_package

   pointing to

     ../installed/my_package/1.2

** Packages containing C code (for shared libraries)

Packages containing C code are more challenging to write, since all
the problems related to C's portability and incompatibilities between
the APIs of the various platforms have to be accounted for.
Fortunately, the GNU Autoconf system simplifies the management of
these problems, and authors of scsh packages containing C code are
strongly encouraged to use it.

Integrating Autoconf into the installation procedure should not be a
major problem thanks to scsh's ability to run separate programs.

* Packaging packages

Most important Unix systems today have one (or several) package
management systems which ease the installation of packages on a
system. In order to avoid confusion between these packages and the
scsh packages discussed above, they will be called "system packages"
in what follows.

It makes perfect sense to provide system packages for scsh packages.
System packages should as much as possible try to use the standard
installation script described above to install scsh packages. This
script currently provides some support for staged installations,
which are required by several packaging systems.

This support is provided through an additional option, --dest-root,
which specifies the package root in which the files have to be copied
by the installation script. The files will then have to be moved from
this location to their final root directory, i.e. the one given
through the --root option, by the system packaging tools.

(The --dest-root option plays the same role as the DEST_DIR
variable which is typically given to "make install", for makefiles
which support staging directories).

* Glossary

TODO define the following terms

Version

Target machine

Package

(Package) unpacking directory

(Packages) root directory

(Package) installation directory

Package loading script

* Version

$Id: proposal.txt,v 1.1 2003/12/14 12:14:21 michel-schinz Exp $