In Emacs, read this file in -*- Outline -*- mode. * Introduction The aim of the following proposal is to define a standard for the packaging, distribution, installation, use and removal of libraries for scsh. Such packaged libraries are called "scsh packages" or simply "packages" below. This proposal attempts to cover both libraries containing only Scheme code and libraries containing additional C code. It does not try to cover applications written in scsh, which are currently considered to be outside of its scope. ** Package identification and naming Packages are identified by a globally-unique name. This name should start with an ASCII letter (a-z or A-Z) and should consist only of ASCII letters, digits or underscore characters (_). Package names are case-sensitive, but there should not be two packages with names which differ only by their capitalisation. Rationale: This restriction on package names ensures that they can be used to name directories on current operating systems. Several versions of a given package can exist. A version is identified by a sequence of non-negative integers. Versions are ordered lexicographically. A version has a printed representation which is obtained by separating (the printed representation of) its components by dots. For example, the printed representation of a version composed of the integer 1 followed by the integer 2 is the string "1.2". Below, versions are usually represented using their printed representation for simplicity, but it is important to keep in mind that versions are sequences of integers, not strings. A specific version of a package is therefore identified by a name and a version. The full name of a version of a package is obtained by concatenating: - the name of the package, - a hyphen (-), - the printed representation of the version. In what follows, the term "package" is often used to designate a specific version of a package, but this should be clear from the context. * Distribution of packages Packages are distributed in TAR archives, which can optionally be compressed by GZIP or BZIP2. The name of the archive is composed by appending: - the full name of the package, - the string ".tar" indicating that it's a TAR archive, - either the string ".gz" if the archive is compressed using GZIP, or the string ".bz2" if the archive is compressed using BZIP2, or nothing if the archive is not compressed. ** Archive contents The archive is organised so that it contains one top-level directory whose name is the full name of the package. This directory is called the package unpacking directory. All the files belonging to the package are stored below it. The unpacking directory contains at least the following files: install-pkg a script performing the installation of the package, README a textual file containing a short description of the package, COPYING a textual file containing the license of the package. * Downloading and installation of packages A package can be installed on a target machine by downloading its archive, expanding it and finally running the installation script located in the unpacking directory. ** Layouts The installation script installs files according to some given layout. A layout maps abstract locations to concrete directories on the target machine. For example, a layout could map the abstract location "doc" (where documentation is stored) to the directory "/usr/local/share/doc/my_package". Currently, the following abstract locations are defined: base The "base" location of a package, where the package loading script "load.scm" resides. active Location containing a symbolic link, with the same name as the package (without the version), pointing to the "base" location of the package. This link is used to designate the "active" version of a package, that is the one to load when a package is requested by giving only its name, without explicit version. scheme Location containing all Scheme code. If the package comes with some examples showing its usage, they are put in a sub-directory called "examples" of this location. lib Location containing platform-dependent files, like shared libraries. This location contains one sub-directory per platform for which packages have been installed, and nothing else. doc Location containing all the package documentation. This location contains one or more sub-directories which store the documentation in various formats. The contents of these sub-directories is standardised as follows, to make it easy for users to find the document they need: html/ Directory containing the HTML documentation of the package, if any; this directory should at least contain one file called "index.html" serving as an entry point to the documentation. pdf/ Directory containing the PDF documentation of the package, if any; this directory should contain at least one file called ".pdf" where "" is the name of the package. ps/ Directory containing the PostScript documentation of the package, if any; this directory should contain at least one file called ".ps" where "" is the name of the package. misc-shared Location containing miscellaneous data which does not belong to any directory above, and which is platform-independant. The directories to which a layout maps these abstract locations are not absolute directories, but rather relative ones. Therefore, a layout alone is not enough to know where files will end up on the target machine: a prefix is also required. This prefix is specified on the command-line during installation, using the "--prefix" option, as explained below. Example : Let's imagine that a user is installing version 1.2 of a package called "foo". This package contains a file called "COPYING" which has to be installed in sub-directory "license" of the "doc" location. If the user chooses to use the default layout, which maps "doc" to directory "/doc" (see below), and specifies "/usr/local/etc/scsh/modules" as a prefix, then the "COPYING" file will end up in: /usr/local/etc/scsh/modules/foo-1.2/doc/license/COPYING \_________________________/ \_________/ \_____________/ 1 2 3 Part 1 is the prefix, part 2 is the layout's mapping for the "doc" location, and part 3 is the file name relative to the location. *** Predefined layouts Every installation script comes with a set of predefined layouts which serve different aims. They are described below. **** The "scsh" layout The "scsh" layout is the default layout. It maps all locations to sub-directories of a single directory, called the package installation directory, which contains all the files of the package being installed and nothing else. Its name is simply the full name of the package in question, and it resides in the "prefix" directory. The "scsh" layout maps locations as follows: base -> / active -> scheme -> //scheme lib -> //lib doc -> //doc misc-shared -> / This layout is well suited for installations performed without the assistance of an additional package manager, because it makes many common operations easy. For example, finding to which package a file belongs is trivial, as is the removal of an installed package. **** The "fhs" layout The "fhs" layout maps locations according to the File Hierarchy Standard (FHS, see ...), as follows: base -> /share/scsh/modules/ active -> /share/scsh/modules scheme -> /share/scsh/modules//scheme lib -> /lib/scsh/modules/ doc -> /share/doc/ misc-shared -> /share/scsh/modules/ The main advantage of this layout is that it adheres to the FHS standard, and is therefore compatible with several packaging policies, like Debian's, Fink's and others. Its main drawback is that files belonging to a given package are scattered, and therefore hard to find when removing or upgrading a package. Its use should therefore be considered only if third-party tools are available to track files belonging to a package. ** File permissions TODO ** Installation procedure Packages are installed using the "install-pkg" script located in the package archive. This script must be given the name of the prefix using the "--prefix" option. It also accepts the following options: --layout Specifies the layout to use. --dry-run Print what actions would be performed to install the package, but do not perform them. --inactive Do not activate package after installing it. ** Creating images TODO (my current idea is to add support to install-lib to easily create an image containing the package being installed, and maybe some structures opened. Then, at install time, users could say that they want an image to be created, and the install script would do that). * Using packages To use a package, its "loading script" must be loaded in Scheme 48's exec package. The loading script for a package is a file written in the Scheme 48 exec language, whose name is "load.scm" and which resides in the "base" location. To load this file, one typically uses scsh's "-lel" option along with a properly defined SCSH_LIB_DIRS environment variable. Scsh has a list of directories, called the library directories, in which it looks for files to load when the options -ll or -lel are used. This list can be given a default value during scsh's configuration, and this value can be overridden by setting the environment variable SCSH_LIB_DIRS before running scsh. In order for scsh to find the package loading scripts, one must make sure that scsh's library search path contains the names of all "active" locations which containing packages. The names of these directories should not end with a slash (/), as this forces scsh to search them recursively. This could *drastically* slow down scsh when looking for packages. Example: Let's imagine a machine on which the system administrator installs scsh packages according to the "fhs" layout in prefix directory "/usr/local". The "active" location for these packages corresponds to the directory "/usr/local/share/scsh/modules", according to the layout specification above. Let's also imagine that there is a user called "john" on this machine, who installs additional scsh packages for himself in his home directory, using "/home/john/scsh-packages" as a prefix. To ease their management, he uses the "scsh" layout. The "active" location for these packages corresponds to the directory "/home/john/scsh-packages", according to the layout specification above. In order to be able to use scsh packages installed both by the administrator and by himself, user "john" needs to put both active directories in his SCSH_LIB_DIRS environment variable. The value of this variable will therefore be: "/usr/local/share/scsh/modules" "/home/john/scsh-packages" Now, in order to use packages "foo" and "bar" in one of his script, user "john" just needs to load their loading script using the -lel option when invoking scsh, as follows: -lel foo/load.scm -lel bar/load.scm * Writing packages Once the Scheme and/or C code for a package has been written, the last step in turning it into a standard package as defined by this proposal is to write the installation script. This script could be written fully by the package author, but in order to simplify this task a small scsh installation framework is provided. This framework is composed of several files which are meant to be included in the package archive. These files are: install-pkg a trivial sh script which launches scsh on the main function of the installation library, passing it all the arguments given by the user, install-lib.scm the code for the installation library, whose public interface is documented below, install-lib-module.scm Scheme 48 interface and structure definitions for the installation library, configure.scm a Scheme library providing a function to query the name of the platform, needed by some layouts. As explained above, when the install-pkg script is invoked, it launches scsh on the main function of the installation library, which does the following: - parse the command line arguments (e.g the --prefix option), - load the package definition file, a (Scheme) file called "pkg-def.scm", which is supplied by the package author and which contains the installation procedure for the package, - install the package which was defined in the previous step. It is actually possible to define several packages in "pkg-def.scm", and all will be installed. It should not be often useful, though. The main job of the package author is therefore to write the package definition file, "pkg-def.scm". This file is mostly composed of a package definition statement, which specifies the name, version and installation code for the package. The package definition statement is expressed using the following syntax exported from the installation library: (define-package ...) (syntax) Define a package to be installed. NAME is the package name (a string), VERSION its version (a list of integers), EXTENSIONS is an association list of extensions, and BODY is the list of statements to be evaluated in order to install the package. The installation statements typically use functions of the installation library in order to install files in their target location. The following functions are currently exported: (install-file [] []) Install the given FILE in the sub-directory TARGET-DIR (which must be a relative directory) of the given LOCATION. If the directory in which the file is about to be installed does not exist, it is created along with all its parents, as needed. If FILE is a string, then the installed file will have the same name as the original one. If FILE is a pair, then its first element specifies the name of the source file, and its second element the name it will have once installed. The second element must be a simple file name, without any directory part. The copied file and all directories created by this command have their permissions set to PERMS, an integer which defaults to #o755 (i.e. read, write and execute for the owner, read and execute for the rest). (install-files [] []) Like install-file but for several files, which are specified as a list. Each element in the list can be either a simple string or a pair, as explained above. (install-directory [] []) Install the given DIRectory and all its contents, including sub-directories, in sub-directory TARGET-DIR of LOCATION. This is similar to what INSTALL-FILE does, but for complete hierarchies. Notice that DIR will be installed as a sub-directory of TARGET-DIR. (install-directories [] []) Install several directories in one go. (install-directory-contents [] []) Install the *contents* of the given DIRectory in sub-directory TARGET of LOCATION. An additional function exists to query the mapping of a location: (get-directory ) Get the absolute name of the directory to which the current layout maps the abstract LOCATION. If INSTALL? is true, the directory is the one valid during installation; If it is false, the directory is the one valid after installation, that is when the package is later used. The distinction between installation-time and usage-time directories is necessary to support staged installation, as performed by package managers like Debian's APT. Example: A typical package definition file for a simple package called "my_package" whose version is 1.2 could look like this: (define-package "my_package" (1 2) (install-file "load.scm" 'base) (install-directory-contents "scheme" 'scheme) (install-file ("LICENSE" . "COPYING") 'doc) (install-directory-contents "doc" 'doc)) With such a definition, invoking the installation script with "/usr/local/" as prefix and "fhs" as layout would have the following effects: 1. The base directory /usr/local/share/scsh/modules/my_package-1.2 would be created and file "load.scm" would be copied to it. 3. All the contents of the directory called "scheme" would be copied to directory /usr/local/share/scsh/modules/my_package-1.2/scheme which would be created before, if needed. 4. File "LICENSE" would be copied to directory /usr/local/share/doc/my_package-1.2/ with name "COPYING". 5. All the contents of the directory called "doc" would be copied to directory /usr/local/share/doc/my_package-1.2/ 6. The package would be activated by creating a symbolic link with name /usr/local/share/scsh/modules/my_package pointing to ./my_package-1.2 ** Packages containing C code (for shared libraries) Packages containing C code are more challenging to write, since all the problems related to C's portability and incompatibilities between the APIs of the various platforms have to be accounted for. Fortunately, the GNU Autoconf system simplifies the management of these problems, and authors of scsh packages containing C code are strongly encouraged to use it. Integrating Autoconf into the installation procedure should not be a major problem thanks to scsh's ability to run separate programs. * Packaging packages Most important Unix systems today have one (or several) package management systems which ease the installation of packages on a system. In order to avoid confusion between these packages and the scsh packages discussed above, they will be called "system packages" in what follows. It makes perfect sense to provide system packages for scsh packages. System packages should as much as possible try to use the standard installation script described above to install scsh packages. This script currently provides some support for staged installations, which are required by several packaging systems. This support is provided through an additional option, --dest-dir, which specifies the root directory in which to install files. The files will then have to be moved from this location to their final location by the system packaging tools. (The --dest-dir option plays the same role as the DESTDIR variable which is typically given to "make install", for makefiles which support staging directories). * Glossary TODO define the following terms Version Target machine Package (Package) unpacking directory Layout (Abstract) location Package loading script * Version $Id: proposal.txt,v 1.4 2003/12/21 21:01:26 michel-schinz Exp $