%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% %W gappkg.tex GAP documentation Werner Nickel %W Alexander Hulpke %% %H @(#)$Id: gappkg.tex,v 4.13.2.2 2008/09/10 11:21:43 gap Exp $ %% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \Chapter{Writing a GAP Package} This chapter explains the basics of how to write a {\GAP} package so that it interfaces properly to {\GAP}. For the role of {\GAP} packages and the ways to load them, see Chapter~"ref:GAP Packages" in the {\GAP} Reference Manual. There are two basic aspects of creating a {\GAP} package. First, it is a convenient possibility to load additional functionality into {\GAP} including a smooth integration of the package documentation. And secondly, a package is a way to make your code available to other {\GAP} users. The {\GAP} Team provides some help with the distribution of packages. In particular, a package can be submitted to a refereeing process. Check out the {\GAP} Web pages \URL{http://www.gap-system.org} for more details. We start this chapter with a description how the directory structure of a {\GAP} package must look like and then add remarks on certain aspects of creating a package, some of these only apply to some packages. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \Section{The Files of a GAP Package} All files of a {\GAP} package must be collected in a single directory. To use the package with {\GAP} this directory must be a subdirectory of a `pkg' directory in (one of) the {\GAP} root directories (see "ref:GAP Root Directory" in the {\GAP} Reference Manual). (For example, if {\GAP} is installed in `/usr/local/gap4' then put the files of your package `MyPack' in the directory `/usr/local/gap4/pkg/mypack'.) Let us call this directory the *home directory* of the package. There are three file names with a special meaning in the home directory of a package: `PackageInfo.g' and `init.g' which must be present and `read.g' which is optional. The file `PackageInfo.g' contains meta-information about the package (package name, version, author(s), relations to other packages, homepage, download archives, banner, ...). This is used by the package loading mechanism and also for the distribution of a package to other users. The content of this file is explained via a template file below (see "The PackageInfo.g File"). The `init.g' is read when the package is loaded (see "ref:LoadPackage" in the {\GAP} Reference Manual). In principle this file could contain the whole {\GAP} code of a package, but usually it contains mainly `Read' or `ReadPackage' statements for reading further files of the package. For many packages it may be useful to have declaration and implementation parts in different files, see "Declaration and Implementation Part" below for more details. In that case it can be useful to read in only the declaration parts from the `init.g' file and to add a file `read.g' which contains the `ReadPackage' statements for the implementation parts. There is one further rule for the location of kernel library modules or external programs which is explained in "Installation of GAP Package Binaries" below. All other files can be organized as you like. But we suggest that you have a look at existing packages and use a similar scheme. For example, collect your {\GAP} code in files in a subdirectory `lib' or `gap', put the documentation in a subdirectory `doc', put source code for compilation in `src', data libraries in extra subdirectories and so on. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \Section{Writing Documentation} \indextt{README!for a GAP package} If you intend to make your package available to other users it is essential to include a documentation how to install and use your programs. Concerning the installation you should produce a file `README' which gives a short description of the purpose of the package and contains proper instructions how to install your package. Again, check out some existing packages to get an idea how this could look like. Concerning the documentation of the use of the package there are currently two recognised ways of producing {\GAP} package documentation. There is the method that has been used to produce the main manuals for {\GAP} which requires the documentation to be written in {\TeX} according to the format described in Chapter "The gapmacro.tex Manual format". There is also an XML-based documentation format that is defined in and can be used with the \package{GAPDoc} package (see~"GAPDoc:Introduction and Example"). In principle it is also possible to use some completely different documentation format. In that case you need to study the Chapter~"Interface to the GAP Help System" to learn how to make your documentation available to the {\GAP} help system. There should be at least a text version of your documenation provided for use in the terminal running {\GAP} and some nicely printable version in `.dvi' and/or `.pdf' format. Many {\GAP} users like to browse the documentation in HTML-format via their Web-browser. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \Section{An Example of a GAP Package} We illustrate the creation of a {\GAP} package by an example of a basic package. %% ???(The following assumes that you are using {\GAP} under UNIX.) Create the following directories in your home area: `pkg' and `pkg/test'. Inside the directory `test' create an empty file `init.g', and a file `PackageInfo.g' with the following contents. \begintt SetPackageInfo( rec( PackageName := "test", Version := "1.0", AvailabilityTest := ReturnTrue, Autoload := false, BannerString := Concatenation( [ "#I loading the GAP package ``test'' in version ", ~.Version, "\n" ] ), PackageDoc := rec( BookName := "test", SixFile := "doc/manual.six", Autoload := true ) ) ); \endtt This file declares the {\GAP} package with name ``test'' in version 1.0. There are no requirements that have to be tested, so `ReturnTrue' (see~"ref:ReturnTrue" in the {\GAP} Reference Manual) is used as test function. The package is not autoloaded, and it has its individual banner string. The package documentation consists of one autoloaded book; the `SixFile' component is needed by the {\GAP} help system. Now start {\GAP} with the command \begintt gap -l "./;" \endtt (the `-l "./;"' option adds the current directory to the {\GAP} root directories and allows {\GAP} to find the packages installed in the `./pkg' directory. %notest \beginexample gap> LoadPackage("test"); #I loading the GAP package ``test'' in version 1.0 true \endexample This {\GAP} package is too simple to be useful, but we have succeeded in loading it via `LoadPackage'. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \Section{The WWW Homepage of a Package} If you want to distribute your package you should create a WWW homepage containing some basic information, archives for download and the `README' file with installation instructions, and maybe a copy of the packages `PackageInfo.g' file. The responsibility for this WWW homepage is with the package authors/maintainers. If you tell us about your package (say, by mail to `support@gap-system.org') we may agree to add a link to your package homepage from the {\GAP} website and to redistribute the current version of your package via the {\GAP} download sites. We can also provide some service for producing several archive formats from the archive you provide (e.g., you produce a `.tar.gz' version of your archive and we produce also a `.tar.bz2', a `.zoo' and a `-win.zip' version from this). Please, consider to submit your package to the {\GAP} package refereeing process. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \Section{The PackageInfo.g File} We suggest to create a `PackageInfo.g' file for your package by copying the one in the `Example' package, distributed with {\GAP}, and to adjust it for your package. Within {\GAP} you can look at that file by \begintt Pager(StringFile(Filename(DirectoriesLibrary(), "../pkg/example/PackageInfo.g"))); \endtt As a first step the example in "An Example of a GAP Package" shows the information needed for the package loading mechanism of a simple package. If your package depends on the functionality of other packages, the component `Dependencies' given in the `PackageInfo.g' file becomes important, see "Requesting one GAP Package from within Another" below. The other entries become relevant if you want to distribute your package: they contain lists of authors and/or maintainers including contact information, URLs of the package archives and README files, status information, text for a package overview Web page, and so on. See the mentioned template file for a list and explanation of all recognized entries. Once you have created the `PackageInfo.g' file for your package, you can test its validity with the command `ValidatePackageInfo(filename);'. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \Section{Requesting one GAP Package from within Another} It is possible for one {\GAP} package `A', say, to require another package `B'. For that, one simply adds the name and the (least) version number of the package `B' to the `NeededOtherPackages' component of the `Dependencies' component of the `PackageInfo.g' file of the package `A'. In this situation, loading the package `A' forces that also the package `B' is loaded, and that `A' cannot be loaded if `B' is not available. If `B' is not essential for `A' but should be loaded if it is available (for example because `B' provides some improvements of the main system that are useful for `A') then the name and the (least) version number of `B' should be added to the `SuggestedOtherPackages' component of the `PackageInfo.g' file of `A'. In this situation, loading `A' forces an attempt to load also `B', but `A' is loaded even if `B' is not available. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \Section{Declaration and Implementation Part} \indextt{init.g!for a GAP package} \indextt{read.g!for a GAP package} When {\GAP} packages require each other in a circular way, a ``bootstrapping'' problem arises of defining functions before they are called. The same problem occurs in the {\GAP} library, it is resolved there by separating declarations (which define global variables such as filters and operations) and implementations (which install global functions and methods) in different files. Any implementation file may use global variables defined in any declaration file. {\GAP} initially reads all declaration files (in the library they have a `.gd' suffix) and afterwards reads all implementation files (which have a `.gi' suffix). Something similar is possible for {\GAP} packages: If a file `read.g' exists in the home directory of the package, this file is read only *after* all the `init.g' files of all (implicitly) required {\GAP} packages are read. Thus one can separate declaration and implementation for a {\GAP} package in the same way as done for the {\GAP} library, by creating a file `read.g', restricting the `ReadPackage' statements in `init.g' to only load those files of the package that provide declarations, and to load the implementation files from `read.g'. See Section~"prg:Declaration and Implementation Part" in the Programmers' Tutorial which discusses further the commands that should appear in the declaration part (i.e., in the files read with `ReadPackage' from `init.g') and in the implementation part (i.e., in the files read with `ReadPackage' from `read.g') of a package. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \Section{Standalone Programs in a GAP Package} {\GAP} packages that involve stand-alone programs are fundamentally different from {\GAP} packages that consist entirely of {\GAP} code. This difference is threefold: A user who installs the {\GAP} package must also compile (or install) the package's binaries, the package must check whether the binaries are indeed available, and finally the {\GAP} code of the package has to start the external binary and to communicate with it. We will treat these three points in the following sections. If the package does not solely consist of an interface to an external binary and if the external program called is not just special-purpose code, but a generally available program, chances are high that sooner or later other {\GAP} packages might also require this program. We therefore strongly suggest to provide a documented {\GAP} function that will call the external binary. We also suggest to create actually two {\GAP} packages; the first providing only the binary and the interface and the second (requiring the first, see~"Requesting one GAP Package from within Another") being the actual {\GAP} package. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \Section{Installation of GAP Package Binaries} The scheme for the installation of package binaries which is described further on is intended to permit the installation on different architectures which share a common file system (and share the architecture independent file). A {\GAP} package which includes external binaries contains a `bin' subdirectory. This subdirectory in turn contains subdirectories for the different architectures on which the {\GAP} package binaries are installed. The names of these directories must be the same as the names of the architecture dependent subdirectories of the main `bin' directory. Unless you use a tool like `autoconf' yourself, you must obtain the correct name of the binary directory from the main {\GAP} branch. To help with this, the main {\GAP} directory contains a file `sysinfo.gap' which assigns the shell variable `GAParch' to the proper name as determined by {\GAP}'s `configure' process. For example on a Linux system, the file `sysinfo.gap' may look like this: \begintt GAParch=i586-unknown-linux2.0.31-gcc \endtt We suggest that your {\GAP} package contains a file `configure' which is called with the path of the {\GAP} root directory as parameter. This file then will read `sysinfo.gap' and set up everything for compiling under the given architecture (for example creating a `Makefile' from `Makefile.in'. The standard {\GAP} distribution contains a {\GAP} package ``example'' whose installation script shows an example way of how to do this. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \Section{Test for the Existence of GAP Package Binaries} If an external binary is essential for the workings of a {\GAP} package, the function stored in the component `AvailabilityTest' of the `PackageInfo.g' file of the package should test whether the program has been compiled on the architecture (and inhibit package loading if this is not the case). This is especially important if the package is loaded automatically. The easiest way to accomplish this is to use `Filename' (see~"ref:Filename" in the {\GAP} Reference Manual) for checking for the actual binaries in the path given by `DirectoriesPackagePrograms' (see~"ref:DirectoriesPackagePrograms" in the {\GAP} Reference Manual) for the respective package. For example the ``example'' {\GAP} package could use the following commands to test whether the binary `hello' has been compiled; they issue a warning if not and will only load if it is indeed available. \begintt ... AvailabilityTest := function() local path,file; # test for existence of the compiled binary path:=DirectoriesPackagePrograms("example"); file:=Filename(path,"hello"); if file=fail then Info(InfoWarning,1, "Package ``example'': The program `hello' is not compiled"); Info(InfoWarning,1, "`HelloWorld()' is thus unavailable"); Info(InfoWarning,1, "See the installation instructions; ", "type: ?Installing the Example package"); fi; return file<>fail; end, ... \endtt (In fact the `AvailabilityTest' function that is actually used in the ``example'' package always returns `true', just the warnings are printed if the binary is not available. This means that the binary is not regarded as essential for this package.) You might also have to cope with the situation that external binaries will only run under UNIX (and not, say on a Macintosh). See~"ref:Testing for the System Architecture" in the {\GAP} Reference Manual for information on how to test for the architecture. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \Section{Calling of and Communication with External Binaries} There are two reasons for this: the input data has to be passed on to the stand-alone program and the stand-alone program has to be started from within {\GAP}. There are two principal ways of doing this. The first possibility is to write all the data for the stand-alone to one or several files, then start the stand-alone with `Process' or `Exec' (see~"ref:Process" and~"ref:Exec" in the {\GAP} Reference Manual) which then writes the output data to file, and finally read in the standalone's output file. The second way is interfacing via iostreams (see Section~"ref:Input-Output Streams" in the {\GAP} Reference Manual). The support for this is in its infancy. %T Is this still correct? %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \Section{Package Completion} Reading a larger package can take a few moments and will take up user workspace. This might be a nuisance to users, especially if the package is loaded automatically. The same problem of course affects the {\GAP} library, the problem there is solved using completion files (see~"ref:Completion Files" in the {\GAP} Reference Manual). Completion files make it possible to read only short function headers initially which are completed to full functions only when the functions are actually called. This section explains how to set up completion for a {\GAP} package. Completion works for those files which are read (using `ReadPackage') from the `read.g' file. (This is no real restriction as completion affects only the implementation part.) To create completion files, load the {\GAP} package, and then use the following command. \>CreateCompletionFilesPackage( <pkgname> ) This will create a new file `read.co' in the home directory of the loaded version of the {\GAP} package <pkgname> (so you must have write permissions there). When the {\GAP} package is loaded, this file is used in place of `read.g', and automatically takes care of completion. When you change files which are completed, {\GAP} will complain about non-matching CRC files and will not load them. In this case simply remove the `read.co' file and create it anew. As a {\GAP} package author you should consider including a completion file with the package. If you start {\GAP} with the command line option `-D', it displays information about reading and completion, the command line option `-N' turns completion off (as if all `.co' files were erased). (Section~"ref:Advanced Features of GAP" in the {\GAP} Reference Manual describes the options `-D' and `-N'.) %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \Section{DeclareAutoreadableVariables} Package files containing method installations must be read when the package is loaded. Note that the completion mechanism used in the main {\GAP} library (see Section~"Completion Files" in the {\GAP} Reference Manual) cannot be used for packages. For package files *not* containing method installations --this applies to many data files-- another mechanism allows one to delay reading such files until the data are actually accessed. \>DeclareAutoreadableVariables( <pkgname>, <filename>, <varlist> ) Let <pkgname> be the name of a package, let <filename> be the name of a file relative to the home directory of this package, and let <varlist> be a list of strings that are the names of global variables which get bound when the file is read. `DeclareAutoreadableVariables' notifies the names in <varlist> such that the first attempt to access one of the variables causes the file <filename> to be read. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \Section{Version Numbers} \indextt{GAPInfo.Version} A version number is a string which contains nonnegative integers separated by non-numeric characters. Examples of valid version numbers are for example: \begintt "1.0" "3.141.59" "2-7-8.3" "5 release 2 patchlevel 666" \endtt Version numbers are interpreted as lists of integers and are compared in that way. Thus version `"2-3"' is larger than version `"2-2-5"' but smaller than `"11.0"'. It is possible for code to require {\GAP} packages in certain versions. In this case, all versions, whose number is equal or larger than the requested number are acceptable. It is the task of the package author to provide upwards compatibility. Loading a specific version of a package (that is, *not* one with a larger version number) can be achieved by prepending `=' to the desired version number. For example, `LoadPackage( "example", "=1.0" )' will load version `"1.0"' of the package "example", even if version `"1.1"' is avaiable. As a consequence, version numbers must not start with `=', so `"=1.0"' is not a valid version number. The global variable `GAPInfo.Version' contains the version number of the version of {\GAP} and also can be checked against (using `CompareVersionNumbers', see~"ref:CompareVersionNumbers" in the {\GAP} Reference Manual). Package authors should choose a version numbering scheme that admits a new version number even after tiny changes to the package. The automatic update of package archives in the {\GAP} distribution will only work if a package has a new version number. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \Section{Wrapping Up a GAP Package} \indextt{zoo} The releases of {\GAP} packages are independent of releases of {\GAP}. Therefore {\GAP} packages should be wrapped up in separate files that can be installed onto any version of {\GAP}. Similarly a {\GAP} package can be upgraded any time without the need to wait for new releases of {\GAP}. Because it is independent of the version of {\GAP} a {\GAP} package should be archived from the {\GAP} `pkg' directory, that is all files are archived with the path starting the package's name. \indextt{GAPDocManualLab} The archive of a {\GAP} package should contain all files necessary for the package to work. In particular there should be a compiled documentation, which includes the `manual.six', `manual.toc' and `manual.lab' file in the documentation subdirectory which are created by {\TeX}ing the documentation, if you use the `gapmacro.tex' or \package{GAPDoc} document formats. (The first two are needed by the {\GAP} help system, and the `manual.lab' file is needed if the main manual is referring to your package. Use the command `GAPDocManualLab( packagename );' to create this file for your help books if you use \package{GAPDoc}.) Currently, the {\GAP} distribution provides archives in four different formats. \beginlist \item{-} `.tar.gz', a standard UNIX `tar' archive, compressed with `gzip' \item{-} `.tar.bz2', a standard UNIX `tar' archive, compressed with `bzip2' \item{-} `.zoo', a special version of `zoo' archives, that can essentially be used on all operating systems with the `unzoo' utility provided with the {\GAP} distribution \item{-} `-win.zip', an archive in `zip' format, where text files should have DOS/Windows style line breaks \endlist For convenience of possible users it is sensible that you archive your package also in one or several of these formats. For packages which are redistributed via the {\GAP} Web site, we offer an automatic conversion of any of the formats listed above to all the others. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% \Section{Migrating Packages from GAP 4.3 to GAP 4.4} %% %% The package loading mechanism has changed from {\GAP}~4.3 to {\GAP}~4.4. %% As a consequence, two questions arise for packages that have been %% available already in {\GAP}~4.3. %% %% \beginlist %% \item{1.} %% Is it possible to use an existing package version also in {\GAP}~4.4? %% \item{2.} %% What should be changed in order to prepare new package versions for %% {\GAP}~4.4 or later? %% \endlist %% %% The answer to question 1 is ``yes'', except that a `PackageInfo.g' file %% must be provided for the package and that two small problems might arise. %% The first of these problems is that the package might use global variables %% that are deprecated in {\GAP}~4.4 (and will be removed in {\GAP}~4.5), %% such as `BANNER' or `LOADED_PACKAGES'; %% all these variables are listed in the file `lib/obsolete.g'. %% Namely, if on loads a package already from the `.gaprc' file then these %% variables are not yet available in {\GAP}~4.4. %% The solution is to read also `lib/obsolete.g' from the `.gaprc' file %% before loading the package in question. %% The second possible problem is that the package might print a banner %% via `Print' statements in one of its files, and that {\GAP}~4.4 will %% additionally print a default banner if no banner string is contained %% in the package's `PackageInfo.g' file. %% The solution is to put an *empty* string as the component `BannerString' %% into the record in the `PackageInfo.g' file. %% %% As for question 2, it is enough to remove the now unnecessary special %% commands `DeclarePackage', `DeclarePackageDocumentation', and %% `RequirePackage' from the package's `init.g' file %% (they have been replaced by entries in the `PackageInfo.g' file), %% to remove a file `banner.g', %% to avoid explicit `Print' statements in the files, %% and to replace the use of the deprecated variables that get bound in %% `lib/obsolete.g'. %% %% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% %E