Design Issues
=============
$OpenPackages: pkgdesign.txt,v 1.2 2001/05/24 00:23:13 alex Exp $
XXX must be merged:
http://openpackages.org/pipermail/op-tech/2001-January/000442.html
General:
The OpenPackages project basically provides three things:
a) Build-tools (e.g. op-make)
b) An infrastructure to build software (bsd.pkg.mk;
a tree of ported software)
c) A new package format and how it will be registered in the
system.
This document will handle c), the package format and its exchange with
the system.
——————————————————————————–
FOREGROUND
——————————————————————————–
You should read the following texts to understand the impact of this
document:
- libh mail from JKH
- what is OpenPackages etc
- libh’s sysinstall2.txt
——————————————————————————–
INFORMATION ABOUT A PACKAGE
——————————————————————————–
Information about package (description, features etc.) could be
obtained using corresponding methods of libpkg.
——————————————————————————–
FEATURES
——————————————————————————–
Packages provide features to the system.
Each package has at least one feature, the name of the package, and
an unlimited list of additional features.
The additional features can also be general features, e.g. “webserver”,
“mailtransportagent”, “printerspool” etc.
Different packages can supply the same general feature.
Non-general features are required to have a feature version.
General features are expected to have a timestamp of the feature
supplied (i.e. “2001-04-27″).
Feature is a name with no spaces and hyphens.
Feature versions are builded as in the FreeBSD Ports collection.
A package can contain an optional script that checks if features
provided by the package conflict with features in the system.
Package dependencies are built on these features and its versions.
[If the required feature is a general feature (e.g. "webserver"),
a package might want to depend on a feature newer than e.g. 6 months
nevertheless!]
The package dependency for each feature contains three versions:
1) Minimal required version
2) Recommended required version
3) Maximal required version.
If a feature isn’t installed on the system:
a) The package contains an optional list of packages
that contain this feature. The administrator is prompted
to choose the best one (can be configured to automatically
use the one with the strongest recommendation, which is the
first in the list). (For me, it’s always annoying to
see apache modules install apache13 by default and not
apache13-mod_ssl, which I wan to have!)
b) The package contains no list of packages.
The package database is queried and a package with the
recommended version of the feature is installed. If there are
multile packages supplying this feature with this version,
prompt the admin (or configurable to do a default action).
Can happen for ghostscript and ghostscirpt-nox11 ports for example.
If a package contains platform dependent binaries it should require
special platform feature.
——————————————————————————–
MAIN PACKAGE AND SUBPACKAGES
——————————————————————————–
Each package contains files which are to be installed during package
installation. Package also may contain subpackages which may be
optionally installed. Subpackages don’t have their own installation
prefixes (they use main package installation prefix), subpackages
provide no features of their own and therefore may not install files
affecting features provided by the main package. Subpackage has no
subpackages. User may choose to install or not to install each
subpackage along with the main package. Subpackages are not recorded
in the database as a separate structure, if subpackage has been
installed it’s files are owned by the main package and will be
uninstalled during main package uninstallation.
A good example of what
to be put into subpackage are manual pages and documentation which
user may omit for installation to save disk space.
Note that it’s not a good idea for Apache-modules for example, since
these have to be updated seperately.
It would be nice to have a standard set of subpackages like doc, static,
headers, etc. I can see systems cramped for storage space telling
the package system never to install the doc subpackage.
Not everything will have certain packages, like some things don’t
need a headers package. We’d want a knob in bsd.pkg.mk to allow
it to decide whether to build one or not.
XXX: Terminology matters: Decide on the terminology “subpackage”
——————————————————————————–
SYSTEM PACKAGE
——————————————————————————–
There is a package called SYSTEM which cannot be removed. It owns no
files. Records corresponding to this package are inserted into the
database during its creation.
SYSTEM package provides the following features:
- OSNAME (FreeBSD, NetBSD, Linux, …)
- OSVERSION
- PLATFORM (i386, alpha, …)
- CPU (386, 486, Pentium, …. ; MMX, 3DNow, ISSE)
- OPVERSION (the thing that is stored
in /var/db/port.mkversion at the moment)
In case of hardware upgrade/downgrade, a hardware_upgrade function
should be called to update the SYSTEM package records and check
all the packages for fitness into this hardware upgrade (e.g.
3DNow optimized MP3 encoders).
——————————————————————————–
INSTALLATION OF FILES
——————————————————————————–
This is done as the current pkg_tools install the files.
See also the “FEATURES” section.
However:
1.) Checks if the current system meets requirements by calling
i check_requirements()
ii check_conflicts()
iii check_users_groups_requirements()
iv check_suggestions()
i. This is handled as described in the “FEATURES” section.
An additional script (security!) can be run, that checks for
other requirements that – for any reason – can’t be expressed
by the “features” mechanism. Note that this case is a design
flaw then and shouldn’t happen.
ii. General features generally conflict on a system, e.g. two
mail transport agents. The administrator should be
prompted. If the admin decides, that this is a migration
from one mailserver to another (as example), this installation
is considered a upgrade installation which will either
remove the other package as part of the usual upgrade process
(see below) or (in case of sendmail in the base tree, which is
replaced by Postfix, as example), registers, that the base-sendmail
is disabled (How this can be done must be decided on later).
An optional script is run (security!) that checks for conflicts.
There are way too many things that could happen that a script
would be justified for this case.
Eric:
Are you describing the case where i already have a package
installed which provides mail-transport-agent and i try to
install another? Debian handles this by simply requiring that all
mail-transport-agents Conflict with other mail-transport-agents.
This is due to the fact that newly installed daemons are
automatically started after installation (yes, this is dumb, it
wasn’t my idea :-/). If we behave intelligently and not start
daemons automatically, we won’t have these kinds of problems.
Will:
I think allowing a script to be specified that will decide what would be
done if this kind of situation is encountered is the best approach.
Because not every kind of feature will have the same implications
regarding conflicting packages. A smart shell script would probably do
the job most efficiently and most flexibly.
An optional script can be supplied that is able to resolve
conflicts. It can either be run before or after installation
of the package. However, if it returns FALSE, the admin
is prompted and the installation can be made undo.
If it returns TRUE, the conflict is seen as resolved and
everything is ok.
— Free Diskspace is checked. If package won’t fit on the system,
abort.
iii. Users and Groups that a package needs (e.g. bind/postfix etc)
must be created accordingly.
iv. (as in Debian:) Packages can declare a Suggests or Recommends
on other packages. A Suggests is a very weak dependency;
it simply suggests to the user that a package may be useful
in conjunction with this one. A Recommends is stronger,
but it still isn’t absolute; it is expected that package
manager frontends will automatically select Recommends
(though you can tell them not to), but the command-line
tools do not.
2.) Checks if features provided by this package are already
in the system. If already in the system and only the version
is higher, consider this as a “upgrade installation” (needed
below). If the new package is marked as a development version
but the installed version is a stable version, prompt the
admin.
4.) Register package with its features etc in the system.
5.) Installs each non-sysinstall specific file supplied with the
main package and directories leading to this file.
a. If directories leading to a file being installed are not present
they are created.
b. Checks if file is already present.
If present, checks what packages own present file using the
database.
Replaces or patches every file that is listed in a
replacement/patch list. Save undo information since
this replacement is probably a package-upgrade or a
security patch.
If installed file is not listed, don’t replace, since
this might damage anything, do warning/error.
In the case this installation is marked as an
“upgrade installation”, and the old file
belongs to the package that is currently updated,
install the file.
If installed file doesn’t exist, blindly replace and
print informational message, or print error message
that it is impossible to patch that file.
c. Inserts in the database record corresponding for the file
installed or patched.
d. Inserts in the database records corresponding for each directory
(except root) leading to files being owned by the package.
6.) If package contains subpackages call the Subpackage installation
routine passing a list of subpackages that shall be installed.
This should either be by a user-prompt, or commandline flags:
–subpackages=blah1,blah2,,doc,libs,headers,examples,…
–skip-subpackages=doc,examples
Installs each file of each subpackage chosen for
installation using the same rules as for files of the main package.
7.) If package contains references to other packages to be installed
install these. If any of references cannot be installed
(package file cannot be found or insolvable problems during
installation or user aborted installation) the main package
installation is undone.
8.) Inserts in the database undo (undo script is automatically
generated, see Undo section) and uninstall records for the package.
9.) Call any install scripts that come with the package.
Comment:
I suppose this is a decent default, but this definitely needs to
be configurable. Since code that makes it to the source
repository is pretty well trusted, many users will simply want to
trust packages from that source and run the maintainer scripts.
That’s how we’ve been doing things in Debian for years and there
have never been any reports of malicious maintainer scripts.
However, it is important to be able to configure this kind of
thing based on the origin of a package. I may trust packages from
OpenPackages, but when i add Joe Hacker’s package repository to
my list and start installing packages, i may want signature
verification going on (though there really isn’t much point in
that in this example, unless the web of trust helps you).
——————————————————————————–
UNINSTALLATION
——————————————————————————–
1.) Call any uninstall scripts that come with the package.
(again: security! only for signed packages)
2.) Gets list of packages requiring features provided by the package
being uninstalled. For each package that needs a feature of
this package that is not in another package print information message
(e.g. list of packages and the features required) and abort
deinstallation.
3.) For each file owned by the package being uninstalled:
a) If file exist and is owned by other packages, it is
removed or left using the rules described in the Files ownership
section.
b) If file exists, is not owned by other packages but the md5
checksum changed, alert the user (configureable).
a) If file doesn’t exit, alert the user (configureable).
A systemwide configuration facility is justified, because we
could allow the user to state beforehand that they would like a
certain action to happen when they run into b) or c) or the
package isn’t signed etc. This sort of thing could be added to
the SYSTEM package in the database.
[This part could probably also merged into the "PACKAGE INSTALLATION
CONSISTY CHECKING" section. On uninstall, the package is then
first checked for consistency. See also there.]
4.) Remove each directory owned by the package if it is not owned by
other packages. If a directory is not owned by other packages and
it is not empty, it is left but saved in an extra list including
an information that this package owned the directory.
If it might be emtpy later, the administrator can check the
list (can also be done automatically) and remove the directory
if either empty or wanted.
However, if the a new package is installed owning this directory,
it should be removed from this extra list of course.
5.) Erase “undo” information.
However, see “Files ownership” for the case that multiple
packages own this file (reverse patch must be done if patched
during installation).
6.) Erases package record from database.
> Just make sure that removal of a package doesn’t automatically
> terminate running daemons. When you roll configuration meta-data
> into the mix, in many cases you can do something like the following:
>
> – Obtain/Validate the new package for installation (crypto/sig/whatnot)
>
> – Remove the non-configuration related portions of the previous
> package from the system, but keep the running instances going.
>
> – Install the new package version
>
> – Handle configuration changes/upgrades. Ensure (as well as you can
> that the new configuration will work). This could be something
> sorta like mergemaster(8) from FreeBSD.
>
> – When we know the configuration jives, restart the service.
>
> At Progeny we’ve been thinking a lot about the integration of
> configuration and meta-data within our packaging system. How can
> we pre-answer difficult questions from a configuration template/script
> on a subsite/site/world-wide basis for config data?
>
> If you can’t provide some form of integration with configuration
> and packages, you might as well go back to using cp and chmod. 
——————————————————————————-
UNDO
——————————————————————————–
Undo information is stored in an extra directory. Each file that
is touched in any way is kept in this directory with its relative
path.
Additional undo information should be saved (I defenitely believe
there is something that can be saved as well, but currently I can’t
imagine, what).
Eric Gillespie:
: I’m not sure this is the right way to do it. Again, i think that
: appropriate logging of what actions are taken at each stage
: combined with a local site policy for configuration file handling
: (as well as having a sane default handler) could give you a
: rollback facility.
[Note: This could of course be done as well, to give a redundant
undo method.]
Undo information is automatically generated and stored in the database
Undo steps:
1.) For each file reference stored in the undo database:
Check if file with original pathname exists.
a) If it does not exist put saved file (if saved is patch, raise an
exception), find packages owning file put and update their records.
b) If file exists find other packages owning this file. If package
being undone is the latest (by installation time) replace or patch
original file and update records of other packages owning this file.
c) If file exists and package being undone is not the latest (by
installation time) original file left untouched.
2.) Remove each directory owned by the package if it is not owned by
other packages. If a directory is not owned by other packages and
it is not empty, do the same thing as done on usual uninstallation
in this case (see above).
3.) Erases undo record from DB.
4.) Erases package record from DB.
Note: We need to think up a way that is for undoing upgrade
of packages!!!
A general-purpose checkpoint/logging mechanism in the packaging database
is recommended. This would allow for backing but a
current command that failed mid-install, and for more business or
production oriented practices like: “Make my machine like last
Thursday, I think Bob whacked it after then.”
As for logging/transactions. The answers to questions like:
– Which user/admin installed this package
– Why was it installed (as a prerequisite? for which?)
This could allow (with a site policy knob of course) removal of
dependencies when the packages that pulled-them-in are removed.
These features are lacking in DPKG/APT in Debian, and are really
necessary when you want to do “high-level” package management
on a large scale.
Ask C. Stephen Gunn sometime about how this can/should all revolve
around install(1) from the ground up. (He suggested that).
——————————————————————————–
UPGRADE
——————————————————————————–
Upgrade is an installation of a new package followed by uninstallation
of all packages providing the same features as the new package but of
different versions (it could be upgrade or downgrade).
This is only for non-general features.
Uninstallation is only done, if the main-feature of the to-uninstalled
package is the feature in question.
Note: Imagine a port that is called “foo” and which is renamed to “bar”
later. bar’s package has two features then: “foo” and “bar”,
with the version of bar.
Uninstallation of upgraded packages is done by package handling
library upon successful completion of upgrade. Note that
package uninstall scripts coming with old packages must not be
called for security reasons.
Why does that work (i.e. install new, uninstall old):
Image foo-1 is upgraded to foo-2.
foo-1 file list:
a, b, c
foo-2 file list:
a, b, d.
File that changed between foo-1 and foo-2:
b
Now, foo-1 is installed. foo-2 is installed over the existing installation.
The installation of foo-2 is considered an “upgrade installation” of foo-1.
Both, a and b are listed as files of foo-1, so they are replaced even
if not listed in the replacement list of foo-2-package.
File d is installed, file c is not touched.
Then, foo-1 is uninstalled.
Files “a” and “b” now belong to foo-2, so foo-1 won’t uninstall them.
File c is removed.
–> Bingo!
——————————————————————————–
UNDO TOOLS
——————————————————————————–
There should be an extra program which lists the undo information for
the user for every specified package.
This way a user can look what files have been patches/removed in an
installation of this package.
[XXX Alex's personal mark, please don't touch]
——————————————————————————–
PACKAGE INSTALLATION CONSISTENCY CHECKING
——————————————————————————–
This check could either check if all installed files still exist and
also have the correct MD5. This helps admins for finding back-doors
and rootkits etc. It also helps to find disk-damage.
Also, it could call a script provided with the package, which checks
the consistency, which returns a list of problems encountered.
Each problem is list of package_file and list of list of the following
elements: string describing problem, e.g. “md5″, “absent”, old value,
new value, suid bit, etc…
Receipts of installed files (configuration or not) should be guaranteed
by a checksum. You can provide a site/machine policy for what to do when
we encounter a file that has been modified since install. Eventually
you could/should leverage this meta-info for security audits for things
like tripwire(1).
This should probably be expanded on a little by security freaks.
——————————————————————————–
FILES OWNERSHIP OF PACKAGES
——————————————————————————–
libpkg’s functions own/disown are used to add/remove file ownership of
the specified package.
A.) If installation puts file somewhere and this file does not exist
there the file considered to be owned by the package. Record about
ownership is to be put in the database including file permissions,
MD5 etc (see above).
B.) If installation wants to put file somewhere and this file already
exists database is queried for packages owning this file.
a) No other packages own this file. Depending on user original file
could be saved in the same directory by adding some suffix
(e.g. “.sysinstall-save-”) or original file could be
copied to some directory before writing file from package. Package
owns this file.
Note: Beter than renaming is saving it AND storing
undo-information for this file.
b) File is owned by other packages.
User is provided with information about other packages owning
this file and depending on user:
1. original file could be saved like in a), it will be owned by those
packages, new file is put and owned by the new package only. I.e.
records of other packages are updated to own saved file but not
newly put file.
2. Original file is left as is, new package owns it as well as old
packages, file from the package is not put.
3. Original file is not saved and new file is put over it. New
package owns it as well as old packages, records for old packages
are probably updated to match new file (md5, inode etc.)
(Undo-information is saved nevertheless!)
4. File is a text file and could be patched to be compatible with all
packages. New package owns it as well as old packages, records for
old packages are probably updated to match new file (md5, inode etc.)
(this must be handled in consistency checking!)
5. Installation of the new package is aborted and undone. User is
asked to remove conflicting packages and restart installation.
Will:
All of this behavior should be configurable, with knobs to support doing
it the various general ways we can come up with, and then one to allow
outsourcing action to a script. Of course, command line options to
select (if not specified in database) action and/or override config
database choice should also be choices.
Eric + Will:
> It is very important to allow the admin to specify local hooks in
> addition to and/or in place of the default handlers. This should
> probably be done in run-parts[1] order, maybe with the default
> hook being 50default. So overriding the default hook would be as
> simple replacing the 50default script.
Sounds good, but we’d have to carefully think out how to do that. The
way things are going, I think we’ll have to offer OP users a set of
standard shell scripts so they won’t have to make their own from scratch
for the various actions where customized behavior might be desirable.
C.) The file is shared between several packages and one of these
packages is being uninstalled. If this file was patched during
installation of the package being currently uninstalled reverse
patch is applied. Otherwise the file is left untouched.
D.) The file is shared between several packages and one of these
packages is being undone. If packages being undone is latest by
installation time among all packages owning this file the file is
replaced with saved (by undo system) version. Otherwise file is left
untouched.
E.) The file is owned by a package being uninstalled only. If MD5 of
the file is the same as recorded in the database the file is silently
removed. If MD5 of the file is not recorded in the database the file
is silently removed.
If MD5 of the file is recorded in the database
and differs from MD5 of the real file user is asked to choose: view
the file and then return to this menu, remove the file, leave the file
and probably have problem of unreferenced file later (note: this
can be avoided by saving this file in the same database that
handles non-emtpy directories), move the file to
another location (preferably into directory which is not handled by
the package system).
A good default should be chosen that is presented to the admin,
that can be forced in the configuration by the admin.
F.) Directory D is owned by package A only. D contains file F which is
not owned by A. User uninstalls A.
a) F is owned by B, D is not owned by B. Ownership of D is transfered
to B, D is not removed.
b) F is not owned by any package. User chooses: view F and then
return to this menu, remove F, leave F and D (not owned by
anything) and probably have problem of unreferenced file later,
move F to another location (preferably into directory which is
not handled by the package system).
So I guess it is time to talk about install(1). I would assert
that all files being installed, whether from a Makefile, or from
a package installer should rely on install(1) (or say libinstall) to
do their dirty work. It’s the UNIX way.
Vest the know-how of atomic installation of a file in a single
command, or library, and your entire system is up to speed on new
features when you add them. An enhanced install could (I’ve seen
patches under Free Software licenses for some of these):
– Sanity check a site policy for a file being installed,
do we install set-uid files in /usr/sbin ?
– Sanity check a file being installed against the current
copy of the file. Did the mode change? Did we get the wrong
owners? Should we alert/interrupt the user?
– Register the file (atomically) with the package manager
database. This could be extended to teach the package manager
about third-party applications installed outside the management
of ports/packages. Given sufficient meta-info, you could use
this to shoehorn the existing BSD-base into the package-manager’s
domain of control. It’s a convenient way to keep the package
database up to date with a daily build-world (for instance).
– Build packages as a non-root user. Using an install(1) look-alike
we simply create the package by “installing” the files into
the package tarball. This only works if you ensure that all
files are put in place by install(1).
– Install should keep the previous versions of installed files
in case you need to back out the change. This trades disk-space
for insurance. If you know what you’re doing, don’t need the
insurance, or don’t have the disk space, twist the knob over
to the “bend over” setting. This makes the “undo” functionality
above much easier to implement. You simply ask install to
take you back in time.
Yeah.. Yeah.. I know Journaled filesystems and all the jazz,
ReiserFS will change the world. Feh.
– Perform auditing/logging in accordance with your site policy,
let people know when/where/why software is installed.
——————————————————————————–
PACKAGE CREATION
——————————————————————————–
New package could be created by calling libpkg’s create_package
function. User should prepare directory with files and directories
to be placed in a new package.
This create_package proc does the following:
1. If package file being created already exists exception is raised.
2. Builds list of files to be put into package
3. Output package stream is opened using Stream_create and then
package_creator is opened using create().
4. Reads in a description of package and saves it in the meta-data for
the package. Same for other meta-data.
Author is saved (yes?)
5. Save features of package.
List of provided features if generated contains the same feature as
package name with version extracted from package name or 0.0.
(and more?)
6. Manual given library dependencies for binary programs are recorded.
Note that automatic checking for library dependencies can result
in a desaster and is therefore not wanted (Debian had this).
More discussion of this problem can be found at
http://lists.debian.org/debian-devel-announce-0011/msg00003.html
and http://lists.debian.org/debian-dpkg-9901/msg00080.html .
7. The package creator’s manual list of conflicts is added.
This happens for OpenSSH port against the OpenSSH version in
the base tree; same for Perl, tcsh, truncate(1) etc.
It could also happen that a port conflicts with another port.
8. Structure of file/tree is saved. Default
prefix is /usr/pkg, as has been decided recently.
If there are files in man subdirectory they are put into man
subpackage. If there are files in doc subdirectory they are put into
doc subpackage — by default. People aren’t forced to accept this,
they can prevent this and of course add additional doc files.
This is more a recommendation.
If there are files in examples subdirectory, they are put
into examples subpackage.
If there are example configuration files (manual list),
put into configuration-examples-subpackage.
If package contains a patches (dir::filename.opkg-patch or similar).
these are considered to be patches for already present (at
the installation time) files with the name w/o .opkg-patch)
If there are files with suffix .opkg-append they are considered to be
appended to probably already present (at the installation time) files
with names sans these suffixes.
9. Save scripts supplied to package.
10. Puts files supplied by the package creator into archive
11. Sign the package
12. Call libpkg’s close().
——————————————————————————–
LOGGING
——————————————————————————–
????
A one-line syslog entry summarising activity if system modifications
were made, eg:
opkg: package ‘foobar-1.4′ upgraded to ‘foobar-1.6′
——————————————————————————–
PSEUDO-PACKAGES
——————————————————————————–
This are meta-packages, that only depend on other packages.
Upgrade of those packages also update all dependencies, even if the
required features are already in the system, but not in the latest version.
This can be done by always requesting the latest version in the dependency,
thus no special handling for pseudo-packages is needed.
Debian’s task packages (for example, task-web-devel) provide a simple
way for the user to choose “system profiles” at system installation.
This is also useful for packages like gnome-desktop, which can keep
track of the myriad and ever-changing list of packages which GNOME
consists of.
——————————————————————————–
GARBAGE COLLECTION
——————————————————————————–
1. Filesystems are scanned (at night, once a week, once a month) for
a directory owned by a package but containing files (or directories)
owned by other packages or not owned at all. This information is
saved in the database and message is sent to sysadmin. Using
sysinstall user could put some of the conflicts found into ignore
list in the database, uninstall conflicting packages, remove or move
not referenced files etc.
Another approach is to create some scripts and let the admin manually
scan for these files. I’m sure, though, that about 50% of the admins
won’t know of this scripts then.
: This is for catching the sorts of files that might have been created by
: the package but not part of the original archive? This could be
: supplemented with some hinting:
:
: – A hint associated with a directory that indicates that any files in
: the directory should be treated (by uninstall, etc.) as belonging to
: the package/feature.
: – A hint indicating that nominated files or directories should be
: considered as being owned by the package (used if eg. the package
: creates a temporary or working directory somewhere).
:
: E.g. users should be able to add “nominated files”, e.g. games
: score files, to the package database.
2. Packages could be divided in two categories: those which have
immediate use (e.g. emacs or ftpd) and those which have no immediate
use (e.g. shared library). I mean disk could be flooded with
not required by other packages shared libraries (e.g. obsolete
versions of them). Package creator could add attribute to his
package marking it has no immediate use. Database could be scanned
for those packages which are not required by other packages at the
moment, report results to user and allow her to uninstall them.
Another script to add to the cronjob by the sysamdin (probably
enabled by a knob, disabled by default or similar).
——————————————————————————–
INTERNATIONALIZATION
——————————————————————————–
Here must be thought what to do for international users.
All outputs could be using gettext() or any similar implementation
that still doesn’t conflict with a BSDL-like license.
Also support for wchar and other charsets is required.
——————————————————————————–
ADDITIONAL PACKAGE INFORMATION
——————————————————————————–
Each package has additional meta information which are stored at the
beginning of the zip archive:
It contains:
– Vendor (FreeBSD, NetBSD, OpenBSD, Darwin, RedHat, …)
– Signature
– Size of package
– Features of package
– Origin of package in the OpenPackage tree (if available).
See FreeBSD’s -o stuff.
– …
We should be able to store all of this information on a centralized
database that can be accessed fast and easy, so that they can get
e.g. the size before they install the package (and its dependencies).
A centralized database (e.g. on FTP/HTTP Server or database query)
would be better, since a static database on the computer of the
user would probably outdated soon.
The package-tools could then get extended by a tool/flag that
just queries the database for information about a package and its
dependencies.
C. Stephen Gunn has a paper from Progeny about package meta-information,
signatures and the like, which can be very useful.
——————————————————————————–
OTHER STUFF
——————————————————————————–
- package meta data should be accessible without former
extraction of file. this is why ZIP is the package format
of choice (hubertf said, this isn’t really needed (–fast-dir)).
The format also needs to support metadata that is separate from the
compressed, encrypted part of the image, so we can put metadata that
depends on the image data (like signing keys) in the package.
XXX ONE MUST TEST THIS!!
- can belong to diffferent categories (list)
- the packaging stuff is available as a lib rather than as tools,
so that I can merge stuff into libh.
The stuff that is currently in the pkg_tools sourcedode
should be moved to the lib. A new API for the lib has to be
created first, based on the ideas in this document.
the pkg-tools only call the lib-functions.
- so called features.
you could then pkg_addfeature “ghostscript” and it searches
all packages that have the feature “ghostscript” and
lets you choose the correct one.
This also helps dependencies, e.g. “require apache”
or “require apache 1.2.4″ (at least 1.2.4).
It doesn’t matter what version.
- support for the PARANOID flag (for setuid binaries that are
!audited)
This variable should be a number that indicates the security status of a
package. For example:
0 – Default. No known security holes, but not audited.
1 – Audited. No known security holes.
2 – Not audited. Security holes found.
3 – Audited. Security holes found.
For case #2 and #3, we have another set of variables:
I. Number of holes.
II->N. Hole #N
a. Severity
b. Versions of package affected.
c. OS’s affected.
1. versions affected, may want to do some
relational math here, i.e. only versions X Y Z
of the package are vulnerable on OS version Z.
.
.
n. Other?
For case #3 only, we would also display the relative auditing involved.
This would be a number from 1 to 10, depending on the degree of auditing
done. More information on precisely what has been audited might be
found at the package-specific security page, below.
In addition to displaying the above information, we should also print a
webpage URL, something like this:
http://www.openpackages.org/security/package.html
Also, the above information could be compared to the system to send the
system administrator a warning whenever a security hole is found,
possibly running as a cronjob (in addition to package db updates). And
of course, such action can be done
- N Full support for wildcards in both “src” as well as binary pkgs
- N Some support for bulk building all packages, to ensure pkgs work
(see
ftp://smaug.fh-regensburg.de/pub/NetBSD/pkgstat/20000819.1320/broken.html
for an example)
- N Support for Linux and Solaris (it is being used!)
- N Installed and binary pkgs contain the size of the pkg in bytes.
Also, the size including all required pkgs is available.
- N Pkgs can be deleted recursively, e.g. to nuke jpeg and everything
that needs it (very handy)
- N Detection of out-of-date pkg_* tools if bsd.pkg.mk uses a feature
that the installed pkg_* tools don’t provide.
- O fake (tmpdir install, pkg create, pkg_add instead of traditional
‘raw’ install w/out a package)
- O sha1/md5/cksum instead of just cksum
- O ‘piecemeal’ packages, aka one egcs compile & fake install becomes
egcs-java*tgz, egcs-fortran*tgz, egcs-c++*tgz; kde is another example
- O flavors (multiple targets/pkg names per package for various
compile time options, e.g. use a different UI, toolkit, etc).
Note: Not sure if this belongs into this document
- It would be VERY nice to save some port configuration stuff along with
the package. I.e. if you compiled a port with optional MySQL support
and you want to upgrade this port, the saved configuration is read
and MySQL is added by default. This way, it’s harder to shoot
into your feet while updating.
Stuff that libh can extend:
– TCL scripts that come with packages and can be executed in a
sandbox enviroment with restricted command set and only
with checksum and maximum security enforements.
this is very good for security advisories etc.
(or use Perl etc) (see sysinstall2.txt, top of file)