Overview

Overview

This section deals with some of the distribution vendor's processes, from getting the latest and updated versions of software up until packaging a new distribution release. This section does not contain (for now, at least) everything, for example: QA processes.

The following processes are covered in this section: Getting the latest versions, making code changes and finally packaging the distribution.

Distribution editors keep updated with the latest software either by themselves or by contributors who send new packages. Information about new or updated software is available from sites such as Freshmeat - see Publishing.

Many packages get into a distribution due to contributors who want to see them in the next release. For example, a contributor might see a nice piece of software in Freshmeat, check the distribution vendor's site to see if anyone has contributed that package, and if not - package the source and send it to the vendor.

When a software piece is considered vital to the distribution (for example, the Linux kernel), it is common for the editor to be involved personally in the update.

Many packages' source code has to be updated to fit with the distribution. Some changes are minor (for example putting on the vendor's logo) while some may be larger (for example configuration managers). Those changes are made by the contributors and the diff file is sent to the editor.

FInally, a release is declared, containing the pieces of software which were tested and seen fit for the release.

We'll now consider some case studies with different distribution vendors.

Mandriva

The "next" release in Mandriva is called "Cooker". It is by all means a distribution, albeit it might be a bit unstable because it is in testing status. About every 6 months a new stable release is out. Before the release (about 3 months before) a beta version is already out for users to play around with and submit bugs. Later, as testing (and subsequent fixing) proceeds, the version becomes more stable and is declared a release candidate.

The packages in a Mandriva linux distribution are divided into two categories: main and contrib. Main includes the packages which are essentially the "sponsored" release. These packages have been tested and verified before making it into the next release. As the Cooker version becomes more and more stable, "freezes" are declared and no more new contributions to the packages are permitted, except for fixing serious bugs.

The other category, contrib, contains pieces of software which are not part of the core of the distribution, but they are still supposed to work along with the release. When "freezes" are declared, it is still possible to contribute and submit new and updated packages to contrib.

Whenever a contributor packages a new piece of software, he has to put it in the "incoming" folder of the Mandriva FTP server. Also, he has to notify the Cooker mailing list and the ditribution editor, so he'll know that it exists and that he has to decide what to do with it. Contributors are encouraged to package only the source code (source packages) and not the binaries, since the editor has to perform some "sanity checks" on the code (to prevent trojan horses, non-licensed software, other legal problems and so on).

Source code is often changed to fit with Mandriva's distribution. Usually, the contributor who packaged the software also makes some changes. In other cases, it will be the distribution editor's duty to tweak the code. Either way, the original sources are kept along with a diff file containing all the changes that were made.

Each package in the distribution has a maintainer. The maintainers are persons who are "trusted" by Mandriva. A new contributor is called "external contributor" and he can only upload packages in the way described previously. He can't be their maintainer. The maintainer is an "internal contributor". Those are people who were once "external contributors" but were deemed trustworthy by the editor due to their activities up until now.

The distinction between Cooker and the latest Official release, along with the 6 months lag between official releases, can cause some problems when new software appears. An interesting case is told here: Apparently, the much-acclaimed browser Mozilla Firefox 1.0 was released about _one week_ after the Official Mandriva 10.1 release. Since then, Firefox has been packaged for Mandriva linux, but - since development work is always on the Cooker version - it was compiled to fit with the Cooker library versions, and not necessarily the 10.1 Official versions. This poses a choice for regular users ("regular" users are users who are not likely to compile Firefox on their own machine with their own library versions): either wait six months for the next official release, or download the latest version and pray… A "backports" project was initiated to deal with these kinds of problems: Cooker packages are "backported" so they'll also work with previous official releases. Even when a backport is prepared, some issues are left: for example, how to differentiate the original RPM from the "backported" one. The problem is that in the RPM format it is not possible to specify which version of a distribution works with it...

Some related links in Mandriva's site:

http://qa.mandriva.com/twiki/bin/view/Main/CookerHowTo

http://qa.mandriva.com/twiki/bin/view/Main/RpmHowTo

http://www.mandriva.com/en/community/developers/development_model

Debian

Debian has three distributions: "Stable", "Testing" and "Unstable". Actually there are two more distributions - "Experimental", which contains volatile elements which - should they have bugs - may bring down the whole system (for example, a new file system), and "Frozen" which is a temporary distribution before "Testing" becomes "Stable". The "Experimental" distribution is not meant for personal use, but rather as a platform for trying out new ideas and testing them. The first 3 distributions are considered okay for home use (even "Unstable", though not recommended for beginners).

A new package usually gets into the "Unstable" distribution (though there are some exceptions, as noted here). This distribution contains packages which are supposed to be - on the whole - stable, according to their developers and sites like Freshmeat. However, those packages haven't been tested and integrated into the whole Debian distribution and so are considered for now to be "Unstable". It is important to note that because the packages in "Unstable" do have some degree of stability, there are some users who prefer to have the "Unstable" distribution installed on their machines - just to be among the first to get new and updated software.

An automatic process evalutes nightly the packages in the "Unstable" distribution. If certain criteria for a package are met (spent X days in "Unstable", has fewer critical bugs than its respective version on "Testing" and additional criteria) then the package is moved to "Testing". "Testing" is the distribution which is the release candidate.

Whenever the Debian release manager decides (which is not very often) a freeze is declared on the "Testing" distribution. At that point buggy packages are removed from the distribution and no new packages can be let in except for bug fixes. After an additional period of time the distribution goes into a "deep freeze" when no changes at all are allowed, except installation-related. When the distribution proves to be stable enough - it becomes the new "Stable" distribution and distributed as such. As implied before, the "Stable" distribution is not updated very often and so fits corporate users and servers, where keeping up with the "bleeding edge" is not a requirement.

As in the Mandriva case, Debian contributors make changes to packages' source code for them to fit with the whole Debian distribution, and the changes are kept alongside the original source. As opposed to Mandriva, however, the Debian hierarchy has no "internal" and "external" distinction among contributors. Practically every one can become a maintainer of one package or more. The maintainer is actually the one responsible for uploading packages to the various distriubtions, while the developers send their source code and diff files to the maintainers.

Debian has a very extensive set of "backports" of software which is available in the "Unstable" or "Testing" distributions, so users of "Stable" could also use it. This gives rise to a similar set of issues to that of the Mandriva case.

Some (quite extensive) documentation on Debian's site:

Debian Policy

Debian maintainer's guide

Debian developer's reference

• • Main.AssafSagi - 31 Jan 2005

Topics Wp4

{metadata}