EDOS Metrics and Evaluation
The objective of this effort is to define parameters to characterize distributions, distribution edition and distribution customization*. The goal of the workpackage is to develop technology and products that will improve the efficiency of two key processes and one system. The two processes are the generation of a new version of a distribution from the previous version and the production of a customized distribution from an existing one. The system is the current inefficient mechanism of mirroring the Cooker data that needs to be replaced by a more efficient system. In the end, we want to demonstrate that we have indeed improved the processes and the system. Thus our goal is to define a set of metrics to measure the efficiency of the processes in question. These metrics will include man power as measured in man months and elapsed time.
Metrics portal
EDOS metrics portal have been set up for Mandriva, Caixa Magica, and OpenOffice at the following URLs:
Indicators
The indicator list is available from
the indicators' page.
Deliverables
All the deliverables are available from
this page.
Description of work
For the measurement to be fair, we must be sure that we address distributions of the same 'nature'. So we will need to take into account the parameters defining the actual complexity of the distribution as defined by the number of packages, the number of different hardware platforms, the speed of refreshment of the packages and so on. The goal is to show a cost improvement for the same complexity of distribution. For distribution customization, similarly parameters must characterize the degree of customization. In this case, simple experiments can be performed. In the case of the new version of a distribution doing experiments for the sake of measurement would be cost ineffective.
For the improvement of the mirroring mechanism, metrics will also be devised to measure the impact of the current system on the development. We will set a priori goals of efficiency improvement, and we will demonstrate in the end through a set of experiments and measurement campaigns that we have reached this goal. We will devise tests and experiments too, which will involve distribution companies and other users who need this kind of tools. We will also involve the Community in this process, to completely validate our approach.
