Software Measurement Validation
Validating the measurement of software system involves two steps −
- Validating the measurement systems
- Validating the prediction systems
Validating the Measurement Systems
Measures or measurement systems are used to asses an existing entity by numerically characterizing one or more of its attributes. A measure is valid if it accurately characterizes the attribute it claims to measure.
Validating a software measurement system is the process of ensuring that the measure is a proper numerical characterization of the claimed attribute by showing that the representation condition is satisfied.
For validating a measurement system, we need both a formal model that describes entities and a numerical mapping that preserves the attribute that we are measuring. For example, if there are two programs P1 and P2, and we want to concatenate those programs, then we expect that any measure m of length to satisfy that,
m(P1+P2) = m(P1) + m(P2)
If a program P1 has more length than program P2, then any measure m should also satisfy,
m(P1) > m(P2)
The length of the program can be measured by counting the lines of code. If this count satisfies the above relationships, we can say that the lines of code are a valid measure of the length.
The formal requirement for validating a measure involves demonstrating that it characterizes the stated attribute in the sense of measurement theory. Validation can be used to make sure that the measurers are defined properly and are consistent with the entity’s real world behavior.
Validating the Prediction Systems
Prediction systems are used to predict some attribute of a future entity involving a mathematical model with associated prediction procedures.
Validating prediction systems in a given environment is the process of establishing the accuracy of the prediction system by empirical means, i.e. by comparing the model performance with known data in the given environment. It involves experimentation and hypothesis testing.
The degree of accuracy acceptable for validation depends upon whether the prediction system is deterministic or stochastic as well as the person doing the assessment. Some stochastic prediction systems are more stochastic than others.
Examples of stochastic prediction systems are systems such as software cost estimation, effort estimation, schedule estimation, etc. Hence, to validate a prediction system formally, we must decide how stochastic it is, then compare the performance of the prediction system with known data.
