Software Measurement

The framework for software measurement is based on three principles −

• Classifying the entities to be examined
• Determining relevant measurement goals
• Identifying the level of maturity that the organization has reached

Classifying the Entities to be Examined

In software engineering, mainly three classes of entities exist. They are −

• Processes
• Products
• Resources

All of these entities have internal as well as external entities.

• Internal attributes are those that can be measured purely in terms of the process, product, or resources itself. For example: Size, complexity, dependency among modules.

• External attributes are those that can be measured only with respect to its relation with the environment. For example: The total number of failures experienced by a user, the length of time it takes to search the database and retrieve information.

The different attributes that can be measured for each of the entities are as follows −

Processes

Processes are collections of software-related activities. Following are some of the internal attributes that can be measured directly for a process −

• The duration of the process or one of its activities

• The effort associated with the process or one of its activities

• The number of incidents of a specified type arising during the process or one of its activities

The different external attributes of a process are cost, controllability, effectiveness, quality and stability.

Products

Products are not only the items that the management is committed to deliver but also any artifact or document produced during the software life cycle.

The different internal product attributes are size, effort, cost, specification, length, functionality, modularity, reuse, redundancy, and syntactic correctness. Among these size, effort, and cost are relatively easy to measure than the others.

The different external product attributes are usability, integrity, efficiency, testability, reusability, portability, and interoperability. These attributes describe not only the code but also the other documents that support the development effort.

Resources

These are entities required by a process activity. It can be any input for the software production. It includes personnel, materials, tools and methods.

The different internal attributes for the resources are age, price, size, speed, memory size, temperature, etc. The different external attributes are productivity, experience, quality, usability, reliability, comfort etc.

Determining Relevant Measurement Goals

A particular measurement will be useful only if it helps to understand the process or one of its resultant products. The improvement in the process or products can be performed only when the project has clearly defined goals for processes and products. A clear understanding of goals can be used to generate suggested metrics for a given project in the context of a process maturity framework.

The Goal–Question–Metric (GQM) paradigm

The GQM approach provides a framework involving the following three steps −

• Listing the major goals of the development or maintenance project

• Deriving the questions from each goal that must be answered to determine if the goals are being met

• Decide what must be measured in order to be able to answer the questions adequately

To use GQM paradigm, first we express the overall goals of the organization. Then, we generate the questions such that the answers are known so that we can determine whether the goals are being met. Later, analyze each question in terms of what measurement we need in order to answer each question.

Typical goals are expressed in terms of productivity, quality, risk, customer satisfaction, etc. Goals and questions are to be constructed in terms of their audience.

To help generate the goals, questions, and metrics, Basili & Rombach provided a series of templates.

• Purpose − To (characterize, evaluate, predict, motivate, etc.) the (process, product, model, metric, etc.) in order to understand, assess, manage, engineer, learn, improve, etc. Example: To characterize the product in order to learn it.

• Perspective − Examine the (cost, effectiveness, correctness, defects, changes, product measures, etc.) from the viewpoint of the developer, manager, customer, etc. Example: Examine the defects from the viewpoint of the customer.

• Environment − The environment consists of the following: process factors, people factors, problem factors, methods, tools, constraints, etc. Example: The customers of this software are those who have no knowledge about the tools.

Measurement and Process Improvement

Normally measurement is useful for −

• Understanding the process and products
• Establishing a baseline
• Accessing and predicting the outcome

According to the maturity level of the process given by SEI, the type of measurement and the measurement program will be different. Following are the different measurement programs that can be applied at each of the maturity level.

Level 1: Ad hoc

At this level, the inputs are ill- defined, while the outputs are expected. The transition from input to output is undefined and uncontrolled. For this level of process maturity, baseline measurements are needed to provide a starting point for measuring.

Level 2: Repeatable

At this level, the inputs and outputs of the process, constraints, and resources are identifiable. A repeatable process can be described by the following diagram.

The input measures can be the size and volatility of the requirements. The output may be measured in terms of system size, the resources in terms of staff effort, and the constraints in terms of cost and schedule.

Level 3: Defined

At this level, intermediate activities are defined, and their inputs and outputs are known and understood. A simple example of the defined process is described in the following figure.

The input to and the output from the intermediate activities can be examined, measured, and assessed.

Level 4: Managed

At this level, the feedback from the early project activities can be used to set priorities for the current activities and later for the project activities. We can measure the effectiveness of the process activities. The measurement reflects the characteristics of the overall process and of the interaction among and across major activities.

Level 5: Optimizing

At this level, the measures from activities are used to improve the process by removing and adding process activities and changing the process structure dynamically in response to measurement feedback. Thus, the process change can affect the organization and the project as well as the process. The process will act as sensors and monitors, and we can change the process significantly in response to warning signs.

At a given maturity level, we can collect the measurements for that level and all levels below it.

Identifying the Level of Maturity

Process maturity suggests to measure only what is visible. Thus, the combination of process maturity with GQM will provide most useful measures.

• At level 1, the project is likely to have ill-defined requirements. At this level, the measurement of requirement characteristics is difficult.

• At level 2, the requirements are well-defined and the additional information such as the type of each requirement and the number of changes to each type can be collected.

• At level 3, intermediate activities are defined with entry and exit criteria for each activity

The goal and question analysis will be the same, but the metric will vary with maturity. The more mature the process, the richer will be the measurements. The GQM paradigm, in concert with the process maturity, has been used as the basis for several tools that assist managers in designing measurement programs.

GQM helps to understand the need for measuring the attribute, and process maturity suggests whether we are capable of measuring it in a meaningful way. Together they provide a context for measurement.