Software Process Assessment

Advertisements

Previous Page
Next Page  

A software process assessment is a disciplined examination of the software processes used by an organization, based on a process model. The assessment includes the identification and characterization of current practices, identifying areas of strengths and weaknesses, and the ability of current practices to control or avoid significant causes of poor (software) quality, cost, and schedule.

A software assessment (or audit) can be of three types.

  • A self-assessment (first-party assessment) is performed internally by an organization's own personnel.

  • A second-party assessment is performed by an external assessment team or the organization is assessed by a customer.

  • A third-party assessment is performed by an external party or (e.g., a supplier being assessed by a third party to verify its ability to enter contracts with a customer).

Software process assessments are performed in an open and collaborative environment. They are for the use of the organization to improve its software processes, and the results are confidential to the organization. The organization being assessed must have members on the assessment team.

Software Process Maturity Assessment

The scope of a software process assessment can cover all the processes in the organization, a selected subset of the software processes, or a specific project. Most of the standard-based process assessment approaches are invariably based on the concept of process maturity.

When the assessment target is the organization, the results of a process assessment may differ, even on successive applications of the same method. There are two reasons for the different results. They are,

  • The organization being investigated must be determined. For a large company, several definitions of organization are possible and therefore the actual scope of appraisal may differ in successive assessments.

  • Even in what appears to be the same organization, the sample of projects selected to represent the organization may affect the scope and outcome.

When the target unit of assessment is at the project level, the assessment should include all meaningful factors that contribute to the success or failure of the project. It should not be limited by established dimensions of a given process maturity model. Here the degree of implementation and their effectiveness as substantiated by project data are assessed.

Process maturity becomes relevant when an organization intends to embark on an overall long-term improvement strategy. Software project assessments should be independent assessments in order to be objective.

Software Process Assessment Cycle

According to Paulk and colleagues (1995), the CMM-based assessment approach uses a six-step cycle. They are −

  • Select a team - The members of the team should be professionals knowledgeable in software engineering and management.

  • The representatives of the site to be appraised complete the standard process maturity questionnaire.

  • The assessment team performs an analysis of the questionnaire responses and identifies the areas that warrant further exploration according to the CMM key process areas.

  • The assessment team conducts a site visit to gain an understanding of the software process followed by the site.

  • The assessment team produces a list of findings that identifies the strengths and weakness of the organization's software process.

  • The assessment team prepares a Key Process Area (KPA) profile analysis and presents the results to the appropriate audience.

For example, the assessment team must be led by an authorized SEI Lead Assessor. The team must consist of between four to ten team members. At least, one team member must be from the organization being assessed, and all team members must complete the SEI's Introduction to the CMM course (or its equivalent) and the SEI's CBA IPI team training course. Team members must also meet some selection guidelines.

With regard to data collection, the CBA IPI relies on four methods −

  • The standard maturity questionnaire
  • Individual and group interviews
  • Document reviews
  • Feedback from the review of the draft findings with the assessment participants

SCAMPI

The Standard CMMI Assessment Method for Process Improvement (SCAMPI) was developed to satisfy the CMMI model requirements (Software Engineering Institute, 2000). It is also based on the CBA IPI. Both the CBA IPI and the SCAMPI consist of three phases −

  • Plan and preparation
  • Conduct the assessment onsite
  • Report results

The activities for the plan and preparation phase include −

  • Identify the assessment scope
  • Develop the assessment plan
  • Prepare and train the assessment team
  • Make a brief assessment of participants
  • Administer the CMMI Appraisal Questionnaire
  • Examine the questionnaire responses
  • Conduct an initial document review

The activities for the onsite assessment phase include −

  • Conduct an opening meeting
  • Conduct interviews
  • Consolidate information
  • Prepare the presentation of draft findings
  • Present the draft findings
  • Consolidate, rate, and prepare the final findings

The activities of the reporting results phase include −

  • Present the final findings
  • Conduct an executive session
  • Wrap up the assessment
Advertisements