InterJournal Complex Systems, 194
Status: Submitted
Manuscript Number: [194]
Submission Date: 980716
Revised On: 980716
Response to the Comments [186] of [106]
Author(s): Javier Dolado ,Alvaro Moreno

Subject(s): CX.65

Category: Brief Article


The main criticism of the article is the presumed misunderstanding of the CMM, and the equivalence assumed between ISO9000 and CMM. First of all, our intention was not to criticize the CMM as the main goal. This short article tries to point out the relativity of some types of process evaluations with respect to the product obtained in the process, and to support it with the current research in the field. Perhaps paragraph 2.1 is misleading in this sense, but it only describes the CMM in the words of the SEI literature (see reference [15]). We have not invented that description. Personally, I have attended several lectures by representatives of the SEI (some well known), and in none of them have I heard words like "self-organization of the tasks" or the use of "emergent methods". The CMM is more similar to an "evolutionary path" than to an "evolutionary model", the path being the steps towards building "disciplined software processes" (see the second reference of M.C. Paulk below). It is true that CMM does not prescribe any specific software process or any other technique, as the referee states, but the CMM establishes the "key process areas" for achieving a higher level (in order). Many these lectures about the CMM take the technical perspective (that is what people want in the software field) and some of the ideas that the referee asserts are not immediately obvious to the practitioner. One of the main possible ways of improvement is to apply the best practices of the field. Hypothetically, they could include self- organization, but at this moment no application of the CMM has been published with that perspective. The CMM dictates Continuous Improvement, but in an orderly way. The CMM has received many critics, and some of them take a similar perspective to that of the present article. The following are an example: 1. J. Bach, "The Immaturity of the CMM", American Programmer, September 1994; also available on the web 2. Some of the works of Capers Jones 3. Other references that take a slightly different perspective to that of the CMM: "The Fifth Discipline" by P. Senge and "Thriving on Chaos" by Tom Peters. Other drawbacks of the CMM are: - The difficulty of applying it to small businesses; this is a recurring theme in all the meetings of the different SPINs (Software Process Improvement Networks). - The subjectivity of the evaluations made by the CMM: this has led to the development of the new standard SPICE model (Software Process Improvement and Capability Determination) at the European Software Institute (Bilbao, Spain) -ESI-. The aim of the SPICE model is to include the best of both worlds, CMM and ISO. See "SPICE. The Theory and Practice of Software Process Improvement and Capability Determination", by K. El Emam et al. (eds.), IEEE Computer Society, 1998. - Due to the inability of the initial CMM to cope with the new demands of the systems engineering, new CMMs are being developed (People CMM, etc.) The subjectivity of some of the concepts of the CMM model is made clear when the referee states that "many people misunderstand the model" and that "... the behavioral aspects are not made more obvious". Also it is suspicious that a model can accommodate concepts for which it was not intended to manage, or which depends on the interpretation. The referee also presumes some difficulties in the CMM. However, we are not denying the intrinsic value of CMM, or the other evaluation methods (SPICE or ISO) -that would be overly na´ve-. In fact, benefits of applying the CMM have already been reported. We merely wish to point out that the evaluation of the process according to those models MAY not tell much about the true capability of the business with respect to the products it builds. As the referee acknowledges, the behavioral aspects are not made obvious. This is one of the reasons why the evaluations made by the CMM are sometimes categorized as "subjective". Either the model is too vague or the interpretation is left to the observer, and often both. This is one of the reasons why the next standard will be SPICE, which is based on the CMM, but attempts to eliminate all its subjectivity. It is true that the CMM is much more flexible than ISO9000. However, the SEI people have already established the equivalence of the ISO model with the CMM. In fact, it is assumed that a level 3 is equivalent (more or less) to the certification ISO 9000, as it has been described in - M. Paulk, "How ISO 9001 Compares with the CMM", IEEE Software pp. 74-83, January 1995. It is even possible to combine both approaches simultaneously as seen in - I. Rozman et al., "PROCESSUS - Integration of SEI CMM and ISO quality models", Software Quality Journal, Vol. 6, N. 1, March 1997, pp. 37-63 No further discussion was undertaken in the present article about ISO and CMM because it was not our intention to cover those technical aspects. This paper does not intend to discredit the CMM, merely to point out its limitations with respect to the product/process relationship. The main thrust of the article is that the evaluation of the process may not tell much about the product, as happens in nature. There are new software processes which, at the present stage, cannot be easily accommodated within structured models. In relation to this discussion it is worth quoting M.C. Paulk: "The CMM does not guarantee that software products will be successfully built or that all problems in software engineering will be adequately resolved." in M.C. Paulk, "The Evolution of the SEI's Capability Maturity Model for Software", Software Process, Improvement and Practice, Vol. 1, N. 1, pp. 3-16, 1995 The main idea of our article is precisely the existing disonance between processes and product, and the application of mechanical perspectives to the evaluation of processes, whether they are carried out under ISO, CMM or SPICE. The metaphor established in the paper tries to exemplify how the product may not tell very much about the process, and viceversa. A great deal remains to be done to implement the concepts of the complex systems area. We understand from the referee's comments that the main problem is how we have presented the CMM in the paper (paragraph 2). Since we think (and the referee assumes that too) that the rest of the paper is not as problematic as paragraph, we have decided to rewrite some of the phrases paragraph 2.1. We will try to accommodate some of the referee's comments by adding sentences to the conclusion, while maintaining the original spirit of the submission. Specifically we will refer to the technical (or practical) application of the CMM as it is carried out in present practice. We do not think that the rest of the paper conveys any misinterpretation related to the CMM.

Retrieve Previous Revision's Abstract
Submit referee report/comment

Public Comments: