This paper reports a joint CMMI/MPS appraisal experience performed in Belo Horizonte, Brazil, at Synos Technologies, in June and July 2009, under four points of view: from MPS assessors’, from the CMMI lead appraiser’s, from the MPS Implementing Institution, and from the appraised company. The lessons learned and a set of recommendations for future appraisals using such new modality are also presented. 1. OVERVIEW The MPS.BR Program is a nationwide mobilization program that aims at improving software process in Brazilian organizations both large companies and small and medium-size enterprises (SME). Thus, it was created the MPS Model with three components: MPS Process Reference Model (MR-MPS), MPS Process Assessment Method (MA-MPS) and MPS Business Model (MN-MPS). MR-MPS and MA-MPS are conformant with the International Standards ISO/IEC 12207 [ISO/IEC, 2008] and ISO/IEC 15504 [ISO/IEC, 2003], compatible with CMMI-DEV® [SEI, 2006], based on software engineering best practices, and according to the reality of Brazilian firms. The MPS.BR Program (acronym of the Portuguese expression “Melhoria de Processo do Software Brasileiro” or Brazilian Software Process Improvement) was created in December 2003 by the Association for Promoting the Brazilian Software Excellence (SOFTEX), a private not-forprofit organization aiming at promoting competitiveness of the Brazilian software industry, which holds a network of 22 SOFTEX agents (in 23 cities of 13 Brazilian states). There are over 1,300 SOFTEX affiliated firms companies – 70% are micro and small-size enterprises (see www.softex.br). Recently, the MPS.BR Program coordination has introduced the joint CMMI/MPS appraisal modality. Such appraisals can be performed by observing the following conditions: • The MPS competent assessor (equivalent to the SCAMPI lead appraiser) must be an experienced competent assessor; • The CMMI lead appraiser must be a member of the MPS appraisal team, must have attended the course “Introduction to MPS.BR”, and must have no bounds with the company responsible for implementing the CMMI or the MPS at the company to be appraised; • The CMMI lead appraiser must speak Portuguese in order to reduce translation problems and to improve communication and interaction to the interviewees and team members; • The entire appraisal team must have attended the MPS.BR and the CMMI Introductory courses, and fulfill the requirements for becoming a member of both MPS and CMMI appraisal teams; ® CMMI and CMM are SEI/CMU trademarks. SCAMPI is a SEI/CMU service mark. MPS.BR, MR-MPS, MA-MPS, MN-MPS, and iMPS are SOFTEX service marks. 1 The MPS.BR Program has one level of provisional assessor and three levels of competent assessors: initial (authorized to lead MPS.BR appraisals on levels G and F); intermediary ((authorized to lead MPS appraisals on levels E, D and C) and experienced (authorized to lead MPS appraisals on level B and A). • The initial MPS appraisal must be carried out during the same time and place as the CMMI readiness and must not be done virtually, with a duration longer than or equal to the duration recommended to the MPS level to be appraised; • All appraisal team members must be present at the initial appraisal and at the final appraisal; • The presence of an official MPS based SPI consultant hired by the appraised organization must be granted in all moments during the initial MPS appraisal. This is important to assure the organization will correctly understand what must be fixed before the final appraisal and also the organization it will have proper guidance of the SPI consultant. • The appraisal must be approved by the SOFTEX considering, besides the above-listed items: o the appraisal process that will be adopted by the CMMI lead appraiser, which must be described in details and attached to the appraisal hiring communication 2 to SOFTEX submitted by the Assessment Institution (AI) of MPS hired for the MPS appraisal. ♣• CMMI lead appraiser's statement indicating that neither he/she, nor the Implementation Institution (II) of MPS and/or the SEI Partner he/she belongs to has provided consulting services for the last three years to the company that will be appraised. By observing such criteria, the first joint CMMI/MPS appraisal was successfully performed at Synos Technologies. On such appraisal, the CMMI lead appraiser was Mr. Andrés Rubinstein from Liveware Inc. and the MPS competent assessor was Dr. Ana Regina Rocha from COPPE/UFRJ Assessment Institution (AI) of MPS. The appraisal also had the participation of three provisional assessors, two from COPPE/UFRJ and one from QualityFocus, as well as two Synos representatives. The Implementation Institutions (II) of MPS actuating at the company were FUMSOFT and ASR. This article describes the lessons learned on this joint appraisal and some recommendations for future appraisals, considering different points of view: from MPS assessors’, from the CMMI lead appraiser’s, from the Implementation Institution (II) of MPS, and from the appraised company. 2. ABOUT SYNOS Synos Technologies was founded on February 2003 with a business vision based on its partners’ experience: worldwide acknowledged IT companies. With strong market relationship, Synos is acknowledged by its professionalism by major global companies, such as Oracle, IBM, Microsoft, and Red Hat, with whom it maintains business partnerships. During its first year of operation, Synos achieved its first million in sales. On its second year, it registered a 300% growth. In June 2009, its turnover is around R$12.000.000,00 (US$ 6 million) per year, and for 2009 the forecast is reaching R$20.000.000,00. Synos Technologies is a company focused on technological integration solutions. Through its business lines, which are software house, outsourcing/body shop, consulting, training, and software licensing, it operates on different segments of the Brazilian national market, always providing solutions with the highest quality standards and fidelity to its customers’ demands. With more than 90 customers all over the country, and more than one hundred projects, Synos currently counts with 120 employees, being 50 of them allocated at its Software House. Synos has a philosophy strongly oriented towards processes and software quality. Its main software development process, Synos UP, has undergone continuous improvements in order to extend its productivity, add new Software Engineering techniques, and adhere to the main quality 2 All MPS.BR appraisals are previously approved by SOFTEX. models in terms of market processes, such as CMMI [SEI, 2006], MR MPS [SOFTEX, 2009a], and ISO 9001 [ABNT, 2000]. Synos UP has been created for organizing and optimizing Synos Software House production. Its processes have been structured based on several models, references and techniques consolidated on the market, such as UML, PMBOK and agile methodologies. Since March 2006, Synos has decided to invest on implanting MR MPS, starting from level F, investing approximately R$ 600.000,00, being appraised on May 2007. On November 2007, it started a new improvement cycle, seeking MR-MPS level C. This new cycle was a challenge for the company since it is a major maturity leap, by incorporating twelve new processes, besides evolving the already deployed processes for the new capability required by level C. The company’s SEPG was restructured by admitting new employees skilled on the new processes to be implemented. The activities have followed the process improvement flow of the Synos UP itself, which consists on the following steps Conception (defining the scope and work goals), Preparation (creating new processes and improving the already existing ones), Implantation (defining pilot projects, training teams, and process usage follow-up), and Appraisal (official evaluation employing some kind of maturity model). When the improvement project was close to the appraisal step, a major opportunity occurred to the company, with the possibility of simultaneously performing CMMI (level 3) and MPS (level C) model appraisals. In spite of the greater complexity arising from meeting both models simultaneously and the fact of the works originally focusing only on MR MPS level C, the Synos Management Board, the Software House Management Board, and SEPG have decided to accept this major challenge, knowing that gains would be equally large for the company and that both models, MPS and CMMI, are compatible, thus enabling the simultaneous appraisal. This way Synos started its simultaneous CMMI and MPS appraisal on June 2009, obtaining a unique achievement on the following month, thus becoming the first company to perform and succeed on a joint appraisal of such two models. This was the best prize for eighteen months of effort, approximately R$ 500,000.00 worth of investments and tens of employees involved on this achievement. Soon after this appraisal, the process improvement works continued for MR MPS level A and CMMI level 5, besides continuing its ISO 9001:2008 project, intending to certify the entire company on this quality model and that is on the final implantation phase. 3. LESSONS LEARNED FROM JOINT CMMI/MPS APPRAISAL This first joint CMMI/MPS appraisal was a very enriching experience, which has provided countless learned lessons. For a better understanding they will be grouped according to four points of view: from MPS assessors’, from the CMMI lead appraiser’s, from the Implementation Institution (II) of MPS, and from the appraised company. For better appraising each testimony, they will be transcribed below. 3.1. Lessons learned from MPS assessors’ point of view “Performing a joint CMMI/MPS appraisal is not a trivial task. It requires experience in appraisals, deep knowledge of both models, and great skill for dealing with the points where differences arise. Not only the MPS competent assessor and the CMMI lead appraiser shall be experienced and knowing both models. The provisional assessors h