CMMI Institute slide template

Why two constellations are better than
one?
(CMMI-DEV ML5 + CMM-SVC ML5)
María Julia Orozco Mendoza
Ultrasist, Chief Operations Officer
Alejandro A. Ramírez Ramos
Ultrasist, Governance, Architecture & QA Leader
Organization Overview
• Ultrasist is a Mexican Enterprise with more than 20 years of
successful experience in IT Consulting and Software
Development.
• As a result, Ultrasist has been working and has been evaluated
as a high maturity organization for more than ten years (CMMDEV ML4 since 2004, CMMI-DEV ML5 since 2009 & CMMSVC ML5 since 2015).
• Ultrasist is a medium sized company, focused on the delivery of
quality products, based on quality processes, with the flexibility,
adaptability and scalability to tackle the biggest challenges
within the software and IT industries.
Business Challenge
• We signed a multi-annual big contract for our software factory,
based in our organizational performance baseline. (A program with
several incremental projects in it)
• Early in the projects, we identified process capability issues
• Some of our most specialized phases (architecture, security
development, transition) started to take longer than estimated
• Our process performance models predicted that we were not going
to meet our goals
• We had a national security contract with not much space to
negotiation, without clear visibility and control of the target
environment.
• We urgently needed to improve our process performance
Business Challenge
Capability issues …
Capacidad de proceso de Desarrollo (desviación en esfuerzo)
Valor individual
Gráfica I
Histograma de capacidad
2
LCS=2,085
1
_
X=0,865
0
Objetivo
LES
E specificaciones
O bjetiv o 1,0
LE S
1,1
LCI=-0,355
1
5
9
13
17
21
25
29
33
37
0,0
Gráfica de rangos móviles
Rango móvil
0,8
1,2
1,6
2,0
Gráfica de prob. Normal
1,6
A D : 0,353, P : 0,448
LCS=1,499
0,8
__
MR=0,459
0,0
LCI=0
1
5
9
13
17
21
25
29
33
37
-1
0
1
Dentro
Desv.Est.
0,4066
Cp
*
Cpk
0,19
PPM
281863,59
1,5
1,0
0,5
Dentro de
General
Especificaciones
20
25
30
Observación
2
Gráfica de capacidad
Últimas 25 observaciones
Valores
0,4
35
40
General
Desv.Est.
0,4914
Pp
*
Ppk
0,16
Cpm
0,07
PPM
316409,49
The Plan
1. Make a root cause analysis to identify the underlying
cause
2. Identify and analyze alternative solutions
3. Execute the selected solution in the program / remove
root causes
4. Evaluate the results
5. Implement the solution at organizational level
1.The Analysis (CAR)
PROBLEM: We were not meeting our process performance objectives for some kind of
projects
1 - Why?
The specialized phases (architecture, security, non-functional testing,
transition) were taking more time than estimated
2- Why?
The agile development teams were not enough to tackle this challenge
3 - Why?
We didn't have enough specialists assigned to the program
4 - Why?
It was not cost-effective to hire too many specialists
5 - Why?
We hadn’t identified the minimum,
maximum and average use of the
specialists
The specialists were more
expensive than regular developers
and there were too much idle times
1.The Analysis (CAR)
PROBLEM: We were not meeting our process performance objectives for some kind of
projects
Conclusion
Those specialized phases
(architecture, security, non-functional testing, transition)
were actually behaving like a service
2.The Solution
PROBLEM: We were not meeting our process performance objectives for some kind of
projects
We decided to use the CMMI for Services constellation
3.The Implementation
SD, CAM,
WP & WMC
CMMI SVC
We implemented a pilot with the following:
 Encapsulated the specialized phases
as a service
 Analyzed the demand, resources,
capacity and performance needed, for
our services
 We established the SLA’ s
 Updated our service request system
to include those services
 Updated our estimation techniques
 Extended our planning, monitoring and
control capabilities from our dev
projects to our services.
3.The Implementation
Optimizing the resource usage, based on the value of the
expected demand …
3.The Implementation
SCON, SST,
IWM, IRP
SD, CAM,
WP & WMC
CMMI SVC
 We identified the essential functions
and resources that support them
 We defined an strategy and a plan
based on that analysis to ensure the
continuity of the service
 We tailored our process to integrate
the appropriate use of the services
 We planed, developed and deployed
the transition of our systems
 We extended our incident
management system to include the
service’s incidents
3.The Implementation
QWM, STSM
SCON, SST,
IWM, IRP
SD, CAM,
WP & WMC
CMMI SVC
 We quantitatively evaluated the results
of the services’ implementation
 Based in the execution results, we
decided to include these services in our
catalog of organizational standard
services
 We formalized the attributes and SLA of
each service
4.The Evaluation
QWM, STSM
SCON, SST,
IWM
SD, CAM,
WP & WMC
 Through an statistical experiment we
found out that the process performance
mean had changed, showing a true
benefit
CMMI SVC
5. Deploy at Organizational Level
QWM, STSM
SCON, SST,
IWM
PP, PMC, IPM,
QPM,
RD, TS, PI,
VER, VAL
SD, CAM,
WP & WMC
 CMMI DEV & CMMI SVC co-existence
CMMI SVC
CMMI DEV
5. Deploy at Organizational Level
CMM-SVC
PA’s
CMMI-DEV
PA’s
 CMMI DEV & CMMI SVC co-existence
CMMI SVC
CMMI DEV
The Results
Prueba de desviación estándar (esfuerzo) de 2 muestras para (DEV) vs (DEV + SVC)
Informe de resumen
Estadísticas
Prueba de desviación estándar
¿Es Desv Horas A mayor que Desv Horas D?
0
0,05
0,1
> 0,5
Sí
No
Tamaño de la muestra
Media
Desviación estándar
IC de 90%
Desv Horas A
Desv Horas D
37
0,86721
0,48767
(0,4215; 0,5905)
37
1,0002
0,025479
(0,0215; 0,0316)
P = 0,000
La desviación estándar de Desv Horas A es
significativamente mayor que Desv Horas D (p < 0,05).
Gráfica de comparación de las desviaciones estándar
Desv Horas A
Comentarios
Desv Horas D
0,00
0,15
0,30
0,45
Distribución de los datos
Compare la dispersión de las muestras.
Desv Horas Antes
Desv Horas Después
0,0
0,4
0,8
1,2
1,6
0,60
-- Prueba: Usted puede concluir que la desviación estándar
de Desv Horas A es mayor que Desv Horas D en el nivel
de significancia de 0,05.
-- Gráfica de comparación: Los intervalos en rojo indican
que las desviaciones estándar difieren. Considere el
tamaño de la diferencia para determinar si tiene
implicaciones prácticas.
-- Distribución de los datos: Compare la dispersión de las
muestras. Busque datos poco comunes antes de
interpretar los resultados de la prueba.
The Results
Capacidad de proceso Desarrollo con Servicios (desviación en esfuerzo)
Gráfica I
Histograma de capacidad
Valor individual
1,1
LCS=1,0938
Objetivo
E specificaciones
LE I
0,9
O bjetiv o 1,0
LE S
1,1
LCI=0,9091
1
11
21
31
41
51
61
71
81
91
0,90 0,93 0,96 0,99 1,02 1,05 1,08
Rango móvil
Gráfica de rangos móviles
Gráfica de prob. Normal
A D: 0,260, P : 0,704
LCS=0,1134
0,10
0,05
__
MR=0,0347
LCI=0
0,00
1
11
21
31
41
51
61
71
81
0,9
91
1,0
Dentro
Desv.Est.
0,03078
Cp
1,08
Cpk
1,07
PPM
1172,04
1,05
1,00
0,95
80
85
90
Observación
1,1
Gráfica de capacidad
Últimas 25 observaciones
Valores
LES
_
X=1,0015
1,0
0,9
LEI
95
100
Dentro de
G eneral
E specificaciones
General
Desv.Est.
0,02894
Pp
1,15
Ppk
1,14
Cpm
1,15
PPM
558,78
The Results
• We successfully integrated the CMMI-SVC constellation to
our established process, solving our process capability
issues
• We improved the customer satisfaction for our national
security projects
• The process performance improvement leveraged the ROI
Lessons Learned
• The high maturity practices helped us to identify CMMIDEV was not enough for some kind of projects
• The use of CMMI-SVC helped us to focus on the missing
pieces
•
•
•
•
Capacity and Availability Management
Service Continuity
Strategic view of our services
Accuracy of project’s and service’s estimation
• The CMMI-DEV maturity at Ultrasist provided a solid
foundation for the SVC adoption
• As an outsourced software factory, SVC provided a new
strategic way to deliver our work, in benefit to our clients