Onderzoeksmethoden 2/het werk/2007-8/Groep01

CMMI-5: Europa vs. India

John Akkermans

Felix Cillessen

Alexander Rachmann

Master Informatiekunde, onderzoeksmethoden vervolg

21 januari 2008 v2.2

Eindpresentatie: Onderzoeksmethoden (vervolg) Groep 1.pdf

---

Probleemstelling

Een aantal laatstejaars master-studenten van de Bèta-faculteit van de Radboud Universiteit in Njjmegen gaan in India onderzoek doen naar CMMI, Capability Maturity Model Integration. Te bezoeken organisaties zijn gelegen in Bangalore en Hyderabad. De Indiase IT industrie claimt voor het leeuwendeel te opereren op de hoogst haalbare niveaus aangaande diverse kwaliteitsnormen. Enkele van de normen zijn CMM, CMMI, P-CMM, ISO, BS en six sigma. In veel Europese landen zoals Nederland wil dit niet vlotten. In dit onderzoek bekijken we de antwoorden van een achttal inhoudsdeskundigen en in hoeverre deze antwoorden met elkaar in overeenstemming zijn. de vraag die is voorgelegd luidt; Kunt u in het kort aangeven wat volgens u de meest relevante redenen zijn dat Indiase bedrijven wel in staat zijn om dergelijke CMMI 5 certificeringen te behalen terwijl dat in heel veel andere Europese landen zoals NL niet wil vlotten.

In een onderzoeksvraag gegoten:

Veel Indiase bedrijven zijn in staat volgens CMMI-5 te werken terwijl dat in heel veel Europese landen zoals Nederland niet wil vlotten. In hoeverre zijn de redenen van experts wat dat betreft met elkaar in overeenstemming?

Methode

Onderzoeksfunctie

De functie van ons onderzoek is vergelijkend. Hierbij vergelijken we antwoorden van inhoudsdeskundigen gegeven per Email. Hoe bepaal je in hoeverre de gegeven antwoorden met elkaar in overeenstemming zijn.

Strategie

Om de onderzoeksvraag te beantwoorden, hebben we 15 experts gemaild met het verzoek om antwoord te geven op de vraag:

Een aantal laatstejaars masterstudenten van de Bèta-faculteit van de Radboud Universiteit in Njjmegen gaan in India onderzoek naar CMMI. Te bezoeken universiteiten en bedrijven zijn gesitueerd in Bangalore en Hyderabad.

Kunt u in het kort aangeven wat volgens u de meest relevante redenen zijn dat Indiase bedrijven wel in staat zijn om CMMI-5 certificeringen te behalen terwijl dat in heel veel andere Europese landen zoals Nederland niet wil vlotten?

Uiteindelijk hebben we van 9 mensen ook daadwerkelijk antwoord gekregen, waardoor we nu dus de beschikking hebben over de antwoorden van 9 mensen met betrekking tot onze probleemstelling.

Deze data in tekstuele vorm zullen we gaan analyseren, en wel met de volgende stappen:

1. ieder afzonderlijk analyseert de data door de argumenten te markeren en vervolgens te taggen
2. we voegen het werk samen en maken het consistent
3. we analyseren en rapporteren de resultaten

De analyse van de resultaten houdt in dat we zullen gaan kijken naar de gevonden tags, en de tags zullen gaan tellen. Als een bepaalde tag veelvuldig wordt genoemd, kan worden vastgesteld dat die reden een voorname reden is met betrekking tot de probleemstelling.

Planning

To-Do

Omschrijving	Werktijd	looptijd
Onderzoeksplan opstellen	2 uur	1 week
Experts selecteren en benaderen per e-mail	2 uur	1 week
Antwoorden afwachten	/	2 weken
Antwoorden ordenen	1 uur	3 dagen
Afzonderlijke analyse en tagging	3 uur	1 week
Samenvoegen en aanbrengen consistentie	1 uur	1 week
Resultaten rapporteren	2 uur	1 week
Conclusies rapporteren	2 uur	1 week

Conceptueel model

Domein

B: Het domein omvat Europese en Indiase bedrijven

Variabelen

RI: Redenen waarom Indiase bedrijven werken volgens CMMI-5

RI(B) = tekstueel antwoord waarom Indiase bedrijven werken volgens CMMI-5

RI -> { … }

Meetniveau: nominaal

RN: Redenen waarom Europese bedrijven slechts sporadisch werken volgens CMMI-5

RN(B) = tekstueel antwoord waarom Europese bedrijven slechts sporadisch werken volgens CMMI-5

RN -> { … }

Meetniveau: nominaal

Relaties

Er wordt een vergelijking gemaakt tussen RI en RN:

We onderzoeken of er relaties aanwezig zijn tussen de redenen van de experts onderling. Dus of de door hen opgegeven Redenen RI gelijkheden vertonen, in welke mate. In schematische weergave:

En of de door hen opgegeven Redenen RN gelijkheden vertonen, in welke mate. In schematische weergave:

Onderzoek

• 8M:OM2-Groep1-Alex
• 8M:OM2-Groep1-John
• 8M:OM2-Groep1-Felix
• 8M:OM2-Groep1-Algemeen
• 8M:OM2-Groep1-Process log

Results

We are busy with analyzing our data. It takes a lot of time to harmonize the tags. Surprisingly, we had a lot of differences: it occured some times that one of our group marked an argument as a disadvantage, whereas the other two tagged the same argument as an advantage (see #14 & 15 and 16 & 17 in the second table). Obviously, these things have to be discussed.

One tag is structured as follows:

<<D|A>> <<D|I>> <<0|1>> <<name>>

whereas

<<name>> is the tag by itself
<<D|A>> is metadata, which determines if this is an disvantage (D) or an advantage (A)
<<D|I>> is metadata, which determines if this (dis)advantage refers to dutch companies (D) or indian companies (I)
<<0|1>> is metadata, which determines if the argument has weak importancy (0) or strong importancy (1)

Importancy describes the way the experts states the argument. Examples for weak importancy are:

• I think ...
• In my opinion ...
• Maybe ...

Examples for strong importancy are:

• There is ...
• It is obvious that ...
• The fact that ...

raw data

#	(meta)tag	1	2	3	4	5	6	7
1	AI1 profileren	X
2	AI1 High Quality	X
3	AI1 Low cost	X
4	AI1 easy to implement	X
5	DD1 old habbits	X
6	DD1 unclear benefits	X
7	DI0 need for certification	X
8	AI0 CMMI is a must	X
9	AD0 just a trend	X
10	AI0 established	X
11	DD0 CMMI will not become a must	X
12	AI1 well promoted	X
13	AI1 need to demonstrate high quality		X
14	DD1 no need to demonstrate high quality		X
15	AI1 easy to get		X
16	DI1 misleading		X
17	AI1 Indians tend to follow		X
18	DD1 Europeans tend to lead		X
19	AI1 easy to get		X
20	DD1 difficult to get		X
21	AI0 easy to get			X
22	AI1 Indias tend to follow				X
23	DD1 E. tend to lead				X
24	DD1 CMMI-Leves are less important				X
25	AI1 CMMI is a must				X
26	AI1 proof of quality				X
27	AI1 need for Marketing					X
28	DD1 no need for Marketing					X
29	AI1 keep employees working					X
30	AI1 Indias tend to follow					X
31	DD1 E. tend to lead					X
32	AI1 need for Marketing						X
33	DD1 no use for CMMI						X
34	AI1 young industry						X
35	AI1 Indias tend to follow						X
36	DD1 E. tend to lead						X
37	AI1 high qualified employees							X
38	AI1 young industry							X
39	AI1 Indias tend to follow							X
40	DD1 E. tend to lead							X
41	DD1 no use for CMMI							X
42	AI1 need for Marketing							X

after tag-combination

Bestand:8m-groep1-experts.jpg

number of tags for each expert

This table is the most imporant data sheet of this research. The columns are

• # - describes the number of the arguments. This is a order of how the arguments occured. It has no natural semantic in itself - if we would have read the texts of the experts in a different order, the # would have been different. We use this to identify an argument.
• tag - is the tag, which combines the tags from Felix, John and Alex
• A./D. - is the metatag, which determines if this is an advantage for India, advantage for the Netherlands, etc.
• 1-7 - are the experts. Through the connection of rows and columns, one can see which expert stated which argument
• Alias - if an argument was named by more than one expert, the argument is listed only once. Yet, this column saves the other # of this argument.

The last row of this table is a sum row. It shows how much arguments are named by how many experts and how many arguments are named overall. An according diagram is available. Of course, the experts stated different amount of arguments:

1. the maximal amount of arguments stated by one expert: 12
2. the minimal amount of arguments stated by one expert: 1
3. the average amount of arguments stated by one expert: 5,85

Note that the sum of 41 counts an argument as often as it was stated. If you do lose sight of arguments stated twice (i. e. it does not matter hoew often an argument was stated), the sum would 30. Further beneath, we will see how this affects our counting of metatags.

#	tag	A./D.	I.	1	2	3	4	5	6	7	quantity	Alias
8	CMMI is a must	AI	0	X							1
25	CMMI is a must	AI	1				X				1
11	CMMI will not become a must	DD	0	X							1
24	CMMI is less important	DD	1				X				1
20	difficult to get	DD	1		X						1
23	E. tend to lead	DD	1				X	X	X	X	4	40, 36, 23, 31
15	easy to get	AI	1		X						1	19 (doubled from the same expert)
21	easy to get	AI	0			X					1
4	easy to implement	AI	1	X							1
10	established	AI	0	X							1
18	Europeans tend to lead	DD	1		X						1
37	high qualified employees	AI	1							X	1
2	High Quality	AI	1	X							1
17	Indians tend to follow	AI	1		X		X	X	X	X	5	35, 22, 30, 39
9	just a trend	AD	0	X							1
29	keep employees working	AI	1					X			1
3	Low cost	AI	1	X							1
16	misleading	Di	1		X						1
7	need for certification	Di	0	X							1
27	need for marketing	AI	1					X	X	X	3	42,32
13	need to demonstrate high quality	AI	1		X						1
28	no need for Marketing	DD	1					X			1
14	no need to demonstrate high quality	DD	1		X						1
33	no use for CMMI	DD	1						X	X	2	41
5	old habbits	DD	1	X							1
1	profileren	AI	1	X							1
26	proof of quality	AI	1				X				1
6	unclear benefits	DD	1	X							1
12	well promoted	AI	1	X							1
34	young industry	AI	1						X	X	2	38
				12	7	1	5	5	5	6	41

Occurences

of arguments, involving (dis)advantages and importancy

This table shows a part of the table above. We got rid of most of the data, to concentrate the attention to the occurence of the arguments. As one can see, most of the arguments are named only one time. Only five of the 30 arguments were named more than one.

Bestand:OM2 groep1 occurences2.jpg

occurence of the arguments

argument #	quantity
1	1
2	1
3	1
4	1
5	1
6	1
7	1
8	1
9	1
10	1
11	1
12	1
13	1
14	1
15	1
16	1
18	1
20	1
21	1
24	1
25	1
26	1
28	1
29	1
37	1
33	2
34	2
27	3
23	4
17	5

of arguments, involving (dis)advantages

This tables involves the (dis-)advantage column, but not the importancy. There are no changes to the table presented directly above. Yet, to be precise, we wrote this table down. We do not see the need to display the diagram, since it will be obviously the same as above.

#	tag	A./D.	1	2	3	4	5	6	7	quantity	Alias
1	profileren	AI	X							1
2	High Quality	AI	X							1
3	Low cost	AI	X							1
4	easy to implement	AI	X							1
5	old habbits	DD	X							1
6	unclear benefits	DD	X							1
7	need for certification	Di	X							1
9	just a trend	AD	X							1
10	established	AI	X							1
11	CMMI will not become a must	DD	X							1
12	well promoted	AI	X							1
13	need to demonstrate high quality	AI		X						1
14	no need to demonstrate high quality	DD		X						1
16	misleading	Di		X						1
18	Europeans tend to lead	DD		X						1
20	difficult to get	DD		X						1
24	CMMI is less important	DD				X				1
26	proof of quality	AI				X				1
28	no need for Marketing	DD					X			1
29	keep employees working	AI					X			1
37	high qualified employees	AI							X	1
8	CMMI is a must	AI	X			X				2	25
15	easy to get	AI		X	X					2	19 (doubled from the same expert), 21
33	no use for CMMI	DD						X	X	2	41
34	young industry	AI						X	X	2	38
27	need for marketing	AI					X	X	X	3	42,32
23	E. tend to lead	DD				X	X	X	X	4	40, 36, 23, 31
17	Indians tend to follow	AI		X		X	X	X	X	5	35, 22, 30, 39

of arguments, involving importancy

This tables shows the occurences of the arguments. This time, we lost sight of the importancy of the arguments. This table is structured as the "combined"-table, except for the missing D./A.-column.

Bestand:OM2 groep1 occurences1.jpg

occurence of the arguments

#	tag	I.	1	2	3	4	5	6	7	quantity
1	profileren	1	X							1
2	High Quality	1	X							1
3	Low cost	1	X							1
4	easy to implement	1	X							1
5	old habbits	1	X							1
6	unclear benefits	1	X							1
7	need for certification	0	X							1
8	CMMI is a must	0	X							1
9	just a trend	0	X							1
10	established	0	X							1
11	CMMI will not become a must	0	X							1
12	well promoted	1	X							1
13	need to demonstrate high quality	1		X						1
14	no need to demonstrate high quality	1		X						1
15	easy to get	1		X						1
16	misleading	1		X						1
18	Europeans tend to lead	1		X						1
20	difficult to get	1		X						1
21	easy to get	0			X					1
24	CMMI is less important	1				X				1
25	CMMI is a must	1				X				1
26	proof of quality	1				X				1
28	no need for Marketing	1					X			1
29	keep employees working	1					X			1
37	high qualified employees	1							X	1
33	no use for CMMI	1						X	X	2
34	young industry	1						X	X	2
27	need for marketing	1					X	X	X	3
23	E. tend to lead	1				X	X	X	X	4
17	Indians tend to follow	1		X		X	X	X	X	5

of metatags

Bestand:8M groep1 (dis)advantage and importancy.jpg

Occurence of metatags

This diagram shows how often the different kinds of arguments are being used:

• DD for disadvantages for dutch companies
• DI for disadvantages for indian companies
• AD for advantages for dutch companies
• AI for advantages for indian companies

Furthermore, this diagram shows how often a strong importancy and a weak importancy is being used. In total, there are 30 different arguments.

Conclusion

Our research question is:

• Indiase bedrijven zijn in staat volgens CMMI-5 te werken terwijl dat in heel veel Europese landen zoals Nederland niet wil vlotten. In hoeverre zijn de redenen van experts wat dat betreft met elkaar in overeenstemming?

We answered this question on different levels:

• we examined the arguments by themselves
• we examined metadata of the arguments, as there are
  • the importancy of the arguments (as defined above)
  • the (dis-)advantage for dutch and indian companies (as defined above)

If you read the answers from the experts (of course, in natural language with all its ambiguity), then you might get the impression, that they are state something similar. It seems as if there is a lot of agreement in their answers. This research did not prove this impression and showed a great variety in arguments.

We came to the conclusions, that there are some arguments common to many experts. According to this research, it is "common knowledge" that Indians tend to follow defined processes (argument 17) and that Europeans tend to think by themselves (argument 23). Note, that the exact formulation of this argument might vary from the formulation above. Those two arguments were stated most often.

The Top 7 arguments, without involving importancy are:

top 7	argument (tag)	times stated
7	CMMI is a must	2
6	easy to get	2
5	no use for CMMI	2
4	young industry	2
3	need for marketing	3
2	E. tend to lead	4
1	Indians tend to follow	5

However, most of the stated arguments are not shared: 25 arguments are stated only once.

The experts seem to agree that there are many advantages for India and many disadvantages for Europe / The Netherlands. This must not surprise, since our question could be interpreted suggestive in this point. However, most of the arguments are stated with a strong importancy. In total, there are 30 different arguments, 17 AI and 10 DD.

One can discuss about the combination of tags. It is difficult to meet the criteria for the combination of tags. The problem is that if one tags an argument, you loose always information. The combination after the "loss of information" is a tricky thing to do.