In the case study in chapter 5 I designed the study, selected the theoretical framework used (Architecture business cycle), chose the architecture scenarios and made the purposeful sample of interviewees. My co-author contributed with choice of the interpretative research approach and the design of the interview process. The interview questions, the interviews and the transcriptions were performed by 3 students as part of their thesis project, which I supervised together with the co-author. The final analysis in the paper was mainly done by me with input from the co-author based on the material of the students.
In the comparative case study in chapter 6 the co-author designed the interview study. We jointly performed the interviews, collected the data, made the analysis and wrote the original conference paper I made most of the extensions of that paper to the present journal article with input from the co-author.
I performed the mapping study in chapter 7: Defined the search criteria, identified the relevant cases and extracted relevant categories of development approaches and architectures from the chosen papers. Based on the results from the mapping study I defined the initial model of 5 approaches which was iterated with the co-author to the present form.
The model of the context of individual teams in chapter 8 was developed jointly with the co-author. The last four case studies were preformed by by me. The measures were developed in cooperation with engineers at Volvo cars.
The architecture in chapter 9 was developed by me with input from the co-author. The case study was performed by me, but the actual implementation in the case was done by an external team. The evaluation
and comparison of the existing platforms was done by me with input from the co-author.
The five key activities in chapter 10 were elaborated by me based on the theoretical foundation of the co-author. The case study was performed by me, but the actual implementation in the case was done by an external team.
The architectures and the infrastructure in chapter 11 were developed by me with input from the co-author, and was based on a theoretical foundation of the co-author. The case study was performed by me, but the
actual implementation in the case was done by an external team.
1 January 2013
31 December 2012
Thesis chapter 4: Structure of the thesis
Chapter 1 gives a general introduction to mass-produced embedded systems and what characterises them.
Chapter 2 gives some examples of problems in current R&D approaches, defines goals for an R&D organisation, and defines the research questions investigated further in the thesis.
Chapter 3 outlines the research process, used epistemology, lists the cases studied and discusses issues with doing research as native in the organisation studied.
Chapter 4 outlines all chapters in the thesis and describes my individual contribution in chapters 5 to 11.
Chapters 5 and 6 describe three detailed cases of how architectures are developed and maintained for automotive embedded systems. The rich description provides a background of current practice of software development of mass-produced embedded systems.
Chapter 7 identifies and models various approaches to develop embedded software, and relates them to two different businesses. The model was based on a mapping study to identify what broad approaches to embedded software development are used in industry. The study aimed to better understand, and potentially point at an answer to RQ1. Of special interest was how these approaches relate to used architectural styles, which supports further investigation of RQ1.2.
Chapter 8 aims to facilitate agile development for individual teams in the context of large MPES projects. This would allow the length of iterations from idea to implementation to be determined by the potential speed of the individual development teams and not the overall product development project. This provides an answer to RQ1.1.
Chapter 9 designs a compositional architecture for embedded software as precursor for open software ecosystems. The study aimed to answer RQ1.2 and detail the architecture-related issues of RQ1.3.
Chapter 10 explores open software ecosystems as possible approach to development of embedded software as an asnwer to RQ1.3. This would extend the base of innovators beyond what is presently hierarchically directed by the OEM to support delivery of innovative features with value for the customer.
Chapter 11 describes innovation experiment systems for embedded products and defines a reference architecture supporting such an R&D approach. The study aimed to an answer RQ1.2 and RQ1.4.
Chapter 12 summarises the answers to the research questions and design goals in terms of developed artefacts. It also lists the key contributions of the thesis.
Chapter 2 gives some examples of problems in current R&D approaches, defines goals for an R&D organisation, and defines the research questions investigated further in the thesis.
Chapter 3 outlines the research process, used epistemology, lists the cases studied and discusses issues with doing research as native in the organisation studied.
Chapter 4 outlines all chapters in the thesis and describes my individual contribution in chapters 5 to 11.
Chapters 5 and 6 describe three detailed cases of how architectures are developed and maintained for automotive embedded systems. The rich description provides a background of current practice of software development of mass-produced embedded systems.
Chapter 7 identifies and models various approaches to develop embedded software, and relates them to two different businesses. The model was based on a mapping study to identify what broad approaches to embedded software development are used in industry. The study aimed to better understand, and potentially point at an answer to RQ1. Of special interest was how these approaches relate to used architectural styles, which supports further investigation of RQ1.2.
Chapter 8 aims to facilitate agile development for individual teams in the context of large MPES projects. This would allow the length of iterations from idea to implementation to be determined by the potential speed of the individual development teams and not the overall product development project. This provides an answer to RQ1.1.
Chapter 9 designs a compositional architecture for embedded software as precursor for open software ecosystems. The study aimed to answer RQ1.2 and detail the architecture-related issues of RQ1.3.
Chapter 10 explores open software ecosystems as possible approach to development of embedded software as an asnwer to RQ1.3. This would extend the base of innovators beyond what is presently hierarchically directed by the OEM to support delivery of innovative features with value for the customer.
Chapter 11 describes innovation experiment systems for embedded products and defines a reference architecture supporting such an R&D approach. The study aimed to an answer RQ1.2 and RQ1.4.
Chapter 12 summarises the answers to the research questions and design goals in terms of developed artefacts. It also lists the key contributions of the thesis.
30 December 2012
Thesis chapter 3.3: Insider issues
I was a PhD student in Software Engineering while being employed by large company, Volvo Car Corporation, where I have worked for 9 years prior to the start of the PhD education. The company has a program for about 25 PhD students in various areas. The agreement between the company and the university states that 80% of my full-time employment should be spent towards obtaining a PhD while 20\% of my time still working in industrial projects.
There is also an expectation locally that the research should be relevant for the EESE department where I am employed. Synergy is expected between the industrial project and the conducted research, and at best the research results should be directly beneficial for the company.
Industrial PhD students are often expected to have dual roles and act as neutral or unbiased researchers by the scientific community, and act as practitioners in the area they are researching by the company where they are employed. Furthermore they have a personal goal of pursuing a research education. These factors drives both the focus of the research (e.g. the formulation of research questions) and affects how the research is conducted. It is not impossible to conduct research under these conditions but there are some concerns which are not relevant for a researcher employed by a university or similar. Börjesson is the only other industrial PhD student I have found mentioning the dual roles in her thesis.
The dual roles have been called different things by different authors; ``participant-observer'', ``insider'' (link, link and link) and ``native'', with the last article focuses on ``how complete members may undertake academic research in their own organizations while retaining the choice of remaining a member within a desired career path when the research is completed'', i.e. not going native for the purpose of research but being native before, during and after. But they all consider the role of a researcher being part of the group that is being researched, with the ``insider'' term mostly used in the context of action research. The insider issues relevant for this thesis are mostly related to qualitative research in general and case study research in particular.
I address the issues of the dual roles in this thesis based on four papers (link, link, link, and link). These papers describe some issues which are particular to a industrial PhD student which are not that relevant to a researcher employed by a university or similar institution.
The only mentioning of ethics I found specifically for software engineering research is Runeson & Höst , a general overview of science and ethics can be found by Rosenthal. Both of these focus on the ethics towards any persons being studied and not the ethical issue of having questionable motives for performing the research. In this research project the motives from the funding agencies, Volvo Car Corporation and VINNOVA, are clearly stated and it is difficult to imagine any ulterior motives beyond satisfying these goals.
All case studies in table follows the spirit of the Ethical Considerations by Runeson & Höst even if the exact implementation may differ to their examples (Case I was even conducted before the publication of this article).
In all case studies in table the participants were informed about my dual roles as participant and observer, and that my participation would lead to published research, such as this thesis, and to internal reports in respective company. The latter was possibly more sensitive, and perhaps restrained the answers since the interviewees' managers would be in the audience. However I feel that this only had a minor influence. This pre-information also included a brief overview of the research process.
Unfortunately the consent to participate was not written, just verbal (although recorded in some interviews when the equipment was working as planned), contrary to the guidelines by Runeson & Höst, and the pre-information was also only verbal.
The participants were anonymised as much as possible, but in those cases where I deemed it would be possible for another insider to determine who participated in the study they were informed of this prior to their participation.
In cases II, III and V the participants had the opportunity to review the transcribed data as it was captured before it was used or reported outside of the group participating in the study. One interviewee in case V declined to have the interview recorded, but note-taking was ok, which was obliged by me.
It may be more difficult for someone to decline to participate in a case study in a industrial company than in a general social setting, due to the fact that the observer may have management buy-in to perform the study. However my general impression is contrary, all interview participants saw the interviews as an opportunity for retrospection.
Volvo Cars has a process of reviewing articles by industrial PhD students before publishing. My experience is that this process becomes less strict the more experienced the student becomes in writing about sensitive company information.
Tellis continues with ``the technique (of participant-observation) provides some unusual opportunities for collecting data, but could face some major problems as well. The researcher could well alter the course of events as part of the group, which may not be helpful to the study''. This is in essence what Yin says where he mentions four major problems for a participating observer:
The first problem is that a participant sometimes must assume roles that could be contrary to good scientific practice. For me as a participant in an engineering company this seem to be a small risk since there is a common view that good engineering is based on science. Therefore the claim that I am doing research is usually a valid rationale for a certain role or behaviour.
The second is that the observer may become a supporter of the unit or group studied. If this is a real concern for an industrial PhD student then there is a real problem with such a role doing research since it is expected of the PhD student to be a supporter of his or her company. I also don't think the rest of the scientific community would expect otherwise if it clearly stated when presenting the research as being made by an insider.
Third, the balance between participating and observing may be skewed. This is a real concern for both me and other PhD colleagues I have spoken to at my company.
I have not found any recipes to mitigate this risk, and have tried to judiciously balance this during the project.
The fourth problem Yin mentions is if the studied organisation is ``physically dispersed'', then it may be ``difficult to be at the right pace at the right time; either to participate in or to observe important events''. The last problem is not unique to researchers with dual roles, but is relevant for anybody doing observations. In such a large organisation as Volvo Cars it is impossible to always be ``at the right place at the right time'', but at least all in-house development is done at a single site, Torslanda, Göteborg, Sweden.
Brannick & Coghlan have a more multifaceted view of doing research as a native. They for example state that native researchers have automatic primary access, while secondary access to documents or people may be harder for a native researcher since the organisation may not distinguish between the roles of a researcher and a practitioner, with the practitioner having free access to certain information while still being restricted to other information on a need-to-know principle. I have not experienced this as a problem at all at Volvo Cars.
Brannick & Coghlan continue to discuss the preunderstanding that a native researcher may put him ``too close'' to the data, but concludes that the research challenges "do not invalidate any of the outlined research traditions" (Their article mentions positivism, interpretative/hermeneutic and critical realism/action research). I assume that ``too close'' means not being able to evaluate the data objectively, which may be risk also in this thesis.
There is also an expectation locally that the research should be relevant for the EESE department where I am employed. Synergy is expected between the industrial project and the conducted research, and at best the research results should be directly beneficial for the company.
Industrial PhD students are often expected to have dual roles and act as neutral or unbiased researchers by the scientific community, and act as practitioners in the area they are researching by the company where they are employed. Furthermore they have a personal goal of pursuing a research education. These factors drives both the focus of the research (e.g. the formulation of research questions) and affects how the research is conducted. It is not impossible to conduct research under these conditions but there are some concerns which are not relevant for a researcher employed by a university or similar. Börjesson is the only other industrial PhD student I have found mentioning the dual roles in her thesis.
The dual roles have been called different things by different authors; ``participant-observer'', ``insider'' (link, link and link) and ``native'', with the last article focuses on ``how complete members may undertake academic research in their own organizations while retaining the choice of remaining a member within a desired career path when the research is completed'', i.e. not going native for the purpose of research but being native before, during and after. But they all consider the role of a researcher being part of the group that is being researched, with the ``insider'' term mostly used in the context of action research. The insider issues relevant for this thesis are mostly related to qualitative research in general and case study research in particular.
I address the issues of the dual roles in this thesis based on four papers (link, link, link, and link). These papers describe some issues which are particular to a industrial PhD student which are not that relevant to a researcher employed by a university or similar institution.
Ethics
A researcher funded by the industry may be questioned if there are some ulterior motives to performing the research or of the funding agency have any influence over the results, or what results are published or withheld. Extreme cases could be tobacco industry funding research about the dangers of smoking or oil industries funding research about climate change due to use of fossilised fuels.The only mentioning of ethics I found specifically for software engineering research is Runeson & Höst , a general overview of science and ethics can be found by Rosenthal. Both of these focus on the ethics towards any persons being studied and not the ethical issue of having questionable motives for performing the research. In this research project the motives from the funding agencies, Volvo Car Corporation and VINNOVA, are clearly stated and it is difficult to imagine any ulterior motives beyond satisfying these goals.
All case studies in table follows the spirit of the Ethical Considerations by Runeson & Höst even if the exact implementation may differ to their examples (Case I was even conducted before the publication of this article).
In all case studies in table the participants were informed about my dual roles as participant and observer, and that my participation would lead to published research, such as this thesis, and to internal reports in respective company. The latter was possibly more sensitive, and perhaps restrained the answers since the interviewees' managers would be in the audience. However I feel that this only had a minor influence. This pre-information also included a brief overview of the research process.
Unfortunately the consent to participate was not written, just verbal (although recorded in some interviews when the equipment was working as planned), contrary to the guidelines by Runeson & Höst, and the pre-information was also only verbal.
The participants were anonymised as much as possible, but in those cases where I deemed it would be possible for another insider to determine who participated in the study they were informed of this prior to their participation.
In cases II, III and V the participants had the opportunity to review the transcribed data as it was captured before it was used or reported outside of the group participating in the study. One interviewee in case V declined to have the interview recorded, but note-taking was ok, which was obliged by me.
It may be more difficult for someone to decline to participate in a case study in a industrial company than in a general social setting, due to the fact that the observer may have management buy-in to perform the study. However my general impression is contrary, all interview participants saw the interviews as an opportunity for retrospection.
Volvo Cars has a process of reviewing articles by industrial PhD students before publishing. My experience is that this process becomes less strict the more experienced the student becomes in writing about sensitive company information.
Observer versus participant
Tellis defines this issue as ``participant-observation makes the researcher into an active participant in the events being studied''. This is not unique for the industrial PhD student, but it is almost unavoidable. At Volvo cars it is expected since the PhD program targets to ``reinforce a scientific approach within product development'', and PhD students are expected to participate in the activities of the company.Tellis continues with ``the technique (of participant-observation) provides some unusual opportunities for collecting data, but could face some major problems as well. The researcher could well alter the course of events as part of the group, which may not be helpful to the study''. This is in essence what Yin says where he mentions four major problems for a participating observer:
The first problem is that a participant sometimes must assume roles that could be contrary to good scientific practice. For me as a participant in an engineering company this seem to be a small risk since there is a common view that good engineering is based on science. Therefore the claim that I am doing research is usually a valid rationale for a certain role or behaviour.
The second is that the observer may become a supporter of the unit or group studied. If this is a real concern for an industrial PhD student then there is a real problem with such a role doing research since it is expected of the PhD student to be a supporter of his or her company. I also don't think the rest of the scientific community would expect otherwise if it clearly stated when presenting the research as being made by an insider.
Third, the balance between participating and observing may be skewed. This is a real concern for both me and other PhD colleagues I have spoken to at my company.
I have not found any recipes to mitigate this risk, and have tried to judiciously balance this during the project.
The fourth problem Yin mentions is if the studied organisation is ``physically dispersed'', then it may be ``difficult to be at the right pace at the right time; either to participate in or to observe important events''. The last problem is not unique to researchers with dual roles, but is relevant for anybody doing observations. In such a large organisation as Volvo Cars it is impossible to always be ``at the right place at the right time'', but at least all in-house development is done at a single site, Torslanda, Göteborg, Sweden.
Brannick & Coghlan have a more multifaceted view of doing research as a native. They for example state that native researchers have automatic primary access, while secondary access to documents or people may be harder for a native researcher since the organisation may not distinguish between the roles of a researcher and a practitioner, with the practitioner having free access to certain information while still being restricted to other information on a need-to-know principle. I have not experienced this as a problem at all at Volvo Cars.
Brannick & Coghlan continue to discuss the preunderstanding that a native researcher may put him ``too close'' to the data, but concludes that the research challenges "do not invalidate any of the outlined research traditions" (Their article mentions positivism, interpretative/hermeneutic and critical realism/action research). I assume that ``too close'' means not being able to evaluate the data objectively, which may be risk also in this thesis.
Benefits
There are a number of benefits of being a industrial PhD student as well. Without much elaboration some benefits I have experienced are:- Access - The industrial PhD student is by almost default an ``insider'', especially if he is already is working at the company before starting his studies, so primary access is granted by default.
- Interpretation - The industrial PhD student has a superior possibility to use an interpretative approach to qualitative research.
- Acceptance - Research results may have a better chance of being seen as ``true'' since the results are coming from one of their own. This is not the same as researcher suggestions are more acceptable to implement, but the argument that the researcher does not ``understand'' issues specific to the organisation is absent.
29 December 2012
Thesis chapter 3.2: Qualitative research
Most of the reasoning and analysis in this project is qualitative, based on industrial observations and experiences rather than elaborate theories.
During the course of the project I, together with colleagues, studied 8 cases* of industrial software and architecture development, seen in the table below. The data gathered in these case studies are almost exclusively qualitative empirical data from industry and thus my scientific conclusions are also of a qualitative nature.
Qualitative research can be performed in many traditions, Walsham mentions three types of epistomologies: Positivism, Interpretative (or Non-positivism) and Normativism. This thesis uses the interpretative approach when collecting and analysing data for three reasons: First, I prioritised insight into the cases over reliability for other researchers to reproduce the studies. Second, I judged it impossible to disregard any pre-conceptions I had based on more than ten years of of professional experience of designing and developing software-intensive embedded systems. Third, most of the descriptions of the case studies are based on interview data which already are the interviewees' interpretation of the actual events/facts/situations/...
The selection of interviewees were in cases I, II, III and V done as purposive samples in order to cover a comprehensive variety of roles and teams at the studied organisations.
* A case study being ``an empirical study that investigates a contemporary phenomena in depth and within its real-life context, especially when the boundaries between the phenomenon and the context are not clearly evident''
Qualitative research can be performed in many traditions, Walsham mentions three types of epistomologies: Positivism, Interpretative (or Non-positivism) and Normativism. This thesis uses the interpretative approach when collecting and analysing data for three reasons: First, I prioritised insight into the cases over reliability for other researchers to reproduce the studies. Second, I judged it impossible to disregard any pre-conceptions I had based on more than ten years of of professional experience of designing and developing software-intensive embedded systems. Third, most of the descriptions of the case studies are based on interview data which already are the interviewees' interpretation of the actual events/facts/situations/...
| Case | Company | Description | Data sources | Author role | Chapter(s) |
| I | VCC | Forces shaping the distributed in-vehicle EE architecture in P2 platform | Interviews with 20 developers + design documents | Insider observer | 5, 7, 10 |
| II | VCC | Comparative case study on architecting process on existing platforms | Interviews with 6 architects + design documents + personal notes | Insider observer | 6, 7, 10 |
| III | Scania | Comparative case study on architecting process on existing platforms | Interviews with 5 architects + design documents + personal notes | Observer | 6 |
| IV | VCC | Case study on architecture decisions for new platform | Observed decisions during 3 months | Participant / insider observer | |
| V | VCC | Post mortem analysis of Sensus infotainment development | Interviews with 6 key developers + design documents | Insider observer | 7, 10 |
| VI | VCC | Proof-of-concept development of android-based infotainment system | All project & design documents of both product owner and Scrum team + personal notes | Participant / insider observer | 9, 10, 11 |
| VII | VCC | Agile development of climate control software | Official meeting notes and presentations + personal notes | Insider observer | 7, 10 |
| VIII | VCC | Agile development of next generation infotainment system | Official meeting notes and presentations + personal notes | Insider observer | 10 |
The selection of interviewees were in cases I, II, III and V done as purposive samples in order to cover a comprehensive variety of roles and teams at the studied organisations.
* A case study being ``an empirical study that investigates a contemporary phenomena in depth and within its real-life context, especially when the boundaries between the phenomenon and the context are not clearly evident''
28 December 2012
Thesis chapter 3.1: Design research process
Blessing & Chakrabarti defines a design research process called the Design Research Methodology (DRM). The process emphasise the focus and goals of the research project, the study of factors contributing to the goals, the development of artefacts to support the goals, and the evaluation the developed artefacts (link).
The design process was scoped by the four stages of the DRM, seen in figure below, although the actual artefacts were developed in a nonlinear fashion. All four stages contributed to new learning, which is included in the respective chapters of the thesis.
This resulted in a set of artefacts to achieve the goals and alleviate problems identified in the previous two stages. The artefacts fall into several categories of ways-of-working, model and architectures:
The DRM process assumes an iteration of the last two phases to optimise the design of the artefacts against the design goals, in this project only one cycle was performed.
The set of artefacts is not sufficient for an organisation to evolve its R&D process, in practice there are numerous other factors affecting the efficiency of an R&D process at an OEM. Factors from almost every area of research, ranging from management theory via economics to semiconductor technology. The set of artefacts developed is thus shown to mitigate the investigated problems, but not necessarily solve them, at least not on their own.
The design process was scoped by the four stages of the DRM, seen in figure below, although the actual artefacts were developed in a nonlinear fashion. All four stages contributed to new learning, which is included in the respective chapters of the thesis.
 |
| The four stages of the Design Research Methodology by Blessing and Chakrabarti |
Criteria
The first stage in the design process defined the overall context, research questions and design goals based on cases~I-V in together with selected problems form published literature, described in detail in chapter 2. Chapters 5 and 6 provides a rich insight into the industrial context of mass-produced embedded systems.Descriptive Study I
The second stage investigated existing literature to further understand the problems, elaborated on evaluation criteria and guided to possible solutions, this is described in chapter 7. The aim of this investigation is to answer RQ1 and guide towards answers of answer RQ1.1 and RQ1.2.Prescriptive Study
The third stage, the Prescriptive Study, investigated specific artefacts related to RQ1.1, RQ1.2, RQ1.3 and RQ1.4.This resulted in a set of artefacts to achieve the goals and alleviate problems identified in the previous two stages. The artefacts fall into several categories of ways-of-working, model and architectures:
- Chapter 7 describes a model of how individual teams interact with large projects, and based on this a method to facilitate a transition to agile development for individual teams.
- Chapters 9 and 11 describe two architectures, for compositional architectures and embedded architecture for innovation experiment systems respectively.
- Chapter 10 describes 5 activities to establish an open software ecosystem for MPES.
Descriptive Study II
The fourth stage evaluated the resulting artefacts, methods and ways-of-working in cases VI-VIII from the previous stage (also the focus of chapters 7, 9, 10 and 11).Limitations
The artefacts were evaluated mainly through observational, analytical and descriptive methods as categorised by Hevner et al:- Observation through case studies
- Architectural analysis of designed artefacts
- Informed argument based on insider knowledge to show the artefact's utility
- Scenarios describing the use of the artefact in the context
The DRM process assumes an iteration of the last two phases to optimise the design of the artefacts against the design goals, in this project only one cycle was performed.
The set of artefacts is not sufficient for an organisation to evolve its R&D process, in practice there are numerous other factors affecting the efficiency of an R&D process at an OEM. Factors from almost every area of research, ranging from management theory via economics to semiconductor technology. The set of artefacts developed is thus shown to mitigate the investigated problems, but not necessarily solve them, at least not on their own.
27 December 2012
Thesis chapter 3: Research design and methodology
There are several research methods used when academia interacts with an industrial setting; experiments, design science or research, action research or participative case studies .
This thesis is in essence a design project; developing artefacts, processes and methods to create opportunities for business for embedded software in mass-produced systems in general, and automotive software in particular. Therefore design research was chosen as the research methodology.
This thesis is in essence a design project; developing artefacts, processes and methods to create opportunities for business for embedded software in mass-produced systems in general, and automotive software in particular. Therefore design research was chosen as the research methodology.
Subscribe to:
Posts (Atom)