Design-Science Research

The research bible On research methods, by Järvinen Pertti; Opinpaja Oy 2004, is very resourceful to me in understanding the research process, selecting a research approach and applying design-science research.

There are four possible purpose of science, namely to describe, the explain, to predict and to control. However, we shall not restrict to them, because they do not cover the study off whether we can or cannot build a new artifact. The purpose of such a study can be understanding the system, its re-engineering and evaluation.

In How to select an appropriate research method in ergonomic studies?, Jarvinen provides a taxonomy of research methods.

My research ideas tends to study their utility as innovation by means of building and evaluation. In building a new innovation, utility aspects are striven and a particular development method is applied. In evaluation of the innovation, the realized final state is compared with the desired goal state, and maybe some criteria are used and some measurements performed. This process is called design science. The goal of design-science research is utility (while behavioral-science research aims at truth).

Research-science research is about answering the questions: Can we build a certain innovation and how useful is a particular innovation? The research questions contains verbs like: build, change, improve, enhance, maintain, extend, correct, adjust, introduce.

“The mission of design science is to develop knowledge for the design and realization of, i.e. to solve construction problem, or to be used in the improvement of the performance of existing entities, i.e. to solve improvement problems”, in other words, to implement some innovation.

The mission of design-science can also be seen as to develop design knowledge.

Design knowledge concerns “three designs: an object-design, the design of the intervention or of the artifact; a realization-design, i.e. the plan of the implementation of the intervention or for the actual building of the artifact; and a process-design, i.e. the professional’s own plan for the problem solving cycle, or, put differently, the method to be used to design the solution to the problem.

The design sciences are not too much interested in what is, but more in what can be.

The integral outcome of both construction and improvement is called a technological rule. Construction has four outcomes (construct, models, methods and instantiations) and two research approaches (build and evaluate).

A technological rule is defined as a chunk of general knowledge, linking an intervention or artifact with a desired outcome or performance in a certain field of application.

A major breakthrough rule occurred with the testing of the technological rule and then grounding on scientific knowledge.

Guidelines as requirements for effective design-science research are:

Design-Science Research guidelines from Alan R. Hevner, Salvatore T. March, Jinsoo Park, Sudha Ram: Design Science in Information Systems Research. MIS Quarterly 28(1): (2004)

The building process
Building is a process of constructing an artifact/innovation for a specific purpose. Early in the process extracting multiple case study is a good research practice to uncover technological rules already used in practice. In the developing multiple case study, the technological rules are developed and tested by researchers in close collaboration with the people in the field and often in the context of application. Following a reflective cycle,after each case, the researcher develops knowledge that can be transferred to similar context on the basis of reflection and cross-case analysis.

Design science products are of four types:
Constructs: or concepts from the vocabulary or language of a domain
Model: a set of propositions or statements expression relationships among constructs
Method: a set of steps (an algorithm or guidelines) used to perform a task
Instantiation: the realization of an artifact in its environment

The purpose of of the construction process is to achieve a movement from the initial state to the goal state (part of the specification process). The descriptive model of the initial state may need to capture the structure of reality in order to be a useful representation. The normative model of the goal state represents how things how to be.

A practitioner ought to describe the building process in detail, argue the selections and explain the decision. The originality of the solution and its superiority to the known solution must also be demonstrated.

The evaluation of the construction results
The research contribution lies on the novelty of the artifact and in the persuasiveness of the claims that is it effective. Evaluation is a process of determining how well the built artifact performs. It the research outcome is totally new, the actual performance evaluation is not required at this stage. When the old outcome exists, significant differences between the old construct, model, method or instantiation and the new one.

Metrics are needed in the evaluate activity to define what the research is trying to accomplish. Proposed universal metrics target all types of artifacs (i.e. constructs, models, methods and instantiation):

• Constructs: evaluation ten to involve completeness, simplicity, elegance understandability, and ease of use
• Models: evaluated in terms of their fidelity with real world phenomena, completeness, level of detail, robustness, and internal consistence. Model have to ignore things exactly because they view the world at a level of abstraction. The models can be represented in many way, physically, mathematically, pictorially etc
• Methods: operationally (the ability to perform the intended task or the ability of humans to effectively use the method if it is algorithmic), efficiency, generality and ease of use.
• Instantiations: efficiency and effectiveness of the artifact and its impacts on the environment and its users. This includes unplanned changes with positive and negative unanticipated outcomes that accompanied these changes.

Once metrics are developed, empirical work may be necessary to perform the evaluation. Constructs, models, methods and instantiations must be exercised within their environments. Often this means obtaining a subject group to do the exercising. Often multiple constructs, models, methods or instantiations are studies and compared.

Action Research
Action Research assumes that both building and evaluating sub-processes closely belong to the same process. It is the production of knowledge to guide practice, with the modification of a given reality occurring as part of the research process itself.

Relation to my thesis: I better understand how I should formulate my research questions and the process to answer them. Design-science research, in its building phase, has many similarities with engineering that I am familiar with (understanding of the initial state, requirements settings, definition of the goal state, instantiation, technological selection, iterative process, fast prototyping, … ). I am less aware of a scientific evaluation process.

As part of my investigation that explores the people perception of discrepancies in the context of collaboration supported by ubiquitous environments, my object-design should aims at improving the individual and group performance in and real-world uncertain ubicomp systems. First I will need to understand uncertainty from the literature and my own experiments (set the initial state), then build case studies (i.e. set the goal state to reach, different approaches and contexts), test field and evaluate them.