Abstract
When considering complex and wide-ranging adaptations of a middleware architecture, there are often several feasible design alternatives. Reasoning in run-time for selecting among such big adaptations would not scale, although implementation of the eventually chosen alternative could be automated. There is therefore a need for early decision support that (in terms of the time needed) can enlighten the effects of the implementation of the various design alternatives. The goal is to integrate the early decision support into the middleware adaptation feedback loop. We propose a decision support method for trade-off analysis considering three aspects: cost, risk, and quality. We have tried out the decision making method in a case study and evaluated the performance of the method with respect to a set of pre-defined success criteria. Apart from the case study itself, the evaluation involved a thought experiment, observations made during the analysis, as well as written feedback. The results of the evaluation indicate practical feasibility of the proposed method. In addition, the evaluation has provided useful insights into strengths and weaknesses of the method and suggested directions for future research and improvements. This paper briefly presents the decision making method, and focuses on reporting the evaluation results.