To main content

Research synthesis in software engineering: A tertiary study

Abstract

Context
Comparing and contrasting evidence from multiple studies is necessary to build knowledge and reach conclusions about the empirical support for a phenomenon. Therefore, research synthesis is at the center of the scientific enterprise in the software engineering discipline.

Objective
The objective of this article is to contribute to a better understanding of the challenges in synthesizing software engineering research and their implications for the progress of research and practice.

Method
A tertiary study of journal articles and full proceedings papers from the inception of evidence-based software engineering was performed to assess the types and methods of research synthesis in systematic reviews in software engineering.

Results
As many as half of the 49 reviews included in the study did not contain any synthesis. Of the studies that did contain synthesis, two thirds performed a narrative or a thematic synthesis. Only a few studies adequately demonstrated a robust, academic approach to research synthesis.

Conclusion
We concluded that, despite the focus on systematic reviews, there is limited attention paid to research synthesis in software engineering. This trend needs to change and a repertoire of synthesis methods needs to be an integral part of systematic reviews to increase their significance and utility for research and practice.

Category

Academic literature review

Language

English

Author(s)

  • Daniela Soares Cruzes
  • Tore Dybå

Affiliation

  • Norwegian University of Science and Technology
  • SINTEF Digital / Software Engineering, Safety and Security

Year

2011

Published in

Information and Software Technology

ISSN

0950-5849

Volume

53

Issue

5

Page(s)

440 - 455

View this publication at Cristin