Abstract
Context
In systems of systems (SoSs), independent constituent systems operated by different organizations cooperate towards a common goal. This empirical paper is the first to explore what these organizations can learn from high-reliability organization (HRO) theory when collaborating on software performance testing, a part of software performance engineering (SPE). Our case study is based on the Norwegian tax report SoS.
Objective
Our overall research objective is to understand the organizational factors of SoS software performance testing. To understand these organizational factors, we also analyze the technical factors of SoS software performance.
Method
Three months before the opening of the tax return system, we observed meetings where three independent organizations coordinated their performance testing. We also attended conference meetings during, and retrospectives after, the opening. These observations were transcribed and deductively coded for the five HRO principles. Further, we analyzed measurement logs from the organizations participating in the tax return system.
Results
The technical root cause of a 90-minute outage in the tax report SoS was a load balancer primary memory shortage caused by too many unserved users. However, the organizational root cause is more interesting: incompatible worldviews reflecting an inadequate holistic SoS management.
Conclusions
HRO theory can benefit organizations preparing for workload peaks in mission-critical SoSs. Using the five HRO principles, the constituent organizations can develop a collective mind, bridging incompatible worldviews in SoS software performance testing.