Abstract
Distributed learning methods in the Industrial Internet of Things (IIoT) face challenges due to their dynamically changing learning environment. This is mainly caused by two facts. Firstly, the complex nature of statistically heterogeneous data, often non-independent and identically distributed (non-IID), from geographically scattered clients hampers the accuracy of the training model. Secondly, the heterogeneity in computational and communication resources among IIoT devices frequently introduces instability in the training process, resulting in a delay referred to as the straggler. Existing literature addresses only the unilateral side of the heterogeneity issue but lacks comprehensive efforts to tackle the joint problem of device and data heterogeneities for the resource-constrained IIoT. To address these multifaceted challenges, in this article, we have introduced device and data Heterogeneity Aware SplitFed Learning, namely Het-SFL, a distributed learning framework designed for deployment within dynamic IIoT networks. The developed Het-SFL framework optimizes resource utilization and reduces the computational burden by dynamically assessing the optimal split point of the training model for the resource-limited devices, considering factors such as training time, device energy consumption, and model accuracy. A clustering mechanism is employed to mitigate the straggler effect and to reduce the solution space to obtain the optimal split point within a significantly shortened deadline. Furthermore, the Het-SFL framework leverages emerging Digital Twin (DT) technology to facilitate real-time analysis of heterogeneous data, thereby improving the performance of the training model in non-IID contexts. The numeric performance analysis reveals that the Het-SFL framework improves training performance in terms of accuracy, training time, and device energy consumption by up to 30%, 40%, and 55%, respectively, compared to other state-of-the-art works.