Explainable AI
Machine learning and artificial intelligence are very flexible and powerful methods for modelling and prediction, but they rarely provide any human understandable explanation or justification for their predictions.
This may have severe consequences if the models are dealing with people e.g. credibility scores or medication recommendations, but even when the consequence may seem less severe, there may be a reluctance to use machine learning methods if they cannot be interpreted and validated.
The SINTEF approach to explainable AI
At SINTEF we approach the challenge from several angles:
Hybrid AI combine domain knowledge (often physics or knowledge-graphs) and data driven methods to provide semi-explainable models with robust and trustworthy behaviour. This also includes development of AI methods for obtaining insight of physical systems.
Probabilistic AI where the model is aware of how certain or uncertain it is. Explanatory frameworks are developed and used to provide user-explanations of relevant variables in high-dimensional data.
End-user involvement in the model development for trust and relevant explanations.
Participatory design approaches considering the human-in-the-loop to improve both explainability of the models and end-users’ trust in the models.
Adoption of methodological approaches for integrating end-users’ values (e.g., value-sensitive design). These activities take place in workshops (in project work) with participants representing the different roles (from users, to developers, to engineers, etc.).