The availability of ever more computing power and data has led to an enormous development in methods related to data in the recent years. In this winter school, we will look at methods for using the knowledge in data to improve our computer algorithms and look at the interplay between these. The school will focus around the following topics and try to connect the dots between these interrelated fields:

• Data assimilation
• Inverse modeling and parameter estimation
• Uncertainty quantification
• Value of information
• Machine learning and artificial intelligence

The school will consist of lectures and interactive sessions and will include a poster session where participants can share their own work with each other. Examples will be given for several different application areas, and the techniques and ideas are applicable to a wide range of fields in eScience.

The fields can be summarized as follows. Data assimilation brings real-world knowledge, such as a measurement, into our computer models, and inverse modeling tries to estimate state which cause the observations we see. Uncertainty quantification tells us how good our models are, where they can be trusted, and perhaps more importantly where they are quite uncertain. To decrease the uncertainty of a model, we can e.g., take more measurements close to where our model is uncertain. This brings in the concept of value of information: what is the value of information fed into our computer models, and what information is most valuable? Finally, machine learning and artificial intelligence is at the extreme end of data-based algorithms which rely solely on the data we observe.

Data assimilation is the process of getting data into our models and making our computer models fit with what we observe. Weather prediction systems use this routinely by assimilating temperature and pressure to adjust the computer model of the atmosphere. These techniques include methods such as Ensemble Kalman filtering, 3DVAR, 4D-var, nonlinear particle filters, etc. Highly related to data assimilation is the process of inverse modeling and parameter estimation. Based on a set of measurements, we want to create a hypothesis for what the state is that caused this observation.

Uncertainty quantification tries to answer the question howuncertain our model is, and specifically when and where it is most uncertain. The Lorentz attractor, for example, will have high uncertainties: After a given amount of time, two points arbitrarily close to each other will have drifted arbitrarily far apart. However, for sufficiently short timespans, the two points will remain close and follow roughly the same trajectory. Determining how accurate a result is, how far into the future our models can predict, and where our models can be trusted is essential knowledge.

Value of information addresses the issue of uncertainties in a model, and how these can be reduced through gathering extra information. However, all information comes at a cost – either computationally, economically, or in other ways. A key question is then what data gives the most value for the least cost? At one end of the scale, it may not be very valuable to have many measurement points which do not reduce the uncertainties in your model. At the other end, a single measurement of the right kind can even remove the illposedness of your model and make the solution unique.

Machine learning and artificial intelligence is a prime example which has undergone a revolution in the last decade. These methods are so efficient that computers are faster and more accurate than humans when determining the breed of a dog on a photograph. At the same time, it is evident that these methods have their shortcomings. A machine learning algorithm can be trained to classify wolf and Alaskan husky (a dog breed) based on photographs. A set of pictures of wolves are fed to the computer to teach it what a wolf looks like, and similarly for the husky. This is called training data, and when confronted with a previously unseen photograph, the computer uses the knowledge it extracted from all the images in the training data to classify as either one or the other. However, we don't really know what knowledge the computer extracted from the initial training data. If the background is winter on all wolf images, and grass on Husky images, we may as well have trained it to classify "snow" versus "grass". (Original source: https://arxiv.org/abs/1602.04938)

Julien Brajard

Machine learning, and more specifically deep learning, has demonstrated impressive skill, over the last decade, over a varied range of tasks, such as playing game or recognize pictures. This incontestable proof of machine learning performance methods over these diverse tasks hints at their potential to improve the accuracy and/or speed of physical modelling, such as climate models. The objective of these lectures is to give a broad overview of machine learning techniques focused on the potential application to physical modelling. During the week, we will cover the following topics:
1) An overview of the most used machine learning techniques.
2) Presentation of some essential concepts for applying machine learning.
3) Details on few machine learning algorithms and deep-learning.
4) Illustrations of the use of machine learning to the physical modelling.

Useful References:
Goodfellow, Ian, et al. Deep learning. Vol. 1. Cambridge: MIT
press, 2016.
https://www.coursera.org/learn/machine-learning

Jo Eidsvik

We constantly use information to make decisions about utilizing and managing resources. For natural resources, there are often complex decision situations and variable interactions, involving spatial (or spatio-temporal) modeling. How can we quantitatively analyze and evaluate different information sources in this context? What is the value of data and how much data is enough?

The course covers multidisciplinary concepts required for conducting value of information analysis in multivariate and spatial models. Participants will gain an understanding for the integration of spatial statistical modeling and decision analysis for evaluating information gathering schemes. The value of information is computed before purchasing data, and can be useful for checking if data acquisition or processing is worth its price, or for comparing various experiments (active learning). The course will build a framework of perfect versus imperfect information, and total versus partial information where only a subset of the data is acquired or processed. The course uses slide presentations and runs hands-on projects on the computer (using R, Python or Matlab). Examples are used to demonstrate value of information analysis in various applications, including environmental sciences and petroleum. In these situations the decision maker could make better decisions by purchasing information (at a price) via for instance surveying, sensor usage or geophysical data.

Lecture 1: Decision analysis and the value of information
Lecture 2: Spatial statistics, design of experiments and value of information analysis
Lecture 3: Approximate computations for value of information analysis
(The course material uses 'Value of Information in the Earth Sciences', 2015, Eidsvik, Mukerji and Bhattacharjya, Cambridge Univ Press)

Hans-Rudolf Künsch

Ensemble data assimilation and particle filters

Data assimilation means sequential estimation and prediction of the state of a dynamical system, e.g. the earth's atmosphere or an animal population of some habitat, based on knowledge of the dynamics of the system and on partial and noisy observations as they become available. The term is used mainly in geophysical applications whereas in engineering the same problem is called filtering. In order to capture the uncertainty, prediction and estimation should be in the form of probability distributions. There is a general two-step recursion for the evolution of these distributions as new observations become available: In the propagation step, the filtering distribution which describes the knowledge about the current state is propagated forward according to the dynamics of the system to become the prediction distribution for the state at the next time step. In the update step, the information from the new observation is used to transform the prediction distribution into the next filtering distribution using Bayes theorem. Except in a few special cases, these recursions cannot be computed in closed form and one uses approximations by Monte Carlo samples instead. These samples are usually called ensembles and their members are called particles. The two main ensemble methods are the Ensemble Kalman Filter (EnKF) and the Particle Filter (PF). They share the same propagation step, but differ in the implementation of the update step.

In the first lecture, I will introduce the problem, the terminology and notation that I will use and then present the basic ideas and algorithms of both EnKF and PF.

The EnKF relies explicitly on linear measurements with Gaussian errors and implicitly also on a Gaussian prediction distribution. However, it performs surprisingly well in high dimensions. The PF on the other hand is in principle able to handle fully nonlinear dynamics and observations, but it collapses easily in high dimensions. A lot of effort has been made to improve the performance of the PF and to develop hybrid methods which combine features of the EnKF and the PF. In the second lecture, I will explain the collapse of the PF and discuss modifications of the PF and some of the hybrid methods.

In many applications the high dimension of the state arises from discretizing continuous functions in space. In this case, all methods need to be localized both for stability and efficient computation. Here, localization means that the update of the state at one location depends only on observations at locations nearby. In the third lecture I will explain the different variants for localizing the EnKF and then discuss the additional complications that arise for PF and hybrid methods. Finally, I will give an introduction to methods used to approximate the smoothing distribution, i.e. the distribution of the whole state evolution until the current time, and to methods for estimating unknown fixed parameters in the dynamics of the state or in the distribution of observation errors.

I plan to include a few tutorial exercies using the software R to allow participants to gain some intuitive understanding of how and when methods work in simple examples.

References:
P. Fearnhead, H. R. Künsch: Particle filters and data assimilation. Annual Review of Statistics and its Application 5 (2018), 421-449.
P. J. van Leeuwen, H. R. Künsch, L. Nerger, R. Potthast, S. Reich: Particle filters for applications in geosciences. Preprint, available at https://arxiv.org/abs/1807.10434

Kjetil Olsen Lye

In this session, we will cover uncertainty quantifications of parametrized partial differential equations (PDEs), where the parameters represent uncertainties in the system (for example initial position, heat conductivity, forcing terms, or geometry). Several problems coming from both application and theory can be formulated as a PDE of this form. We begin by surveying Monte Carlo methods for high-dimensional integrals of computationally expensive integrands, including Quasi-Monte Carlo and Multilevel Monte Carlo, and give a pragmatic overview of the theory. Time will be given to hands-on examples and exercises. In the end, we will apply the methods to hyperbolic conservation laws.

References / Software:
Please follow the installation instructions to install the required packages.

Hans R. Künsch is Professor Emeritus of Mathematics at ETH Zurich. He has been professor at ETH since 1983, and has been working with Spatial statistics and random fields, time series analysis, environmental modeling, and robust statistics and model selection. Ha has made significant contributions to these fields, and has 6500 citations  to his works on Google Scholar. 

• Topics: Particle filters and nonlinear data assimilation
• Home page

Jo Eidsvik is Professor at NTNU and his research interests are spatio-temporal statistics and computational statistics, often applied to the Earth sciences. He completed his PhD in statistics at NTNU in 2003 and has recently published his book entitled Value of Information in the Earth Sciences together with his co-authors Tapan Mukerji (Stanford) and Debarun Bhattacharjya (IBM Research). 

• Topics: Value of information
• Home page 

Remus G. Hanea is Professor II at the University of Stavanger and Specialist of Reservoir Technology at Equinor (previously Statoil). His research interests include Ensemble Kalman filtering, data assimilation, and inverse modeling. He completed his PhD in 2006 at TU Delft, and has been teaching the topic of data assimilation to students at PhD level for many years.

• Topics: Ensemble Kalman filtering and data assimilation
• Home page

 Julien Brajard is Assistant Professor at Sorbonne University and currently a visiting scientist at NERSC - Nansen Environmental and Remote Sensing Center. He completed his PhD at the University of Pierre et Marie Curie in 2006 and his research interests include remote sensing, inverse modelling, machine learning and data assimilation. He has been working in the intersection of these themes in the context of physical oceanography.

• Topics: The interplay between physical models, data assimilation, and machine learning
• Home page

Kjetil Olsen Lye is a research scientist at SINTEF and currently completing in his PhD thesis at ETH Zurich. His PhD work has centered around uncertainty quantification and the statistical solution of partial differential equations. He has also been running massively parallel simulations on supercomputers with the Alsvid-UQ software, utilizing over 1000 GPUs for his numerical simulations.  

• Topics: Multilevel Monte carlo sampling and uncertainty quantification in Python 
• Home page

André R. Brodtkorb is a research scientist at SINTEF and associate professor at Oslo Metropolitan university. He completed his PhD in scientific computing at the university of Oslo 2010, and his research interests are centered around applied mathematics and include numerical simulation, accelerated scientific computing, machine learning, and real-time scientific visualization.

• Topics: Multilevel Monte carlo sampling and uncertainty quantification in Python
• Home page

More to come soon. 

The 2019 winter school starts on the afternoon of Sunday January 20th and ends on Friday January 25th after lunch. The school consists of 10-14 double lectures , and the typical daily schedule consists of breakfast until 9:00, lectures until 10:30, and then a long break from 10:30 to 15:00 that is well spent socializing and skiing. The afternoon consists of two double lectures separated by a coffee break. Additionally, there is a poster session scheduled for Tuesday afternoon. Participants eat a common dinner each evening. The schedule is shown in the timetable below, but please note that changes may occur. 

 Presentations

• Brodtkorb: Welcome, Summary, UQ in Python
• Jo Eidsvik: Lecture 1, Lecture 2, Lecture 3
• Hans R. Künsch: Lecture notes
• Kjetil Olsen Lye: Lecture 1, Lecture 2, Lecture 3

Posters

• Håvard Heitlo Holm, GPU Ocean: Efficient simplified ocean models and non-linear data assimilation
• Amjed Mohammed, Turbulent and financial time series analysis
  
• Elisa Rebolini, Gustav Baardsen, Audun Skau Hansen, Karl Roald Leikanger, and Thomas Bondo Pedersen, Pair cutoff determination based on subsets of simulation data points
• Tobias Herb, Daniel Thilo Schroeder, EXA - A distributed computation
  environment
• Fabio Zeiser et al., Stress testing the Oslo Method for heavy nuclei
• Cécile HAUTECOEUR and François GLINEUR, Nonnegative matrix factorization with polynomial signals via hierarchical alternating least squares
• Jean Rabault, Miroslav Kuchta & Atle Jensen, Discovering Active Flow Control strategies using Neural Networks and Deep Reinforcement Learning
• Kaja Kvello, UTILIZING DATA IN HEALTHCARE
• Sebastian Mitusch and Simon W. Funke, An algorithmic differentiation tool (not only) for FEniCS
• Ralf Schulze-Riegert, Michael Nwakile, Sergey Skripkin, James Baffoe, Dirk Geissenhoener, Olympus Challenge – Data Driven Optimization for Field Development Plan and Competition from Domain Knowledge
• Lars Frogner, Helle Bakke and Boris Vilhelm Gudiksen, Simulating nanoflares
  in the solar atmosphere
• Yingguang Chu, Digital Twinsfor VesselLife CycleService (DigiTwinship)
• Xiaokang Zhang, Inge Jonassen, EFSIS: Ensemble Feature Selection Integrating Stability
• Nico Reissmann, Helge Bahmann, Jan Christian Meyer, Magnus Själander, Compiling with the Regionalized Value State Dependence Graph
• Alexander J Stasik, Espen Hagen, Jan-Eirik W Skaar, Torbjørn V Ness, Gaute T Einevoll, Estimating parameters of biological neuronal networks using Convolutional Neural Networks
• Snorri Ingvarsson and Movaffaq Kateb, Structural properties and static- and dynamic magnetization in soft ferromagnetic materials
• Aya Saad, Autonomous Imaging and Learning Ai Robot identifying planktON taxa in-situ

Applying for the winter school

There is no registration fee for the winter school, but participants must cover their own travel costs and hotel costs at Dr. Holms. You only have to register on the registration page. 

Cost of participating

There is no registration fee for the winter school, but participants must cover their own travel costs and hotel costs at Dr. Holms. 

Room allocation

The winter school has a limited number of rooms at Dr. Holms which will be reserved on a first come first served basis. We have in the previous years exceeded our room allocation, so please register as early as possible!

Posters

The winter school welcomes all posters to be presented. The aim of the poster session is to make new contacts and share your research, and it is an informal event. You need to indicate in your registration if you want to present a poster during the poster session. Please limit your poster to A0 in portrait orientation. 

Important information and links

• Registration page
• Registration deadline: See registration page