Supervisors: Thomas Christie & Colin Doumont 

Now that bo-arena (and bencher) provide the infrastructure for easily running standard BO experiments, it would be great to extend these libraries to also provide an accurate picture of which methods are state-of-the-art for different flavours of BO. As a starting point we will consider the setting of BO with unknown constraints. The project will lead to open-source contributions to bo-arena and bencher, deliver insights into what drives performance in constrained BO, and question whether more complex features added to the BO loop in recent years (such as trust regions) are really necessary.

Supervisor: Thomas Christie

Recent work has shown that, for a data distribution generated by a Markov chain, the quantities required for generating samples from the corresponding masked discrete diffusion process are available exactly, cheaply, and in closed-form via the forward-backward algorithm, and forward-filtering backward-sampling. Given that we know what an optimal algorithm looks like, it would be interesting to train a transformer on this task and try to reverse-engineer what it has learned, drawing on techniques from the field of mechanistic interpretability. We can also monitor the training dynamics of the model - to what extent does the model just memorise the examples it has seen during training, and how does its ability to generate novel (and valid) samples improve over the course of training? Does the Grokking phenomenon occur and, if so, can we tie it to learning the forward-backward algorithm via the mechanistic interpretability aspect of this project?

Prerequisites:

▶ Proficiency with PyTorch or JAX

▶ Strong maths skills.

▶ The ability to work independently.

Supervisor: Paul Fischer 

Estimating the parameters of a differential equation from noisy measurements of its solution is an inverse problem, relevant to many real world applications. The literature offers a long list of methods for it, each with its own trade-offs between robustness, cost, and statistical interpretation. Probabilistic ODE filters, which solve a differential equation as a Bayesian inference problem, offer a single framework in which model, data, and unknown parameters are treated jointly. This project studies how established parameter-inference methods relate to each other when viewed through this framework, and how the framework can be used to design inference schemes that are more robust and scale to larger problems. Depending on interest, the work can lean towards theory, towards efficient implementation, or towards applying the resulting methods to challenging inverse problems from science and engineering. You should be interested in Bayesian inference, numerical methods for differential equations, and the connections between them.

Supervisors: Paul Fischer & Nathaël da Costa

Probabilistic ODE filters return not just a numerical solution of a differential equation but a posterior distribution over it, and the usefulness of that posterior depends on whether its uncertainty is calibrated: whether the reported error bars actually match the error being made. This project investigates calibration from a theoretical angle, asking how fast the estimated uncertainty converges to the true error as the discretization is refined, whether the existing guarantees can be sharpened, and how the picture changes when the filter is combined with measurements or with unknown external inputs. The tools come from Gaussian-process regression, Bayesian filtering, and the statistical theory of hyperparameter estimation. You should enjoy mathematical analysis and be interested in the statistical foundations of numerical methods. A background in mathematics, in particular probability and some functional or numerical analysis is expected.

Supervisor: Paul Fischer 

Battery management systems estimate the state of charge and state of health of lithium-ion cells by fusing a physical cell model with voltage and current measurements, usually through a Kalman filter. These estimates are only as good as the information the measurements carry, and standard filters give little indication of when that information runs out. This project explores probabilistic ODE filters, which treat the cell model and the data as joint observations of a Gaussian-process prior over the trajectory, as a framework for SOC and SOH estimation with calibrated uncertainty. Possible directions include quantifying when the battery state and its slowly drifting parameters are identifiable, comparing the calibration of the resulting posteriors against established filters, and handling unknown or faulty inputs. The work combines Bayesian filtering, Gaussian processes, and identifiability analysis on public battery datasets. You should be interested in probabilistic inference for dynamical systems and in seeing it work on a real engineering problem.

Supervisor: Tim Weiland

Probabilistic numerical methods represent uncertainty arising from the numerical approximation of differential equations, but shock waves challenge their commonly used smoothness assumptions. This thesis investigates how shocks can be represented and treated within probabilistic solvers for hyperbolic conservation laws. The main task is to develop and evaluate a suitable approach on one-dimensional benchmark problems, exploring modifications to the prior or the governing equations, such as information geometric regularization. The expected outcome is a prototype method and an assessment of its accuracy, conservation properties, and representation of numerical uncertainty near shocks. Prerequisites: Some background in numerical analysis and probability, together with programming experience, is expected. Familiarity with partial differential equations or Gaussian processes is very helpful.

Supervisors: Tim Weiland & Tobias Weber

Neural processes learn to perform inference over functions, but their performance can depend strongly on how information is represented and propagated through the architecture. This thesis investigates whether classical insights from Gaussian process approximation and numerical PDE solvers carry over to neural processes: global or spectral methods are often effective for smooth functions, while rough Matérn processes exhibit local structure that can be exploited by sparse methods such as Vecchia approximations or Gaussian Markov random fields. The main task is to systematically compare global, spectral, and graph-based neural process architectures on Gaussian process regression problems with varying smoothness and observation patterns. The expected outcome is a better understanding of how properties of the underlying stochastic process should inform neural architecture design, ideally establishing a connection between neural processes and classical low-rank, sparse, and numerical approximation methods. Prerequisites: A solid background in machine learning, probability, and linear algebra, together with programming experience, is expected. Familiarity with Gaussian processes, numerical methods, or partial differential equations is helpful but not required.

Supervisor: Tobias Weber

Neural operators can learn solution maps of partial differential equations, but often require large amounts of expensive high-resolution simulation data. Multi-resolution training aims to reduce this cost by combining many cheap coarse simulations with fewer fine-resolution solutions. Existing approaches, however, usually treat coarse simulations as deterministic training data and ignore information about their numerical accuracy. This project explores simulator-aware multi-resolution neural operator training, where uncertainty estimates from probabilistic numerical solvers are used to inform how coarse- and fine-resolution data are combined. We will review existing approaches to multi-resolution neural operator training, and investigate whether numerical uncertainty can help neural operators use coarse simulation data more effectively.

Prerequisites:

▶ Experience with deep learning experiments in JAX or PyTorch.

▶ Interest in scientific machine learning and numerical simulation.