Supervisor: Paul Fischer

Estimating the parameters of a differential equation from noisy measurements of its solution is an inverse problem, relevant to many real world applications. The literature offers a long list of methods for it, each with its own trade-offs between robustness, cost, and statistical interpretation. Probabilistic ODE filters, which solve a differential equation as a Bayesian inference problem, offer a single framework in which model, data, and unknown parameters are treated jointly. This project studies how established parameter-inference methods relate to each other when viewed through this framework, and how the framework can be used to design inference schemes that are more robust and scale to larger problems. Depending on interest, the work can lean towards theory, towards efficient implementation, or towards applying the resulting methods to challenging inverse problems from science and engineering. You should be interested in Bayesian inference, numerical methods for differential equations, and the connections between them.

Supervisor: Thomas Christie

Recent work has shown that, for a data distribution generated by a Markov chain, the quantities required for generating samples from the corresponding masked discrete diffusion process are available exactly, cheaply, and in closed-form via the forward-backward algorithm, and forward-filtering backward-sampling. Given that we know what an optimal algorithm looks like, it would be interesting to train a transformer on this task and try to reverse-engineer what it has learned, drawing on techniques from the field of mechanistic interpretability. We can also monitor the training dynamics of the model - to what extent does the model just memorise the examples it has seen during training, and how does its ability to generate novel (and valid) samples improve over the course of training? Does the Grokking phenomenon occur and, if so, can we tie it to learning the forward-backward algorithm via the mechanistic interpretability aspect of this project?

Prerequisites:

▶ Proficiency with PyTorch or JAX

▶ Strong maths skills.

▶ The ability to work independently.

Supervisor: Tim Weiland 

The year is 2025. Big tech companies churn out humongous deep learning models on what feels like a weekly basis. In physical applications, people are slowly buying into the idea that all they need is yet another stack of layers in their neural networks. In these dark times, one small and humble hero resists: the Gaussian process. Gaussian processes are naturally able to enforce physical conservation laws, and as such, are often marketed as a great fit for physical applications. Yet at the same time, it is undeniable that the ”amortization aspect” of neural operators (i.e. the property that they ”learn” from related simulations) is incredibly powerful. The goal of this project is to combine both of these properties in one,  through a deep, physics-informed Gaussian operator. Interested? Reach out to Tim for the details.

Prerequisites:
Solid prior knowledge in Probabilistic Machine Learning, particularly GPs
Prior experience in PyTorch / JAX / …; pick your poison

Supervisor: Bernardo Fichera

Airplane design requires selecting the appropriate structural configuration and materials. Here, appropriate means achieving the desired performance across the intended flight regime. From an aerodynamic perspective, performance is strongly influenced by the structural deformations of the aircraft in flight. The central design challenge, then, is: what structural configuration (e.g., wing length) and material properties (e.g., Young’s modulus) will produce the desired deformed shape during flight? Addressing this challenge amounts to solving the inverse aeroelastic problem -- that is, determining the structural and material design from the coupled interaction between aerodynamics and structural deformation. Both the structural and aerodynamic aspects of this problem involve solving PDEs. We aim to leverage probabilistic PDE solvers to tackle such inverse problems.

This approach offers two main advantages:

• Natural incorporation of uncertainty quantification into the inverse solution.
• A framework that recasts the inverse problem as a form of Bayesian optimization.
   

Supervisor: Joanna Sliwa

A Hessian matrix captures the second-order partial derivatives of a model’s loss function with respect to its parameters. While it provides valuable information for optimization, calibration, and uncertainty estimation, computing the exact Hessian is infeasible for modern deep networks. This project focuses on evaluating scalable Hessian approximations in JAX. Using the laplax library, we will conduct exploratory studies in some applications (depending on interest) such as continual and transfer learning, curvature-aware model merging and decomposition, and LoRA fine-tuning for large language models. The objective is to both showcase and extend the practical use cases of Hessian-based methods by contributing implementations of common applications directly into the laplax library.

Prerequisites:
> Prior experience with JAX
> Familiarity with neural network training and optimization
> Some exposure to second-order methods beneficial but not strictly required