Adjusting Model Size in Continual Gaussian Processes:
How Big is Big Enough?

Pescador-Barrios, G., Filippi, S., and van der Wilk, M.
ICML 2025. Spotlight.

We propose a method to dynamically adjust the model size of a Gaussian Process during training in a continual learning setting.

Read the Paper

Figure: Three continual learning scenarios. The model size is adjusted during training to match the capacity requirements of each task.

A Bayesian Nonparametric View on Adapting Neuron Count During Training

Pescador-Barrios, G., van der Ouderaa, T., Nockels-Stewart, O.J., Filippi, S., and van der Wilk, M.

Soon on arXiv

We argue that selecting inductive bias and model size are inextricably linked, and that model selection procedures should solve both problems jointly.

Figure: Model predictions, ELBO, and neuron count evolve together during training, with growth and pruning as periodic structure is learned.