TMCF workshop: Physics-informed machine learning

Introduction

This online workshop focuses on Statistical Mechanics (SM) and its application to Artificial Neural Networks (ANN) and Explainable Artificial Intelligence (XAI)

SM is one of the pillars of Theoretical Physics, characterising the collective phenomena in systems composed of numerous interacting microscopic entities. Interestingly, these principles of SM are also applicable to biological neural networks, and thus, to artificial neural networks. It underlines the concept that a single neuron cannot recognise an information pattern, rather, it emerges from the interaction among neurons.

Deep within the realm of ANNs, Restricted Boltzmann Machine (RBM) are a fundamental part of these modern learning architectures. Their structure, comprising a visible layer to process empirical data and a hidden layer to infer correlations and store patterns, plays a key role in understanding both shallow and deep learning networks.

In the biological realm, the Hebbian learning rule, the backbone of the Hopfield model, is used as a reference for associative memory and pattern recognition. This model is viewed as a "spin-glass" in the SM terminology and the tools used to study it can also be applied to machine learning, particularly when investigating the emergent properties of RBMs and their generalisations.

About the event

This workshop will showcase new methodologies developed during the Turing Theory and Methods Challenge Fortnight event that took place in January 2023. One of the primary objectives of this workshop is to comprehend the emergent skills deep artificial networks display when their control parameters are varied. It is presently unclear what these skills are and how many emergent properties these networks possess, but deep networks' capability to variably adjust their signal-to-noise ratios offers a clear avenue for investigation.

We also aim to explore the spectral properties of these networks, which are crucial in avoiding overfitting even when they are hyper-parametrised. While the workings of these networks are clear on random datasets, their control on structured datasets needs further investigation.

Lastly, the workshop will explore the potential of including feed-forward networks in SM treatment by relaxing the requirement of detailed balance. Though this is a challenging task that leads us away from equilibrium statistical mechanics towards a completely off-equilibrium state, it holds the promise of paving the way towards a comprehensive SM perspective on Theoretical Artificial Intelligence.

Please contact Dr Andrea Pizzoferrato if you have any queries regarding the event.

Recordings of the event are available here.

Agenda

Time (London time - UTC 01)	Title	Speaker
9:30-9:40	Welcome	Ioannis Kosmidis
9:40-9:50	Workshop summary	Andrea Pizzoferrato
Morning Session Chair		Adriano Barra
10:00-10:40	From machine learning to glassy dynamics and back: aging and memory	Peter Sollich
10:40-11:20	Explaining and exploiting out-of-equilibrium effects in the training of generative models	Beatriz Seoane
11:20-12:00	Phase transitions in the dynamics of probabilistic consensus	Alex Mozeika
Lunch break
Afternoon Session Chair		Elena Agliari
14:00-14:40	Fundamental limits in structured PCA, and how to reach them	Jean Barbier
14:40-15:20	Bipartite neural network: effect of non-linearities in the encoding latent variables	Aurélien Decelle
15:20-16:00	Replica symmetry breaking in neural networks	Alessia Annibale
16:00-16:40	Decontamination of GLM inferences using replica-based theory of overfitting	Ton Coolen
16:40	Final remarks	Andrea Pizzoferrato

Part of the fundings for the scientific contents of this workshop come from the BULBUL grant (Brain inspired Ultra-fast and Ultra-sharp machines for assisted healthcare), jointly funded by Ministero degli Affari Esteri e della Cooperazione Internazionale (MAECI, IT) and Ministry of Science, Technology and Space (MOST, IL).