Introduction
This bi-monthly seminar series explores real-world applications of physics-informed machine learning (Φ-ML) methods to the engineering practice. They cover a wide range of topics, offering a cross-sectional view of the state of the art on Φ-ML research, worldwide.
Participants have the opportunity to hear from leading researchers and learn about the latest developments in this emerging field. These seminars also offer the chance to identify and spark collaboration opportunities.
About the event
Choosing a set of hyperparameters that guarantees an efficient convergence in machine learning is a key factor for effective and precise training. Despite intense research activity in this direction so far, a fundamental theory describing the effects of hyperparameters on training remains elusive. In this talk, we present a numerical determination of the empirical phase diagram of a one-hidden-layer non-linear random neural network, showing that three distinct phases can be identified based on qualitatively and quantitatively different dynamical behaviours of the singular values of weight matrices. The control parameters of the phase structure are α/|B| and σ^2_W. α/|B| is the combination of two hyperparameters of the optimiser, learning rate α and batch size |B|, which can be interpreted as an effective temperature according to the Dyson Brownian motion of the singular values. σ^2_W is the variance of the initial weight matrices, which describes the strength of the initial disorder in the model. We argue that the phase diagram of the system can be employed for choosing optimal hyperparameters. Moreover, using tools from spin-glass models, we outline possible directions for an analytical derivation of the numerical results.
