This interest group aims at studying and discussing how concepts from the theory of 'rough paths' can be deployed to address specific challenges in streamed-data-science. The schedule will consists of a weekly group conversation at the Turing, lasting between one and two hours. Occasionally, the group will organise workshops, lectures and host external speakers.
Explaining the science
Imagine one wanted to describe the state of a complex system (such as the weather, the brain, or a helicopter) that evolves as a function of its present state and the infinitesimal variation of an external action (for instance the temperature, the air pressure near the ears, or the wind pressure on the helicopter blades respectively). The theory of rough paths provides a mathematically sound way to address these kind of questions.
Its central object, the path signature, provides a very efficient way to capture the information contained in any type of signal, including ones taking values in high dimensional spaces or of a very oscillatory nature.
From a machine learning perspective, the signature can be seen as a feature map, which happens to be invariant under time re-parametrisation, and offers a natural way to reduce the dimensionality of the input path. From a deep learning viewpoint it can be interpreted as a universal pooling function within a neural network architecture.
The rich and astonishing algebraic structure of the space where signatures live allows for the writing of efficient software to compute signatures rapidly and with very high precision. These are readily available and wrapped in easy-to-use python packages (esig). Applications of rough paths theory to data science have been extremely successful in recent years. These include Chinese handwriting recognition, option pricing in finance, natural language processing, audio signals processing, and mental health.
This group will be particularly interested in learning functions on streams. When dealing with streamed data common challenges include missing data, multimodality and high dimensionality. The signature provides a powerful tool to address all these challenges.
State-of-the-art results can be achieved when the rough paths machinery is combined with cutting edge machine learning techniques. Therefore, the purpose of this group will be to discuss and develop innovative ideas aimed at combining these two areas.
Deep Signatures: how the signature transform can be utilised within a neural network architecture
Challenges: Current deep learning technologies (Pytorch, Tensorflow etc.) are devised to treat input streams as tensors in a high dimensional space.
Consequently, the resulting back-propagation mechanism is entirely based on the pointwise differentiation of the function parameterised by the neural net with respect to the input path.
A core result from rough paths theory tells us that looking at an oscillatory signal pointwise is a very ineffcient way to capture information contained within it. Instead, one should store and process the signatures of the stream over coarser intervals. How can this fact be made compatible with current deep learning mechanisms?
How to get involved
To join us, please email [email protected]
Cristopher Salvi, [email protected]