12/08/16 Adaptivity in Oscillator-based Pulse and Metre Perception

Beat induction is the perceptual and cognitive process by which we listen to music and perceive a steady pulse. Computationally modelling beat induction is important for many MIR methods and is in general an open problem, especially when processing expressive timing, e.g. tempo changes or rubato.

Large et al. (2010) have proposed a neuro-cognitive model, the Gradient Frequency Neural Network (GFNN), which can be model the perception of pulse and metre. GFNNs have been applied successfully to a range of 'difficult' music perception problems (see Angelis et. al., 2013; Velasco and Large, 2011).

We have found that GFNNs perform poorly when dealing with tempo changes in the stimulus. We have thus developed the novel Adaptive Frequency Neural Network (AFNN) that extends the GFNN with Righetti et al.'s (2006) Hebbian learning rule. Two new adaptive behaviours (attraction and elasticity) increase entrainment, and increase model efficiency by allowing for a great reduction in the size of the network.

We will extend our comparative study presented at this year’s ISMIR with a new set of results incorporating different forms of expressive timing and we will discuss the biological and cognitive plausibility of the AFNN.

View Poster »

poster, software, music


08/08/16 Adaptive Frequency Neural Networks for Dynamic Pulse and Metre Perception

Beat induction, the means by which humans listen to music and perceive a steady pulse, is achieved via a perceptual and cognitive process. Computationally modelling this phenomenon is an open problem, especially when processing expressive shaping of the music such as tempo change. To meet this challenge we propose Adaptive Frequency Neural Networks (AFNNs), an extension of Gradient Frequency Neural Networks (GFNNs).

GFNNs are based on neurodynamic models and have been applied successfully to a range of difficult music perception problems including those with syncopated and polyrhythmic stimuli. AFNNs extend GFNNs by applying a Hebbian learning rule to the oscillator frequencies. Thus the frequencies in an AFNN adapt to the stimulus through an attraction to local areas of resonance, and allow for a great dimensionality reduction in the network.

Where previous work with GFNNs has focused on frequency and amplitude responses, we also consider phase information as critical for pulse perception. Evaluating the time-based output, we find significantly improved responses of AFNNs compared to GFNNs to stimuli with both steady and varying pulse frequencies. This leads us to believe that AFNNs could replace the linear filtering methods commonly used in beat tracking and tempo estimation systems, and lead to more accurate methods.

Read Paper »

paper, software, music


27/07/16 Metrical Flux: Towards Rhythm Generation in Continuous Time

This work-in-progress report describes our approach to expressive rhythm generation. So far, music generation systems have mostly focused on discrete time modelling. Since musical performance and perception unfolds in time, we see continuous time modelling as a more realistic approach. Here we present work towards a continuous time rhythm generation system.

In our model, two neural networks are combined within one integrated system. A novel Adaptive Frequency Neural Network (AFNN) models the perception of changing periodicities in metrical structures by entraining and resonating nonlinearly with a rhythmic input. A Recurrent Neural Network models longer-term temporal relations based on the AFNN's response and generates rhythmic events.

We outline an experiment and evaluation method to validate the model and invite the MUME community's feedback.

Read Paper »

paper, software, music


27/07/15 Perceiving and Predicting Expressive Rhythm with Recurrent Neural Networks

In the quest for a convincing musical agent that performs in real time alongside human performers, the issues surrounding expressively timed rhythm must be addressed. Automatically following rhythms by beat tracking is by no means a solved problem, especially when dealing with varying tempo and expressive timing. In the generation of rhythm, some existing interactive systems ignore the pulse entirely, or fixed a tempo after some time spent listening to input. We take the view that time is a generative property to be utilised by the system.

In this research, we present a connectionist machine learning approach to expressive rhythm metacreation, based on cognitive and neurological models. We utilise a multi-layered recurrent neural network combining two complementary network models as hidden layers within one system.

The first layer is a Gradient Frequency Neural Network (GFNN), a network of nonlinear oscillators which acts as an entraining and learning resonant filter to an audio signal. GFNNs model the perception of metrical structures in the stimulus by resonating nonlinearly to the inherent periodicities within the signal, creating a hierarchy of strong and weak periods.

The GFNN resonances are then used as inputs to a second layer, a Long Short-term Memory Recurrent Neural Network (LSTM). The LSTM learns the long-term temporal structures present in the GFNN's output, the metrical structure implicit within it. From these inferences, the LSTM predicts when the next rhythmic event is likely to occur.

We have trained the system on a dataset selected for its expressive timing qualities and evaluate the system on its ability to predict rhythmic events. These predictions can be used to produce new rhythms, forming a generative model. As such, the GFNN-LSTM model has the potential to be used in a real-time interactive system, following and generating expressive rhythmic structures.

Read Paper » Read More »

paper, software, music


03/10/14 Studying the Effect of Metre Perception on Rhythm and Melody Modelling with LSTMs

In this paper we take a connectionist machine learning approach to the problem of metre perception and melody learning in musical signals. We present a multi-layered network consisting of a nonlinear oscillator network and a recurrent neural network. The oscillator network acts as an entrained resonant filter to the musical signal. It `perceives' metre by resonating nonlinearly to the inherent periodicities within the signal, creating a hierarchy of strong and weak periods. The neural network learns the long-term temporal structures present in this signal. We show that this network outperforms our previous approach of a single layer recurrent neural network in a melody and rhythm prediction task.

We hypothesise that our system is enabled to make use of the relatively long temporal resonance in the oscillator network output, and therefore model more coherent long-term structures. A system such as this could be used in a multitude of analytic and generative scenarios, including live performance applications.

Read Paper »

paper, software, music


14/09/14 Beyond the Beat: Towards Metre, Rhythm and Melody Modelling with Hybrid Oscillator Networks

International Computer Music Conference

In this paper we take a connectionist machine learning approach to the problem of metre perception and learning in musical signals. We present a hybrid network consisting of a nonlinear oscillator network and a recurrent neural network. The oscillator network acts as an entrained resonant filter to the musical signal. It ‘perceives’ metre by resonating nonlinearly to the inherent periodicities within the signal, creating a hierarchy of strong and weak periods. The neural network learns the long-term temporal structures present in this signal. We show that this hybrid network outperforms our previous approach of a single layer recurrent neural network in a melody prediction task.
We hypothesise that our hybrid system is enabled to make use of the relatively long temporal resonance in the oscillator network output, and therefore model more coherent long-term structures. A system such as this could be used in a multitude of analytic and generative scenarios, including live performance applications.

Read Paper » View Poster »

paper, poster, software, music


17/12/13 Deep Rhythms: Towards Structured Meter Perception, Learning and Generation with Deep Recurrent Oscillator Networks

Beat induction allows us to “tap along” to the beat of music, perceiving its pulse. This perceived pulse can be present in the stimulus, but it is often only implied by the musical events. What's more, performed music is rarely periodic and subject to the performer’s expressive timing. This makes beat induction difficult to model computationally...

View Poster »

poster, software, music


09/09/12 A Stigmergic Model for Oscillator Synchronisation and its Application in Music Systems

International Computer Music Conference

Non-linear and chaotic dynamics, predominantly used in engineering, have become a pervasive influence in contemporary culture. Artists, philosophers and commentators are increasingly drawing upon the richness of these systems in their work. This paper explores one area of this territory: the synchronisation of a population of non-linear oscillators used for the generation of rhythm as applied in musical systems.

Synchronisation is taken as a basis for complex rhythmic dynamics. Through the self-organisation notion of stigmergy, where entities are indirectly influenced by each other, the notion of local field coupling is introduced as a qualitatively stigmergic alternative to the Kuramoto model and noise, distance, delay and influence are incorporated.

An interactive system of stigmergic synchronised oscillators was developed, that is open to be used across many fields. The user is allowed to become part of the stigmergy through influencing the environment. The system is then applied to the field of music, generating rhythms and sounds by mapping its state

Read Paper » View Software »

paper, software, music


© Andrew Elmsley 2017 | andy [at] andyroid.co.uk