Abstract
It is not difficult for most people to distinguish one music style from another. However, how the brain processes this simple task is still unknown. In order to shed light on this problem, and explore ways to apply cutting-edge deep learning technology in the music engineering field, two tasks have been conducted using convolutional neural networks (CNNs). CNNs, inspired by biological visual systems, have been widely used for image -related applications and achieved great success, but they have rarely been applied in audio-related field. In this thesis study, we examined the possibility of deploying a CNN in audio-related tasks and the potential of using it as a creative music composition tool. The first task applied a CNN model with three convolutional and two fully Âconnected layers to a binary music style classification task. The trained CNN is designed to distinguish a five-second Chinese population music ( C-pop) clip from a same duration melodic death metal music (MDM) clip by using the raw audio signal as input. With 4800 training examples and 20 epochs, it obtained about 80% accuracy on 480 testing examples. The second task was based on the trained CNN model and analogous to the DeepDream visual project. The DeepDream project uses a CNN that is trained for a visual classification task to enhance the emergence of elements that may not exist in an input image. The resulting image has a dreamlike appearance and, depending on which CNN layer is used for the enhancement, the emerging elements will be different. For lower layers, the image appears with more elementary shapes, and for higher layers, it displays more complete objects. Also, if using a reference image to guide the modification, the elements of the reference image will be blended into the input. In this thesis study, similar procedures were done with a randomlyÂselected C-pop clip using the trained CNN model from the classification task. The goal is to modify the input audio signal to increase activations from a particular convolutional layer such that extra elements stored in this layer can be obtained along with the original audio signal. The resulting non-guided audio clips were hierarchical from bursts and pulses to a mixing of original C-pop with some metal textures, based on different convolutional layers from lower to higher depths. The guided audio clips gained some metal style features, but lost the original timing of dynamic changes.