.rst .pdf

repository open issue suggest edit

Musikinformatik SoSe2021

Welcome to the course Musikinformatik of the Robert Schumann Hochschule Düsseldorf for SoSe 2021. The goal of this course is to introduce and learn common data analytic tools and machine learning algorithms for generating sounds. It will not just take a look at the practical aspects of this endeavor but also discuss philosophical and mathematical aspects of the discussed problems.

Within the course we will rely on the programming language Python - a high level scripting language - which became a per-se standard for data analytics and machine learning over the last years.

We will also rely on the sound platform SuperCollider for generating sounds.

Open for contributions!

If one comes across any mistake or has an suggestion for any improvements - do not hesitate to contribute! The source code for this course is open source and can be reviewed and improved by anybody. See Contribute.

Course information

• Environment setup
  • Binder
  • Repository setup
  • SuperCollider setup
  • Python setup
  • Setup via Docker
  • Updating and working with the repository
• Contribute
  • Jupyter notebook conventions
  • pre-commit
  • Building the documentation
• Bibliography

Basics

• Math basics
  • Data structures
  • Functions
  • Working with vectors and matrices
  • Statistics
• Dimensionality in SuperCollider
  • From scalar to vector to matrix
  • Working with matrices
• Python basics
• Dimensionality in Python
  • From scalar to vector to matrix
  • Working with matrices
• Machine Learning basics
  • Datasets
  • Classification
  • Regression
  • Clustering
  • Dimensionality reduction
• Introduction to neural networks
  • Loading the MNIST dataset
  • Creating our first neural network
  • Training a neural network
  • Tuning the neural network
  • Analysing the results
• Convolutions and deep neural networks
  • Convolutions
  • Convolutional neural networks
• Autoencoders
  • Generating examples
• RNNs
  • Motivation
  • Pre-processing
  • Building the LSTM network
  • Model on word basis
• Communicating between SuperCollider and Python
  • OSC
  • JSON
• Generating MIDI notes via RNNs
  • Loading the data
  • Convert data to sequences
  • Writing the RNN
  • Training
  • Predict
  • Making predictions in real time with SuperCollider
  • Extending

Using Python to synthesize sound

• Generating sounds via Python
  • The canvas of an audio file
  • Audio signal synthesis via numpy
• Resynthesizing sound via spectogram
  • Loading a sound file
  • FFT
  • Spectogram
  • Re-Arranging the spectogram
  • Re-writing into a class
  • Fine tuning the results
  • Where to go from here
• Matrix decomposition on STFT
  • Load data
  • Decompose STFT signal
  • Splitting complex and imaginary component
  • Exchanging components
  • Next steps
  • NMF and machine learning
• Wavesets in Python
  • Calculating the split points
  • Wavesets extensions

Lesson 1

• Extracting information from MIDI files
  • Getting the dataset
  • Parsing the dataset
  • Loading MIDI files in Python
  • Processing all files
  • Analysis of metadata
  • Filter and extract the data
  • Conclusion

Environment setup