Skip to main content

What are the Music Motion Tools?

The Music Motion Tools (MuMo App) provide the technical basis for low-threshold motion tracking in various applications in the context of music education. A camera is used to capture the movement of hands playing "in the air" and transfer it to a virtual instrument.

Mallets
Hand gestures
Percussion

Functions

🎼Mallets

The available tone selection and the current playing position are visualised to make playing easier. You can select real instrument settings as well as experimental tone arrangements. One or three layers can be displayed and a looper can be activated.

🥁Percussion

It is possible to use common percussion instruments (e.g. congas) in the percussion section. These can be combined with each other using a looper function, e.g. to create more complex patterns with several rhythms/instruments.

👋Hand gestures

Blocks can be moved with the hands, allowing various sound parameters to be changed. There is also a virtual mixer and samples can be used. In addition, pitch progressions can be drawn and the timbre can be varied using different hand positions, similar to playing a theremin in the air.

Contactless interaction with camera-based hand tracking

Curious?

Try out the Music Motion Tools for yourself and share your ideas with us!

System requirements

  • Mac with M1 chip or
  • new Windows computer with latest AMD or Intel chip (Intel i7 minimum)

Extras

 

Feedback

📩 We want to continually improve the app - so we look forward to your feedback

Thanks to

🎙 The idea for the app was developed as part of the RE|LEVEL project at the Nuremberg University of Music.