Maya Sound Scape Plugin
Software
Audio Analyzer written in C++ with fmod library
Maya Command Plugin written in Python
Description
I wanted to use this project as an opportunity to explore Maya from a programmer's perspective. So I came up with the idea of extending its functionality by creating a plugin. I asked myself the question, "If an audio file represented a city, what would it look like?" I thought it would be really interesting to compare different genres of music, in terms of the structure they would create.
Audio Analyzer
The first step of this project was to create a program that could analyze an audio file. I wanted to be able to determine the energy levels of various frequencies of the sound waveform. So the first task was to run the waveform through a Fast Fourier Transform in order to perform a spectrum analysis. The output of an FFT function is a linear spectrum of energy levels in the frequency domain. However, humans interpret sound in terms of octaves, which are exponential. So, I took the data and transformed it again using a base two logarithmic function.
So what does all that accomplish for me?
| Ten Band Frequency Equalizer (Hz) | |||||||||
| 31.5 | 63 | 125 | 250 | 500 | 1000 | 2000 | 4000 | 8000 | 16000 |
At any given moment during a song, I can tell you which frequencies are the strongest. So if the song contains a lots of low base notes, the lower frequency bands will be strongest. If there are high trill notes, the higher frequency bands will be strongest. In fact, I measure these frequencies at 10 different levels. The table above shows the midpoint of the frequency bands that are being measured. I also wrote some simple code that detects when a beat has occurred. Basically, I measure how the bands are trending over a short period of time, and if there is a spike then I declare that a beat must have occurred. This portion of the project was written in C++ with the help of the fmod library. It takes an audio file as input and creates a text file that records all of this information. Each second contains 30 samples because I planned on making the resultant video at 30 frames per second.
Maya Plugin
The audio analyzer served to generate data that represented how each of the audio files changed over time. The goal of the Maya Plugin is to take that data and create a visualization so that it can easily be interpreted, and enjoyed as an experiment in conceptual art. I decided it would be interesting to view the data as a city. Each frame creates a single city block. Each block is comprised of nine buildings.The buildings' heights represent the energy levels of each of the frequency bands. The buildings are created with a red hue unless a beat was detected in that frequency band (in which case they are blue).
There are several ways to interact with the Maya API. I chose to use the Python bindings for quick iteration.
The plugin includes a simple GUI that renders a menu in the Maya interface. Then the user chooses the audio file created by the audio analyzer tool. Clicking the "Create Sound Scape" button finishes the process. The user renders the animation just like any other Maya file!
Important Note
This project was created in Maya 2011 before MASH was available.
