Post-Processing Functionality

I began work today on implementing post-processing modules. The mGen now loads and keeps track of an unlimited number of post-processing modules and also stores and retains the configuration data, just like generative modules.

The only part left to do is handle the actual generation, in which mGen will have to pass the entire compositional data structure to the post-processing modules and update the structure based on the output of the modules.

Unlike the generative modules, which have permission to view the entire data structure, but can only to add note data to their respective channels, post-processing modules will be granted complete read and write access to the entire set of data to maximize the flexibility of these special modules.

Structural Updates & Touch-Ups

Several fixes and minor changes are going on in the mainframe. Loading and saving have each been completely reworked to use the GDS data system instead of INI file storage. The result is a smaller file that is easier to work with. The plugin loading algorithm has also been tweaked to no longer function on absolute paths but rather on relative ones. Now I won't have problems loading the project files I saved on my laptop into my desktop's mGen (previously the project files were not transferrable because of the differences in the file path names on my two systems, even though the differences are not at all relevent to mGen).

GGrewve Intelligent Decision Making

Major progress has been made with the GGrewve drum plugin. It now has the ability to determine which words are more intense based on an extremely scalable and versatile analysis engine. Using this knowledge, GGrewve is able to place words more appropriately than before. Without even needing to look at the context of the words in the original MIDI file from which they were extracted, GGrewve uses the words to simulate the way a real drummer plays. Accents and fills will occur more during even measures, with a greater weight every fourth measure. In this way, GGrewve allows listeners to more effectively keep track of the measure, which gives the composition a much greater overall coherency.

The intelligent GGrewve engine is a step forward in autonomous music generation!

GGrewveWriter Analysis Tool

The development of GGrewve is going well, but rather slowly. This is partly because of the difficulty in expanding the dictionary. This is not an unfamiliar problem: I ran into the same issues with Variating Drummer. It's generally difficult to write drum patterns in anything short of a full-blown sequencer. Since writing a sequencer is beyond my capabilities, I have developed a pretty cool alternative.

Enter GGrewveWriter, the intelligent drum pattern analysis tool that disects a drum beat and packages it up for use with GGrewve.

In a nutshell, GGrewveWriter takes a MIDI file that contains a drum beat, splits the beat up into sections of an appropriate length, and derives "words" from these small sections. GGrewveWriter pays close attention to the placement of words within the original beat and the subtleties of wordly interactions. GGrewveWriter then creates a compressed dictionary file that contains all the grammatical data about the drum beat found in the MIDI file. This compressed dictionary can be read directly by GGrewve, and is actually more efficient than the uncompressed dictionaries used previously.

The whole thing is very impressive in practice.

It took me no more than a few minutes to write eight measures of a neat little drum beat in a sequencer and export it to a MIDI file. After exporting, I let GGrewveWriter go to work with the file. After only a few seconds, I had a fresh compressed dictionary sitting in the folder. I loaded the dictionary into GGrewve and let the generation begin. Sure enough, out came forty measures (I had requested this much) of well-placed, humanized drum beat. I had actually made the original MIDI beat sound very stale (I did not vary the volume levels at ALL, so it sounded very robotic) to see how well the humanization routines within Grewve were working. Sure enough, the computer's output sounded a heck of a lot more human than the beat that I, a human, had created. The output was varied and clearly derived from my beat, but not exactly a copy and pleasantly unpredecatable.

Writing the beat, creating the dictionary, and generating took about seven minutes. Had I tried to achieve the same thing with the Variating Drummer, it would have taken hours and hours and the final result would not have been as impressive.

This is a very successful test for GGrewve, and I look forward to more impressive results in the future (we may finally be ready for drum fills!)

GDS Inspector

A few months ago, I developed my own custom, flexible data structure designed to handle the challenges presented by the project in terms of data storage and transfer. I refer to these structures as GDS, or generic data structures. I have been improving on the capabilities of GDS since I first created them. Among the many important features of the GDS is it's ability to store other another GDS within itself. A GDS may, this way, be nested up to ten times within another. In other words, you can have a data structure within a data structure within a data structure within a data structure, and so on, which is extremely handy for keeping data organized yet easy to transfer and work with at the same time. I can basically store an entire heirarchicaly organized pool of data in a single variable.

These structures, however, can get quite massive (especially the main structure that mGen uses to pass between modules - this structure usually has hundreds of sections and up to four nested levels).

Today I created a small tool called the "GDS Inspector" to facilitate the process of examining GDS blocks when debugging. The problem arose while working on GGrewv that I needed to inspect the dictionary block because I was having problems. Opening the block in notepad, however, was very messy because of the way GDS stores data (which is flexible and efficient, but not notepad-friendly). The program I wrote display all the data in a given GDS block in a manner much more conducive to analysis - a treeview. A treeview is basically a hiearchy. One can see the nested levels of GDS storage and click on each entry to view the data held therein. This greatly facilitated the debugging process, not to mention I had a great deal of fun looking at how complex the main data structure actually is (notepad doesn't really do it justice).

So that's a nice little improvement in my working methods.

GGrewve

GGrewve generated output for the first time today. The dictionary right now is very small and the subdivision rules are very strict, so there's really no variety in the drumming yet, but that was intentional so that I could test the engine. Everything worked very nicely.

One of the cooler parts of the engine is the "humanization" function that varies the velocities to simulate a "groove" as well as add ghost notes to the snare drum like professional drummers do automatically. Even with a tiny dictionary (right now it has only five simple words that correspond to four-beat patterns) the drumming quality already surpasses the Variating Drummer, so I think I'm on the right track with this plugin.

I've still got obstacles to conquer. Most notably, I have to devise a way for GGrewve to determine which words are "fancier" and "heavier," so that it knows in what context to use the words. For example, a fill should obviously not come on the first beat of the first measure of a section...it should be saved for a turning point or a transition into another section. I could just include some quantitative specifications in the word files to let GGrewve know where they should be used, but I think that's taking too much of the computer work out of it. I'll write the words but GGrewve has to figure out how to use them. This will, of course, be a lot more work that just telling it when to use words, but I think the results will be more satisfying. Expanding the dictionary will also be a lot easier if I don't have to take the time to think about when each word should be used.

In conclusion, I'm excited about where the drumming side of mGen is going and I think GGrewve has a bright future.

GGrewve

I'm still in the process of implementing the grammatical subdivision idea in a drum plugin. So far everything is going smooth and it's shaping up to be a really nice module! It's called GGrewve (grammatical groove).

I hope to have some simple output in a few days. The idea is a LOT easier to work with than that of the Variating Drummer, which has become so convoluted that even I have trouble figuring out how to build a drum pattern (which shouldn't happen, considering the computer is supposed to be the one doing the work).

At any rate, more to come on grammatical subdivision and GGrewve in the near future.