Thursday, December 24, 2009

Moved

For those still interested, the project development blog has been relocated to

Tuesday, June 23, 2009

airframe - Lindenmayer Upgrade

I've been working on airframe lately in hopes of getting the structure up to par with the other modules (progression module is my next target). I've come to the conclusion that generative modules need to be a lot smarter than they are right now, because they shouldn't rely too heavily on the structure module to provide "solid" structure data. Structures are really not very complex. To be perfectly honest, the best option would probably be a very simple KBS with a pre-programmed list of structure forms (ABABCB, etc). Until then, I'm overshooting the complexity. So the generative modules will be responsible for keeping the piece coherent and figuring out what to do if the structure modules sends overly-complicated (or overly-simplified) instructions.

The core system of airframe has been decided upon - at least for now. I'm proud to introduce it as the first mGen plugin to use a Lindenmayer system, also known as an L-system, which is basically a simple grammar system at first glance but has roots in fractals. I think the L-system will provide a refreshing dose of simplicity and organized complexity. I really won't know what to expect until I hear it, and if it works, it'll be far too good to be true given how easy an L-system is to implement.

Of course I plan on layering other processes over the L-system engine to refine the structure output (maybe an ear module?), so airframe will technically still be a hybrid plugin.

EvoSpeak is also still progressing.

Monday, June 22, 2009

EvoSpeak - Optimization

Yes, I'm STILL working on getting the analysis part of EvoSpeak working. I now have the structure of the species' brains figured out and I've optimized the analysis engine a LOT, thanks to the new storage method of the analysis filters. So things are looking pretty good and soon enough I should be working within the interface of EvoSpeak instead of grinding around in the code.

I'll admit, progress on mGen is coming slowly. It still looks like I'm going to make the code deadline that I set for Wednesday (17,000 lines), which is encouraging. How is mGen shaping up in the big picture/long run? That's what I'm more worried about. I'm going to have to step back and take a serious look at what's going on after I hit this deadline. I don't even really have anything for Mr. Taranto to help with yet, even though our meeting is in under two weeks. It's time to step up to the plate.

Wednesday, June 17, 2009

EvoSpeak - Analysis

Work is starting to get pretty messy with EvoSpeak. I'm trying to design a very generalized analysis framework to allow easy analysis of just about any relationship. Doing so is not at all easy. I'm trying to set up a "perception" matrix that simulates the state of any given sample at any given point in time. The idea is that an analysis function can built a filter matrix and then call the perception function, which will then compare the filter matrix to the perception matrix and gather some statistics.

The first analysis I'm designing is, of course, a zeroeth-order Markov. Fancy language aside, what it boils down to is this: did the species use a certain word (melodic or rhythmic) in a certain sample? So a zeroeth-order Markov simply deals with the innate "quality" of certain words over other words, not taking into account ANY contextual variables.

Problems are arising with this general framework. It's very difficult to obtain certain state values to populate the matrix because of the grammar engine design. The melodic and rhythmic data streams are asynchronous, so melodic events don't necessarily line up with rhythmic events, which makes finding synchronous data (like perception data) very difficulty. Apparently I've messed up in trying to separate the streams because some of the preliminary statistics are doing some strange things.

On top of all that, the analysis is a LOT slower than I thought it would be, even after serious reconstruction and optimization. I knew that it would take a lot of loops and recursions to do the analysis...but I thought the computer would just chew through them. Already a simple zeroeth-order Markov analysis on the melody alone costs about 2.6 seconds. Using that number and extrapolating, a second-order Markov analysis would take a whopping sixteen minutes, which is simply unacceptable. And that's only to level-up once. I'm definitely going to have to figure something out there.

While I'm running into some obstacles, EvoSpeak is still advancing steadily and I'm confident that the analysis functions will soon be fully-functional.

Tuesday, June 16, 2009

EvoSpeak - Experience & Dörf

I finished a lot of EvoSpeak coding tonight. The training process is mostly coded. The species will spit out samples, the user listens and grades the performance, and then the results are submitted and stored to the species' memory. It's also possible to create new species now from within the configuration utility.

I created my first creature, Dörf, today. He speaks the default language. Why the name? I'm not sure, I just liked it. I've trained him 24 times so far, so he has 120 xp. He's actually ready to level up to level 2, since it only requires 100 xp to do so. Well, I guess as soon as I finish making the leveling-up algorithm, (which is the real meat of this whole process since it provides the "brains" for each species) he'll be good to go.

I look forward to working with Dörf; I hope he's a memorable first EvoSpeak species.

Saturday, June 13, 2009

EvoSpeak - Getting Closer

I finished the preview builder and now have a working random pattern generator and previewer for EvoSpeak. I still can't submit ratings so species don't gain experience yet, but the hardest work is done...until it comes time to build the "leveling" mechanism (i.e. the Markov analysis tool).

And the results of the initial grammar runs? Good! Overall, I am very satisfied with what I'm hearing. Based off of the twenty-or-so previews that I've listened to so far, the engine is much more interesting than GrammGen. It sounds a lot better.

The thing I really like, however, is that switching languages dramatically changes the previews. Of course the same was true for GrammGen, but I never built a second language for GrammGen because of the relative difficulty of editing the languages. In EvoSpeak there's a built-in language editor. It's as easy as slapping in some pipe-delimited numbers for rhythm and melody and listening to the results.

It took me thirty seconds to build a language that could be used for repetitive arps in the background. So I think I've found my solution for arpeggiation! The simple the language, the more likely it is to repeat words - which is exactly what you want in a background pattern. After listening to some previews of the new language, I'm certain that this will be a very promising and flexible system.

So far EvoSpeak is going very well! The real question, however, has yet to be answered: will the "experience" and analysis system actually allow EvoSpeak to improve the quality of its output? The answer would seem to be a very obvious yes if I do everything right. But at the same time, it's hard to believe that listening to samples and pressing buttons can train a program to make better music. But who knows, I guess I'll just have to find out.

PS - It's worth noting, in case I was never clear about this, that EvoSpeak is NOT a grammatical subdivision engine like GGrewve, rather, it's a grammatical chain engine like GrammGen. Chains are simpler and easier to work with but subdivision is more powerful. And yes, I coined both of those terms, which is why you won't find information on them anywhere else :)

Tuesday, June 9, 2009

EvoSpeak - Progress and Ideas

I'm still working on EvoSpeak, getting the engine all set up. I finished the random generating algorithms that will provide training material from which EvoSpeak will "learn." They also define the basis of the new grammar system, whose syntax is simpler even than that of GrammGen, but whose power is much greater.

Next I need to create the functions that will analyze the training material to figure out what attributes they have in terms of melody and rhythm. All of this analysis data will be stored in a training file that will also indicate how well the user likes the material. After a certain number of training pieces have been graded by the user, EvoSpeak will dig up all the analysis data and perform an extensive statistical analysis on it to try find correlations and develop a "brain," so to speak, that will allow the program to function as an output device.

I'm still trying to figure out exactly what variables/attributes should be part of the "brain." This has always been my problem with statistical models; I've never known exactly what variables to draw statistics from. Now I've got to tackle the issue. I'll start simple - state variables (such as what beat the rhythmic or melodic object falls on) and first-order memory variables (what the last rhythmic or melodic object was) should work fine for the first version.

I plan to have EvoSpeak set up in an intuitive "leveling" kind of way that reflects a simple game. Before EvoSpeak will work, the user must first create a new "creature" that speaks a certain "language." At first the creature will have no idea how to speak the language; like a child, the creature must be shown how to use words to make sentences. The user "trains" the creature by listening to samples and rating them on a scale of 1 (strong dislike) to 5 (strong like). The creature gains XP (experience points) when the user listens to samples and submits ratings. When the creature has enough XP, it can "level up." During the leveling-up process (unbeknownst to the user), the creature actually goes back and analyzes all of the samples and ratings and essentially "learns" from the previous batch of material. The leveling system is good because it will ensure that correlations are relatively strong before they will be used to generate (i.e. the creature won't work without the user having trained it to a certain level).

At higher levels, creatures may learn the ability to analyze deeper variables other than states and first-order memories. Perhaps the creature gains more memory with each level (this is equivalent to increasing the order of the Markov chain analysis). Or perhaps the creature starts analyzing surface contours (3-variable functions) instead of 2-dimensional dependencies.

These are pretty abstract and crazy ideas, but I think they make sense, and I think they will provide a refreshing and intuitive break from the usual grind of KBSs. I'm interested to start training my first creature! And if the leveling system actually makes the music sound better (as intended)...well...I think I could spend all day leveling my creatures (is this starting to sound like Pokemon? That's neither my intent nor my inspiration).

Monday, June 8, 2009

EvoSpeak

Yes, I know. Way too many new plugins lately. I can't help it, I have to find some new inspiration. The assisted compositions are falling into a rut and mGen needs some variety pretty badly. So I'm trying a new melody method, EvoSpeak.

EvoSpeak will be the first plugin featuring a true hybrid engine. It will incorporate an evolutionary system based on a grammar engine. The grammar engine will not be a derivative of WordSmith or the WordEngine, nor will it copy the format of GGrewve. This grammar engine will be based off of the lighter and more manageable GrammGen engine, with some obvious modifications for greater efficiency. I've basically developed two grammar systems and I have yet to hit the sweet spot for melodies. GrammGen introduced a very simple and very light-weight grammar system based off of loosely-defined words. The results were interesting but have failed to keep my attention for very long. The GGrewve engine brought with it a much deeper and extremely effective grammar engine. The GGrewve (and, subsequently, WordSmith) engine, however, is not altogether flexible. It basically requires MIDI files to create styles since the words are much more complex than those of GrammGen.

With the EvoSpeak engine, I hope to achieve the impressive coherence and variety of the GGrewve engine but with the efficiency and originality of the GrammGen system. The evolutionary model should allow a fundamentally simple grammar to come together into a much more complex system. Loosely speaking, the EvoSpeak engine is also a statistical model. The evolutionary model actually "evolves" via a statistical model, but "speaks" via a grammar.

Implementation is in alpha stages right now so I'm really not sure what to expect since this is my first real endeavor into evolutionary models (I tried a very, very basic evolutionary drum model as one of my first plugins ever, but it was way too simple and poorly-coded to achieve anything).

Wednesday, June 3, 2009

Slow

Work's going a little slow this week as I have a lot on my plate, especially starting my internship (32 hour week).  I'm brushing up WordSmith here and there, trying to improve the dictionary analysis quality so renders choose more appropriate words. What worked well for drums is taking quite a bit more messing with to work well for other instruments.

Unfortunately I'm having doubts concerning the viability of grammatical subdividion with non-percussive instruments.  While it may be possible, I think it's lacking in structure.  Perhaps there also needs to be a higher-order grammatical structure to guide motifs so that they do not lack direction.

The question is, then, how does one classify a higher-order grammar?  What symbols make up such a grammar?  Perhaps phrases could be classified as "suspenseful," "tense," or "declining."  Of course there could be loads of other adjectives.  The point is that sentences.  Sense do not make?  When words a sense of direction lack!  Yes, that was a bit of a punny example of how grammar can go wrong.

Rough days ahead for algorithmic composition.  Fasten your seatbelts and keep a pale ready for sea-sickness.  We will be experiencing some turbulent, nonsensical grammar.

Sunday, May 31, 2009

WordSmith - Generalized Grammar

Today I began construction of a more generalized grammar framework - very similar to the one upon which GGrewve runs, except extended to work for melodic instruments as well as percussive ones.  I began a crucial part of this rewriting today with WordSmith - the new equivalent to GGrewveWriter.

WordSmith is a generalized grammar/dictionary synthesizer capable of taking a "training file" in MIDI form and constructing a dictionary from the file.  WordSmith, unlike GGrewveWriter, actually has a very flexible interface and of course will work for any instrument, not just drums.  In addition to the usual features of the last grammar analysis tool, WordSmith will also include extended features such as time compression and maybe even synthesis tools like hybridization and mutation for more random possibilities.

As a minor under-the-hood tweak, WordSmith will perform extensive quantitative abstract analysis on individual words (categorizing intensity and such) so that plugins like GGrewve don't have to waste valuable processing time doing analysis that can be done once up front instead of once every run.  This will also greatly increase the possibilites for such analysis, since a one-time processing cost will allow for much more extensive and in-depth analysis of the grammar. Users won't mind having to wait half a minute to create a dictionary, as long as it doesn't take the same amount of time every time the dictionary is used by a plugin.  Which it won't.  It'll only take milliseconds for the plugins to access that analysis data that will have already been created by WordSmith.

Grammatical subdivision is going full speed ahead, and I hope this promising method won't let me down.

Saturday, May 30, 2009

airframe

I'm working on a new structure module. Easy Contoured Structure worked fine for a while, but I've definitely outgrown it's capabilities and it's time for a completely new system. This new system will be called airframe, a name that embodies the simple, light, and flexible system that is to come.  I have yet to decide the actual algorithm type for airframe, at the moment I'm thinking a combination of KBS and grammar.

More to come on this.

Monday, May 25, 2009

Swing Thing

It's another landmark day for mGen, as today was the first time a post-process module was ever used in the composition process. I made a very simple swing module called Swing Thing that offsets certain notes in the composition to give it a swing feel ranging from "very light swing" to "heavy swing."

The test run sounded great after I got the math down for the swing algorithm, which took a little trial-and-error. I was very impressed with the run time; mGen was showing the swing module as taking only 6% of the total composition time, even though the module had to loop through every single generative part and modify the timing of every single note that fell on a certain beat, which seems like a processor-intensive task. Nonetheless, Swing Thing chewed through the loops in no time.

A swing module was really the only immediate implementation that I could think of for post-processing modules, but it's already proven the power of post-processing to transform the entire composition. I'm sure there will be many more plugins to come.

Friday, May 22, 2009

Main Interface Upgrades

A good amount of progress was made on the main interface today.  First of all, I fixed the control drawing/resizing bug once and for all!  It's very exciting because it was glaring flaw that I couldn't do anything about and it made mGen look a lot less professional because controls would draw incorrectly or in the wrong place.  All I had to do was change all the Gui, Move calls to Gui, MoveDraw! And voila!  The interface stays intact now regardless of how vigorously the user resizes it.  Beautiful.

I also filled the second modular panel of the main interface today with a "Run Time Breakdown," which basically keeps track of where the time is being spent during the generating process.  It displays an entry for the "render  time," which includes sorting the note data and converting to MIDI, as well as entries for the structure, progression, generative, and post-processing modules (coordination module yet to come).  So after mGen finishes a composition, it will show exactly what percent of the time it spent on each area, and color codes the entries appropriately so that resource hogs will show in less-friendly colors.  GGrewve, being the most cpu intensive plugin, takes up about 70% of the runtime in my current projects.

It's nice to be able to see where the time is being spent.  It's also nice to not have controls freaking out on my every time I want to maximize the window.

Thursday, May 21, 2009

GGrewve Fill Dictionaries

Today was yet another huge leap forward for GGrewve.  I coded a second dictionary loading into the program so that it now keeps two dictionaries in memory: one for the main style, and one for fills.  Basically, a fill dictionary looks and functions exactly like a normal dictionary.  I wrote a small MIDI file that contained several nice fills and then passed it through GGrewveWriter, yielding a fill dictionary.  Along the way I had to do a lot of bugfixing on GGrewveWriter, which threw a fit because the fill patterns were a bit different from normal patterns.

At any rate, after everything was said and done, GGrewve was intelligently using a fill dictionary to make entrances as well as highlight fourth-measures.  The result?  Awesome.  Simply awesome.  It sounds great - it sounds real.  Even more impressive, GGrewve displays great inventiveness in using the fills.  As I explained the process of grammatical subdivision before, GGrewve basically chops up the MIDI files I give it and recombines them using intelligent analysis.  Not only were many of the fills very inventive (I wouldn't have thought to make them), they were coherent and well-placed!  I wasn't just hearing a repeat of the file I had fed into GGrewveWriter.  I was hearing an elaboration on those files - the same style, but not the same pattern.  GGrewve was not mimicking my speech, it was mimicking my ideas with its own speech (to use a linguistic metaphor).

GGrewve is by far my most impressive plugin so far, and with a solid drum plugin to depend on, I know it can only get better from here.

Tuesday, May 19, 2009

Post-Processing Functionality

I began work today on implementing post-processing modules. The mGen now loads and keeps track of an unlimited number of post-processing modules and also stores and retains the configuration data, just like generative modules.

The only part left to do is handle the actual generation, in which mGen will have to pass the entire compositional data structure to the post-processing modules and update the structure based on the output of the modules.

Unlike the generative modules, which have permission to view the entire data structure, but can only to add note data to their respective channels, post-processing modules will be granted complete read and write access to the entire set of data to maximize the flexibility of these special modules.

Structural Updates & Touch-Ups

Several fixes and minor changes are going on in the mainframe. Loading and saving have each been completely reworked to use the GDS data system instead of INI file storage. The result is a smaller file that is easier to work with. The plugin loading algorithm has also been tweaked to no longer function on absolute paths but rather on relative ones. Now I won't have problems loading the project files I saved on my laptop into my desktop's mGen (previously the project files were not transferrable because of the differences in the file path names on my two systems, even though the differences are not at all relevent to mGen).

Monday, May 18, 2009

GGrewve Intelligent Decision Making

Major progress has been made with the GGrewve drum plugin. It now has the ability to determine which words are more intense based on an extremely scalable and versatile analysis engine. Using this knowledge, GGrewve is able to place words more appropriately than before. Without even needing to look at the context of the words in the original MIDI file from which they were extracted, GGrewve uses the words to simulate the way a real drummer plays. Accents and fills will occur more during even measures, with a greater weight every fourth measure. In this way, GGrewve allows listeners to more effectively keep track of the measure, which gives the composition a much greater overall coherency.

The intelligent GGrewve engine is a step forward in autonomous music generation!

Tuesday, May 12, 2009

GGrewveWriter Analysis Tool

The development of GGrewve is going well, but rather slowly. This is partly because of the difficulty in expanding the dictionary. This is not an unfamiliar problem: I ran into the same issues with Variating Drummer. It's generally difficult to write drum patterns in anything short of a full-blown sequencer. Since writing a sequencer is beyond my capabilities, I have developed a pretty cool alternative.

Enter GGrewveWriter, the intelligent drum pattern analysis tool that disects a drum beat and packages it up for use with GGrewve.

In a nutshell, GGrewveWriter takes a MIDI file that contains a drum beat, splits the beat up into sections of an appropriate length, and derives "words" from these small sections. GGrewveWriter pays close attention to the placement of words within the original beat and the subtleties of wordly interactions. GGrewveWriter then creates a compressed dictionary file that contains all the grammatical data about the drum beat found in the MIDI file. This compressed dictionary can be read directly by GGrewve, and is actually more efficient than the uncompressed dictionaries used previously.

The whole thing is very impressive in practice.

It took me no more than a few minutes to write eight measures of a neat little drum beat in a sequencer and export it to a MIDI file. After exporting, I let GGrewveWriter go to work with the file. After only a few seconds, I had a fresh compressed dictionary sitting in the folder. I loaded the dictionary into GGrewve and let the generation begin. Sure enough, out came forty measures (I had requested this much) of well-placed, humanized drum beat. I had actually made the original MIDI beat sound very stale (I did not vary the volume levels at ALL, so it sounded very robotic) to see how well the humanization routines within Grewve were working. Sure enough, the computer's output sounded a heck of a lot more human than the beat that I, a human, had created. The output was varied and clearly derived from my beat, but not exactly a copy and pleasantly unpredecatable.

Writing the beat, creating the dictionary, and generating took about seven minutes. Had I tried to achieve the same thing with the Variating Drummer, it would have taken hours and hours and the final result would not have been as impressive.

This is a very successful test for GGrewve, and I look forward to more impressive results in the future (we may finally be ready for drum fills!)

Monday, May 11, 2009

GDS Inspector

A few months ago, I developed my own custom, flexible data structure designed to handle the challenges presented by the project in terms of data storage and transfer. I refer to these structures as GDS, or generic data structures. I have been improving on the capabilities of GDS since I first created them. Among the many important features of the GDS is it's ability to store other another GDS within itself. A GDS may, this way, be nested up to ten times within another. In other words, you can have a data structure within a data structure within a data structure within a data structure, and so on, which is extremely handy for keeping data organized yet easy to transfer and work with at the same time. I can basically store an entire heirarchicaly organized pool of data in a single variable.

These structures, however, can get quite massive (especially the main structure that mGen uses to pass between modules - this structure usually has hundreds of sections and up to four nested levels).

Today I created a small tool called the "GDS Inspector" to facilitate the process of examining GDS blocks when debugging. The problem arose while working on GGrewv that I needed to inspect the dictionary block because I was having problems. Opening the block in notepad, however, was very messy because of the way GDS stores data (which is flexible and efficient, but not notepad-friendly). The program I wrote display all the data in a given GDS block in a manner much more conducive to analysis - a treeview. A treeview is basically a hiearchy. One can see the nested levels of GDS storage and click on each entry to view the data held therein. This greatly facilitated the debugging process, not to mention I had a great deal of fun looking at how complex the main data structure actually is (notepad doesn't really do it justice).

So that's a nice little improvement in my working methods.

Sunday, May 3, 2009

GGrewve

GGrewve generated output for the first time today. The dictionary right now is very small and the subdivision rules are very strict, so there's really no variety in the drumming yet, but that was intentional so that I could test the engine. Everything worked very nicely.

One of the cooler parts of the engine is the "humanization" function that varies the velocities to simulate a "groove" as well as add ghost notes to the snare drum like professional drummers do automatically. Even with a tiny dictionary (right now it has only five simple words that correspond to four-beat patterns) the drumming quality already surpasses the Variating Drummer, so I think I'm on the right track with this plugin.

I've still got obstacles to conquer. Most notably, I have to devise a way for GGrewve to determine which words are "fancier" and "heavier," so that it knows in what context to use the words. For example, a fill should obviously not come on the first beat of the first measure of a section...it should be saved for a turning point or a transition into another section. I could just include some quantitative specifications in the word files to let GGrewve know where they should be used, but I think that's taking too much of the computer work out of it. I'll write the words but GGrewve has to figure out how to use them. This will, of course, be a lot more work that just telling it when to use words, but I think the results will be more satisfying. Expanding the dictionary will also be a lot easier if I don't have to take the time to think about when each word should be used.

In conclusion, I'm excited about where the drumming side of mGen is going and I think GGrewve has a bright future.

Friday, May 1, 2009

GGrewve

I'm still in the process of implementing the grammatical subdivision idea in a drum plugin. So far everything is going smooth and it's shaping up to be a really nice module! It's called GGrewve (grammatical groove).

I hope to have some simple output in a few days. The idea is a LOT easier to work with than that of the Variating Drummer, which has become so convoluted that even I have trouble figuring out how to build a drum pattern (which shouldn't happen, considering the computer is supposed to be the one doing the work).

At any rate, more to come on grammatical subdivision and GGrewve in the near future.

Wednesday, April 29, 2009

More on Grammatical Subdivision

Here's a rough diagram of the process with the entities shown on the left and the process shown on the right.

Grammatical Subdivision

I had somewhat of a small breakthrough several minutes ago. Grammatical percussion is really all about subdividing time. It's about taking a given chunk of time and doing something with it. Unlike the melodic grammatical system, the percussive system should have "words" that correspond to different ways to subdivide a set amount of time. This way, one does not have to deal with overlapping words and other problems that arise when the length of a word isn't absolutely defined.

The process of subdivision is recursive, so the process might start at the level of a measure and ask, "ok, how do I want to subdivide this measure?" The response may be, "I want to equal subdivisions." The process will then make another decision: "Do I want to go ahead and replace this chunk of time (half a measure) with an 8-beat word? Or do I want to subdivide it further?" Let's say the computer decides to split the first chunk of time into two quarter notes (4 beats, also 1/4 of a measure). It decides to replace the second chunk with a word.

So the system really operates around two entities and one process: time "chunks ," words, and subdivision, respectively.

Conceptually this is a big step forward for grammatical drumming. With any luck it will make the process a lot less painful.

Tuesday, April 28, 2009

Representing Drums With Grammar

I'm still trying to resolve the grammatical method to work with drums and other percussive instruments. The problem is that, while melodic instruments only require notes (and details such as velocity, panning, etc.), drums require notes that correspond to different drums. Thus, the notes are not related in the same way that melodic notes are related in. Drums have no "key" and do not play contoured riffs.

Representing drum patterns within a grammar will require a more flexible grammar that allows for multiple notes on the same beat. I'm thinking of using a very low-level grammar to describe individual hit patterns on the different drums, then combining these in higher-level grammars that describe riffs, fills, and grooves that finally combine into the highest-level grammar that describes an entire style of playing.

Monday, April 27, 2009

Module Instructions

After the recent data structure overhaul, adding new features to mGen feels like a breeze, which is a nice treat. Today I wrote a basic module instruction mechanism that allows the structure module to work with the generative modules to coordinate the composition at a higher level. This is, of course, essential to the coherence of the composition and is a feature that will require a lot of refining if I hope to get good material out of mGen.

Basically, the structure module can now coordinate, for instance, when the piano should come in, when the drums should make an entrance, when things should get softer, and when things should get heavier. There's now an overlaying set of general module instructions to help the generative modules achieve a greater coherence.

Also, the data structure now allows other modules to effectively "see" each other even before they have generated any output. This was necessary at first because the structure module needed to be able to see the generative modules before it could start giving part instructions...you can't rely on a nonexistent pianist to start a song, nor a nonexistent drummer to get fancy with a solo! As a consequence, modules can also now see each other. Conceivably, this could be used for a dynamic interactivity between them. Although there is no data structure in place yet to allow modules to communicate between each other, that may be a feature in the future. This could allow, for example, the drum and the bass modules to "establish a groove" before they start generative the composition. Communication is essential in a real band, so it should be essential in mGen as well.

Lots of progress is being made these past few days. mGen's compositions are taking less and less of my intervention to sound good. I usually just plop in a synth doing some rhythm work or harmony and then lay down a drum groove. Throw in some nice mixer effects and it all sounds pretty darn impressive, or at least I think. Soon enough mGen will be doing all of that autonomously. It's a scary thought. But it's the future of music.

Thursday, April 23, 2009

Complete Internal Overhaul

Tonight was a long night for mGen. At the begin of the night, the poor thing was told it had an obsolete internal data structure that lacked much structure to speak of. Furthermore, mGen was accused of inefficient data handling that was making other modules work harder than necessary.

After an immense surgery that lasted about four hours, mGen is now smiling with a brand-new, sparkly internal data structure. The structure now conforms to the data structures used by other modules and makes it much easier for all other components of the program to access information about the composition on the fly. The surgery has, however, rendered mGen uncompatible with most of the previous plugins I wrote to accompany it. They too much schedule an appointment for surgery to become capable of taking advantage of mGen's new data structure.

In short, I rebuilt the internals of the mGen framework from scratch. It's an investment in the future, where data handling will be done much more efficiently by the program. It's really quite a huge change/improvement, as reflected by the half a thousand line increase in code length.

Wednesday, April 22, 2009

Tweaking the GUI

I made the GUI look a little prettier today. I am still withholding screenshots from the blog, however, because I do not want it to be seen until it is ready.

I am also working on filling up the main panels of the program, since the border regions are populated with functional tools, but the center area is completely blank. I'm not sure what to put there.

Monday, April 20, 2009

Grammar: Success?

Well, after coming back from a long break, I have some good progress to report.

I tried implementing a grammatical system for algorithmic composition over the break and had a good deal of success in my endeavors. Although I created only a rudimentary composition language and a very basic phrase generator, the results seem more natural and interesting than any other method explored thus far, which means progress!

After a great deal of thinking on the subject, I've decided that a grammatical system might be the key I've been looking for to a successful path to my goals. The trick is that I can use a grammatical system as the underlying paradigm for other methods. In other words, I could have an evolutionary model that uses an underlying higher-level grammar to generate phrases. In theory, this idea is really no different than using an evolutionary model to generate an equally-abstract number that corresponds to a certain pitch (a MIDI note event). I could do the same thing with Markov chains and state-transition matrices.

There are a lot of places to go with grammatical algorithmic composition. I have a feeling I'm just scraping the surface of something big. Let's hope I'm not let down.

Thursday, April 9, 2009

Tuesday, April 7, 2009

Rhythm and Meter

As the foundations upon which music is built, rhythm and meter will play an obvious and pivotal role in my program. Unfortunately, I have read very little on the topics, as Music, the Brain, and Ecstasy devoted only a single chapter to the subject in general. I need to delve deeper into the topic. To do so, I'll need some good sources.

Here are some books I'm looking at:
The first one looks extremely comprehensive and helpful.

Monday, April 6, 2009

The Musicality of Language

The Musicality of Language

A very interesting article that touches on some rhythmic similarities between language and music.

Sunday, April 5, 2009

Sweet Success.

As dramatic as it may seem, I was almost moved to tears earlier tonight when mGen generated something completely unexpected. I can't really say that what I heard was entirely original or breathtaking, but I wasn't expecting it. I had pretty much given up on the Conscious Model plugin and was getting ready to move back to ChillZone and try something else. I had also thrown together a new progression plugin based off of hard-coded progressions since I was tired of hearing crappy progressions. And then I hit generate with a ChillZone pianist plugin, a Conscious Model plugin, and a Contour Arp plugin, just for old time's sake.

What I heard was a gentle ambient pad laying down two simple seventh chords. A very pleasing background. A low arpeggiated drone created an eerie feeling. But the best part was up top...the Conscious Model plugin had generated an indescribably simple but beautiful high melody. The most unexpected part, however, was how the melody remained coherent throughout the whole piece - returning to certain motifs and elaborating on them - while changing and undergoing subtle variations. The melody was perfectly predictable and perfectly unpredictable. It made sense but I couldn't say for sure what would come next.

I'll post the new sample clip in a day or two. It's probably nothing amazing to anyone else. But I think it's the first time I've really been taken aback by the creative ability of the program. I didn't know I had programmed a plugin capable of doing anything coherent to this point. And yet there it was; here it is.

I feel accomplished. If mGen fails in terms of everyone else's standards, if the generated music makes the ears of others bleed, if people say I failed and computers will never know anything about music, at least I have this. I could listen to this kind of output for hours on end. I could sleep to this, I could dream to this, I could do homework to this. It doesn't even matter anymore. I succeeded. I would listen to this music. That's all I cared about in the first place.

I succeeded.

Saturday, April 4, 2009

Computer Models of Musical Creativity

Computer Models of Musical Creativity
Chapter 4: Recombinance

  • Western tonal music generally follows simple principles that drive melody, harmony, voice leading, and hierarchical form
  • One can create music by programming such principles into a computer
  • Such an approach often creates stale music
  • Recombinance is a method of using existing music and recombining it logically to create new music
  • Cope uses destination pitches and beat-size groupings to split chorales into smaller groups called lexicons that can be recombined using the pitch and beat data
  • Such syntactic networking actually preserves a great deal of the music's integrity while generating new output
  • To further extend the abilities of recombinance, Cope had his program analyze the source piece's "distance to cadence, position of groupings in relation to meter, and other context-sensitive features"
  • Artists often use musical signatures, patterns of notes that recur in many works of a composer
  • Recombinance can be described in terms of Markov chains
  • Recombinance can work both vertically and horizontally
  • Generation of music must start with an abstract hierarchy and move towards specifics (this is exactly what I foresaw and intended when I made the structure module the foundation upon which mGen works! Cope agrees!)
  • Rule acquisition from music models the musical training of humans
  • Machine renditions of music are often crude and dead...successful algorithmic composition requires dynamics
  • An improviser basically has a repertory and an idea of how he or she wants an improvised idea to flow into the next
  • "Recombinance, or rules acquisition, provides more logical and successful approaches to composing in tonal music styles"
  • "Every work of music, I feel, contains a set of instructions for creating different but highly related replications of itself"
  • "The secret of successful creativity lies not in the invention of new alphabet letters or musical pitches, but in the elegance of the combination and recombination of existing letters and pitches"
  • "In recombination, rules are not necessary, since the destination notes provide all of the requisite information"
  • "While recombinance of this type ensures beat-to-beat logic in new compositions, it does not guarantee the same logic at higher levels"
  • "The initial and final groupings of a phrase are most pivotal"
  • "Experiments in Musical Intelligence protects signatures from being fragmented into smaller groupings, thus ensuring that these signatures will survive the recombination process"
  • "A Markovian description of recombinant processes does not allow for the broader control of larger-scale structure"
  • "In music, what happens in measure 5 may directly influence what happens in measure 55, without necessarily affecting any of the intervening measures"
  • "The top-down approach is necessary because choosing new beat-to-beat groupings must be informed by hierarchy, and not the reverse. No new grouping of a work-in-progress can be selected until its implications for the entire structure of the work are determined"
  • "Acquired rules are often more accurate since, by default, they originate from the music itself and not from generalizations about the music"
  • "Having a program first derive rules and then apply these rules during composition, though a simple notion, is critically important to the basic thrust of my modeling creativity"
  • "I continue to maintain that computer-composed music in any style is as real as human-composed music in any style"
  • "I see no reason why computer-created music cannot move us to tears, find roots in our cultures, and reveal or obscure its internal implications as much as any music composed in more traditional ways"
  • "Improvisation consists of either generating music associatively to maintain continuity, or interruptively striking out in apparently new directions"
  • "Improvisers associate rhythmic patterns, melodic contours, and harmony"
  • "Improvisation tends to function as a series of gestures that themselves have a sense of beat and that, when performed one after another, make musical, rhythmic, and metric sense"
These constitute the notes I took on my reading today.

Thursday, April 2, 2009

Coordination Module

As I have continued to work on generative plugins and attempted to reverse-engineer familiar music, I have found something lacking in my Core Modules group. I need more than just a structure and progression module - I need a meter/rhythmic module. Such a module would generate information concerning where the strongest beats of each measure lie, how melody should interact with harmony, - with room for syncopation and counterpoint - and more for each segment of a composition.

Although such a module wouldn't make much difference in the case of a standard 4/4 beat where ,beats 1, 5, 9, and 13 are heavily accented, it would shine through when unusual rhythmic and metric patterns occured. Almost any metric pattern can sound good if used consistently and coherently. This new module would allow the drum plugin to make sure it hits the accented beats harder - and choose its base beat accordingly, as well as enable the melody to follow a predictable metric pattern.

In short, I need a coordination module.

Now the real question: what rules govern the metric and rhythmic patterns of compositions?

cgMusic

I just have one question - how is cgMusic able to create coherency? I need to dig deeper into the scripts to find out how it works. I should learn from my predecessors.

Wednesday, April 1, 2009

ChillZone Pianist

One of the first plugins I ever wrote for mGen was the ChillZone Pianist, designed to be a piano module for down tempo applications like Buddha Bar, Hotel Costes, and Cafe del Mar. The ChillZone Pianist uses an interesting data structure for generation: hands. The pianist essentially has two "hands," each consisting of five fingers. The fingers can move around different keys, and the plugin generates output by invoking functions built into the hand data structures that 'play' the fingers in a desired order and with a desired time map. Using this method, I hope to recreate more realistic patterns that a real pianist would play.

After dropping the ChillZone plugins for quite a while in favor of certain others, I have come back to the pianist. I experimented with other models for generation such as my consciousness model theory, but ultimatelely came back to the original thought that mimicing a pianist could produce solid results. Thus, I am now back at work on the ChillZone pianist and hope to complete all the features, such as rhythmic chords and abstract motifs.

Monday, March 30, 2009

Computer Models of Musical Creativity

Computer Models of Musical Creativity
Chapter 3: Current Models of Musical Creativity
  • Although randomness often competes with creativity in terms of surprise, it is no substitute for a creative process
  • Most random processes are simply too complex to predict
  • Randomness arises from a lack of predictability using logic, not a lack of determinism
  • Good creativity simply requires good algorithms
  • Some models of creativity include cellular automata, mathematical models, fuzzy logic, neural networks, and Markov chains
  • Using Markov chains, one can analyze a piece of music and produce new music in roughly the same style
  • Genetic algorithms operate on the principle of natural selection
  • Genetic algorithms and cellular automata can both generate very complex output
  • In rule-based programs, creativity really belongs to the programmer, not the program
  • Neural networks use "hidden unit" networks to simulate the output of a given situation based on a sample input and output
  • Neural networks simulate the workings of the human brain
  • Mathematical formulas can be used to produce quasi randomness

  • "Randomness is not an engaging mystery, but a simple reflection of ignorance"
  • "Randomness refers to behavior that is either too complex, too patternless, or too irrelevant to make prediction possible"
  • "For those believing that using algorithms to create music somehow removes imagination, inspiration, and intuition from the composing process, know that defining a good algorithm requires as much imagination, inspiration, and intuition as does composing a good melody or harmony"
  • "Neither good algorithms nor good musical ideas grow on trees"
  • "Integrating association-based procedures with data-driven processes increases the creative potential of this approach to music composition"
  • "GAs typically involve DNA-like inheritance of characteristics as well as crossover and mutation techniques to develop new traits"
  • "Neural networks then cycle through a series of forward or back propagations that compare output with input and alter hidden unit values, until the output values match or approximate the relationships of the input and output data upon which they were trained"
  • "Sandwiched between these nodes are variable numbers of layers of hidden units, as well as variable numbers of connections between these inputs, outputs, and hidden units, making the training process extremely complex"
  • "We should not overestimate the abilities of neural networks or let comtivity mask a lack of true creativity"
  • "Mathematical origins for algorithmic music, while occasionally producing interesting results, in no way indicate the presence of creativity"
  • "Computer programs must be sufficiently independent of their programmers and users in order to qualify as truly creative. Most apparently creative algorithmic composing programs either produce enormous output from which users make preferential choices or invoke so many programmer-defined rules that the software only proves the creativity of the programmer"
These constitute the notes I took on my reading today.

Thursday, March 26, 2009

Bland Statistics

Unfortunately, the analysis tool used to create a permutation of Super Mario has ultimately led me to a dead-end. While such analytic methods might help for basic applications such as chord progressions, they clearly don't work as well when applied to individual notes, since duration, velocity, and padding must also be taken into account.

If I had the time to develop a more comprehensive analysis tool that used more than just a first-order transitional Markov matrix, I might extract better results from this analytic method. I feel, however, that this would simply take too long. Such intense analysis would merit an entire thesis all of its own.

Tuesday, March 24, 2009

Super Mario Statistics

Today I created a program to statistically analyze midi files and use the collected data to create a new, random 'permutation' of the file using the same statistics. I used the super mario brothers game theme song as my source midi file and the result was surprisingly interesting. Considering the simplicity of the statistics collected, I think the results demonstrate the feasability of a stylistic permutation tool as discussed in a previous post.

Wednesday, March 18, 2009

Shifted, Narrower Focus

I need to stop fumbling around with stuff for which I don't have a really solid goal. I need to start thinking in terms of concrete genres and concrete goals instead of trying to construct some kind of amorphous compisition from myriad modules of different production methods. I said in the beginning I wanted coherence. I've diverged a bit too much.

I'm thinking simple rock. The majority of voters in my music pole said they preferred rock. I like rock too. Plus, rock is not that hard of a genre to replicate in my opinion. Now I need to start looking at actual songs and thinking about how I would reconstruct them algorithmically, then generalize the processes.

Tuesday, March 17, 2009

Three Lines

Three Lines
Will define
An ardent mind's
Expense of time

They bring about
No change at all
But in and out

They are three lines.

(This poem is about the project's current code count: 9997, three lines short of a huge landmark)

Stuck

For some reason I'm just not feeling much inspiration lately. I've been pushing into the conscious model and it's a lot of work and, while the results are cool, I'm ready to hear something really inspiring. At almost ten thousand lines of code, the program is huge and I feel like I'm not getting my money's worth, so to speak. I want to hear something awesome. Guess I'll just have to keep going.

Wednesday, March 11, 2009

Consciousness Model Ideas

In order to create a sense of flow and retained motifs yet at the same time stylistic diversity, a generative module could at all times retain several streams of 'subconscious ideas' that represent viable solutions to the composition. Each idea uses a different set of parameter values to create an independent behavior for each idea. In this sense, each idea possesses its own personality of sorts. The module's 'conscious' thought starts on any given idea stream. The conscious thought represents the output of the module.

At any given time, however, the module may switch to another idea stream. In this way, the style changes yet the other styles (and motifs) are preserved, because the module continues to process them 'unconsciously.' Effective stream transitions could be made when two streams 'intersect,' so to speak, at a given point. In other words, if the contents of stream A at a time t are similar to stream B at the same time, consciousness may effectively change from stream A to stream B smoothly. The idea stream will then continue off in another direction, until it intersects with another stream at another point.

Here's a diagram of these ideas:
As the diagram indicates, this module has three streams in its subconscious. The streams are being processed independently (thus they have individual characteristic behaviors) but in parallel. The box of consciousness starts on a given idea stream and uses the stream to determine the module's output. The box can switch, however, to a new stream based on switch conditions (such as instantaneous stream similarity).

I think that this idea represents a step forward in both coherence and creativity. I'll try to implement it soon so I can see how the theory plays out in practice.

Computer Models of Musical Creativity

Computer Models of Musical Creativity
Chapter 2: Background
  • Surprise is a necessary ingredient in creativity
  • Associationism - a view that creativity stems of unconscious or subconscious associations - linking thoughts, ideas, experiences, and images
  • Letter Spirit, a program used to creatively make new fonts, operates with decision waves - unifying operations that consist of "the gradual, slow-but-sure tightening up of internal consistency all across the structure under construction"
  • Cope feels that focusing on details is not the important part of a creative work
  • The Turing test may also be an adequate gauge of creativity
  • Creativity is not an output, rather, it is a function
  • Creativity is not necessarily audible - it is only perceptible when the method of creation is known
  • "Most recent research in artificial intelligence and creativity tends to focus on connectionism, geneticism, and expertism"
  • "Full-scale creativity consists in having a keen sense for what is interesting, following it recursively, applying it at the meta-level, and modifying it accordingly" - Hofstadter
  • "I do not believe ... that all humans share a common aesthetic"
  • "It is the processes, rather than their output, that best identify creativity"
  • "The results of creativity should be different from the results of other processes"
  • "Creativity is a process, not the result of a process"
These constitute the notes I took on my reading today.

Tuesday, March 10, 2009

Computer Models of Musical Creativity

Computer Models of Musical Creativity
Chapter 1: Definitions
  • Many see creativity as restricted to humans
  • If humans can't create creative machines, then are they really creative in the first place?
  • Two important questions when considering creativity: must the creator know what he or she is creating and must they appreciate his or her own creation?
  • Creativity is directly related to intelligence
  • Intelligence consists of the following abilities: to learn, to remember, to infer, to analogize, and to create
  • Combinatorial accidents can be powerful creative tools and also produce striking originality
  • "Originality must be useful"
  • "Cross-wiring can produce interesting, unique, and important creative connections"
  • "Vertically thinking is selective and analytical, while lateral thinking is generative and instigative"
  • Cope's definition of creativity: "The initialization of connections between two or more multifaceted things, ideas, or phenomena hitherto not otherwise considered actively connected"
  • "I do not feel that creativity requires intelligence, nor does it consequently require life"
  • "Creativity should never be confused with arbitrary or convenient contrivances that simply take any road untried for the sake of novelty"
These constitute the notes I took on my reading today.

Monday, March 9, 2009

mGen Sample 2

Believe it or not, the program has undergone many changes since the last sample. I created a brand new structure plugin based around the idea of a smooth, contoured intensity structure. I also injected a few lines into most of the other plugins to give them a primitive obey-the-structure rule. Most of the instruments now actually pay attention when the structure module tells them to play loud versus when to play soft. Some, however, still don't have this capability, and most lack any real sense of volume other than the most basic loud and soft abilities.

Still, these improvements constitute serious advancements in mGen. I find this sample to be moderately pleasing. Although it gets repetitive in some places, I'm very pleased with the dynamics. I'm aware that they are still laughably lacking, but at least there ARE dynamics in this sample, unlike the last.

And yes, I know that it sounds very similar. That's due to the lack of rendering options that I have given the program to this point. I'm more concerned with the scoring than the rendering at the moment, but I'll get around to expanding the instrument catalog eventually.

Oh, and a little side note - the drum "solo" at the end was actually a very unintentional and very strange glitch. I've no idea why the drums played a whole extra measure...I'm still trying to figure it out. Three months in and the program already has a mind of its own - I guess I should be proud and worried at the same time!

Sunday, March 8, 2009

Music, the Brain, and Ecstasy

Music, the Brain, and Ecstasy
Chapter 6: Composition
  • Imagery describes what perception cannot
  • The brain always works through abstract hierarchies, connecting ideas with a web of relations rather than storing a single data bank for a specific subject
  • Composers' brains group notes, chords, and progressions with such relations
  • [My own deduction] "Categorization circuitry" is essentially what is needed to build a computer capable of composition
  • A soloist draws upon a vocabulary of sounds a phrases that comprise a musical language
  • In terms of improvisation, a brain can't generate complex, far-reaching structures enough to improvise a piece of significant depth [but perhaps a computer can?]
  • Mozart mapped out the structure of his compositions before filling in the details (this is great news, because it's how my program is designed! So at least I know this method is workable!)
  • Scores can limit the capacity of composers (this again is good news: my program is not bounded by traditional scoring)
  • Intelligence comes from neural networks
  • "It is memory that is the composer's workshop"
  • "The phenomenon of musical ideas arriving full-blown in the composer's mind is called inspiration"
  • "[The soloist] works from a vocabulary of sounds, and a kind of grammar for juxtaposing them, that has become his musical language"
  • "Musicians work through a hierarchy of ready-made movements. Thousands of patterns of scales and arpeggios and chord progressions are deeply channeled in their nervous system"
  • "[Mozart] began by mapping a composition's structure. Only later did he go back to fill in supporting voices and embellishments that could be written in any number of ways without changing the basic character of the piece"
  • "Music notation tends to discourage complex melodies and rhythms"
  • "By promoting abstracted, hierarchical thinking, the score can seduce composers into a theoretical, unmusical approach to composition"
  • "In the end, intellectual power of any kind arises from the laborious creation of networks of neurons"
These constitute the notes I took on my reading today.

Thursday, March 5, 2009

Ideas on the Quantification of Style

Today I worked on developing a structural outline for a system capable of analyzing and quantifying the essence of musical "style." The analysis will consist primarily of Markov chains with criteria automatically developed and analyzed by the program in the style of a nodal or neural network. In this way, the program will learn autonomously what criteria best 'define' a style and thus learn to reproduce new music in this style.

Monday, March 2, 2009

mGen Sample 1

Well, here it is (the clip should be playing when you load the blog).

Eighty-nine hundred lines worth of code, embodied in a single audio clip. Yes, I know it's bad, probably pretty much intolerable by the standards of any of you guys. So am I wasting my time? Thousands of lines and all you get is an overly simplistic, predictable drum beat and uninspiring arpeggiations layered on top of overly simplistic chord progressions? No, not at all. Most of the coding so far has been dedicated to creating the framework for mGen, not the actual generative modules. The real work so far was getting a progression, getting arpeggiations, getting a drum beat, and pulling them all together into the same mp3 file and rendering them all automatically (everything in this sample was done automatically, I was literally one button click away from my rendered mp3 file).

I'd say I'm pretty much on target for my goal. For starters I've already achieved the one-click philosophy cited in my proposal. My system has now effectively demonstrated its ability to go from nothing to a finished mp3 file in only a single button click. Also, when one considers briefly the complexity of computer music, the fact that mGen's first sample sounds even remotely close to rhythmically and harmonically sound is quite an impressive feat. There are no chords that sound bad, most of the progressions actually sound good. The arpeggiations sound relatively good too. Both of those items were non-deterministic, meaning mGen completely determined each chord in the progression and each note in the arpeggios (actually it also individually determined the notes in each chord of the progression).

So, in defense of my somewhat-lacking first sample, I have clearly shown that what I am wanting to do is possible. I have not spend too much time coding the actual generative part of my program, so I didn't expect a diamond on my first try. I poured in a lot of effort, and I think I reaped a fair reward.

From here, onward to better-sounding samples.

Music, the Brain, and Ecstasy

Music, the Brain, and Ecstasy
Chapter 5: Rhythm
  • Meter vs. Phrasing
  • Phrasing is a higher-level unit 0f perception than meter, and encompasses harmonic tension, contour, and dynamics
  • Pulse lies at the core of meter
  • Perception of meter is based on prime numbers
  • Polyrhythms are made by playing more than one meter at a time
  • Syncopation is created when beats are accentuated apart from the regular metrical pattern; often the offbeats are regular enough to anticipate
  • Memory vs. Anticipation
  • Memory recalls what has already happened, anticipation draws on memory to predict notes to come (usually only a beat or two in the future)
  • Importance of tempo: if music moves too slowly, the relations are not close enough to be intelligible; if music moves too quickly, the brain cannot keep up with the relation modeling and has to move to shallower relations, missing the nuances of the piece
  • Music needs some gradual changes in tempo; sounds unnatural without them
  • Harmonic complexity vs. Metrical Complexity have an inverse relationship
  • Most listeners and composers alike now opt for harmonic complexity since harmony information is parallel, while metric information is serial, thus more harmony information can be modeled in a shorter time
  • "Memory is music's canvas"
  • "In music, it is phrasing that reaches farthest across time to encompass the deepest relations"
  • "Composers gain maximum effect by interweaving the tensions created by music's various aspects"
  • "Tempo matters because the mechanics of music perception are exceedingly sensitive to the rate at which musical structures are presented to the brain"
  • "Most tempo fluctuations are made intentionally. Music just doesn't sound right without them"
  • "The more harmony wanders from its tonal center, the more it requires rhythmic buttressing"
These constitute the notes I took on my reading today.

Notes from Meeting with Machover

Although I was unable to write down many exact quotes from Professor Machover (thoughts were rolling quite fast), I snatched the general idea of a few of his important thoughts:
  • "I think music is more than entertainment"
  • Music should be made to touch people
  • Melodies have order but they are diverse, probably the most important part of a piece (relatively speaking)
  • Nobody has figured out a way to design a composition system that can determine the difference between pretty good and really good
Tod also directed me to the following sources:

Sunday, March 1, 2009

Progress Report #7

A Quick Off-Topic Note
I spent the passed week in Boston visiting MIT (for those who aren't sure what that is, it's the Massachusetts Institute of Technology, widely known as the best engineering/mathematics university in the world, check it out at mit.edu). It was everything I expected and more...I'm simply in love with it. I met with many people there, including a professor of computer science and contributor to the field of cryptography working closely with the National Security Agency, a physics graduate student helping design a massive laser to detect and prove the existence of gravitational waves as postulated by Einstein's General Theory of Relativity, a famous composer and opera writer, and a programmer responsible for an innovative music composition tool. What a week. Now the only problem: 10% admissions rate. Ouch. Wish me luck.

Reflection
I actually did a lot this week. I took my laptop with me to Boston and worked on the program every night. I finished a progression module that works based on the interval observations I posted in my blog a few days ago. It turned out quite nicely, and I'm continuing to refine it.

By far the most productive thing I did this week, however, was meet with Tod Machover (or here), a well-known composer, opera writer, digital composition innovator, and just about everything else you can imagine. After getting over my initial feeling of awe when I met him, I managed to talk with him for a good while and told him about my project. He was genuinely interested and gave me some suggestions but overall seemed to think I was heading down a very promising path. He also introduced me to Peter Torpey, one of the programmers that helped make the Hyperscore software a reality. Peter also took a great interest in my project after I explained it in a fair amount of detail to him. He told me to keep in touch and let him know how the project goes. So I may have found some BETA-testers for my project! (That's a technical term for the people that test a software before it gets released) Peter also told me that a modularized structure (like the one I currently have built in my program) is definitely the way to go. He also said creative interfaces are important for making things intuitive to the public.

Goals
  1. Render a music sample
    Obviously this is a recurring goal for me. I have yet to achieve it, just because I don't want the first music sample to be embarrassingly bad. I know it won't be great, but I want it to at least sound somewhat like music. I think I'm getting pretty darn close to achieving this goal.

  2. Read more of Music, the Brain, and Ecstasy
    This is an important part of my research. I'm doing heavy note taking and annotating on this book.

  3. Organize everything into Noodlenotes
    In order to prepare for my research portfolio, I'll need to organize what I have into NoodeNotes. I should also start printing screen captures of the program's interface so that I can visually prove some of the work I've been doing. Ideally I could even have a sound clip as part of my portfolio.

  4. Expand the progression data structure
    This is a pretty specific goal. Right now the progression data structure stores everything in terms of which notes will play and how long they will play. I should at least separate the chord from the bass note so that I can have interesting splits between bass and treble.

  5. Add capabilities for post-processing
    I need to make the program capable of running post-processing modules after it finishes running generative modules. An example of a plugin that would fall under this category would be one that applies a swing groove to the piece, or perhaps various playing styles. This is done after the actual notes of the piece have been generated.

Thursday, February 26, 2009

Meeting with Machover

Well today was the day! My meeting with professor Machover went better than I ever could have dreamed. We talked for about forty minutes. He listened to my thesis ideas and project, pointed out some resources and told me I was headed in the right direction. He liked the design plans I have so far and shared some words of wisdom on music. Being single-handedly responsible for the composition program HyperScore, he was also able to discuss the more technical side of algorithmic composition.

After a lengthy conversation with Tod, he was kind enough to walk me over to the office of his research assistant for the Hyperscore project, who took a great interest in my program after talking for a short while. I was able to get contact information from both Machover and his assistant, both of whom expressed a great interest in keeping up with the project and hearing how my endeavors go. Perhaps I can have some professional beta-testers for the program!

I'll post more specific details of everything later. Today was all a little surreal. I love my project.

Monday, February 23, 2009

Questions for Tod Machover

These probably need to be refined and I need to think of a few more, preferably more over-arching questions so I can get the most out of my time with Machover. Here's what I've got so far, in order of importance:

  • Given your experience with assisted-composition tools such as Hyperscore, what are your thoughts on completely autonomous randomly-generated music?
  • How, in your opinion, should one approach simulating the human's creative ability to compose?
  • Do you think there would be enough demand for a generative mp3 player to warrant serious commercial consideration?


Also, if anyone reads this post before Thursday and would like the chance to have a music-related question answered by a famous composer, theorist, and musician, comment back and I'll consider asking it if I have any time left after I pose my questions. Robin, Jules, this is your chance to find out whether the tritone really is the devil or not!

Tod Machover

Tod Machover
  • World-renowned composer and music technology innovator
  • Professor of Music and Media at the MIT Media Lab
  • Head of the Hyperinstruments group at MIT
  • Designed custom instruments for renowned artists including Prince and Peter Gabriel
  • Contributed to the development of Guitar Hero
  • Composed a huge number of pieces and worked on several operas
And the best part? I will be getting fifteen to thirty minutes of his time in three days! I can't wait to talk to him and hear what he has to say about algorithmic composition. He's a BIG deal. He's won numerous awards and is on the cutting edge of digital music.

I'm psyched out of my mind right now.

Sunday, February 22, 2009

Interval Analysis

In order to make a better progression module that retains both originality and coherence, I need a method for evaluating certain qualities of progressions. One can break progressions into individual chords for easier analysis. Even better, one can break chords down into intervals for the most specific analysis. Intervals, which are more specific than individual notes, yet more general than chords, provide a good basis upon which to judge qualities of chords and progressions with fair accuracy.

I made up three criteria for intervals and evaluated every interval in the minor key based on the criteria (which have values between one a five, inclusive). Higher mood values indicate a 'happier' feel while low mood values indicate a 'somber' feel. High suspense values indicate a sense of tension in the interval, while low values designate chords with greater stability. Low consonance values indicate a clashing of harmonics, which high values indicate a relatively consonant interval.

Below is the table of intervals and quality values I wrote down. 'Bad' intervals, or those with a consonance value of 1 have been marked in red.

Interval: Mood, Suspense, Consonance
1-2: 1,5,2
1-3: 2,2,5
1-4: 2,3,4
1-5: 3,4,4
1-6: 2,4,4
1-7: 2,3,2

2-3: 1,4,1
2-4: 3,2,3
2-5: 2,3,3
2-6: 2,5,1
2-7: 4,2,4

3-4: 3,4,2
3-5: 4,2,4
3-6: 5,1,5
3-7: 5,2,4

4-5: 3,4,2
4-6: 3,3,3
4-7: 4,3,3

5-6: 3,4,1
5-7: 4,2,4

6-7: 4,4,1

Friday, February 20, 2009

Markov Drumming Ideas

I need to create a list of statistical attributes that can be analyzed and linked to form a statistical Markov profile for drum styles. I also need to establish how attributes will be stored. Here are some ideas:

Format for statistical attribute: Node-(Trigger:Boolean Operator:Trigger...)=[Correlation],[Strength]

Examples
Snare-(Beat5:AND:CHatLast1)=26.73,2
Snare-(Beat5:AND:!CHatLast1:AND:ChatLast2)=53.9,4


The above attributes specify a few things. First, there is a 26.73% chance that a snare hit will directly follow a closed hi-hat hit on beat 5 (meaning the hi-hat hit falls on beat 4 and the snare hit falls on beat 5). Second, there is a 53.9% chance that a snare hit will follow a closed hi-hat hit by two beats on beat 5 provided that it does not directly follow a closed hi-hat hit (meaning the hi-hat hit falls on beat 3, beat 4 must not be filled by a hi-hat hit, and the snare hit falls on beat 5).

I think that these kinds of logical combinations will allow a thorough analysis of percussive styles.

Absolute Beat
The most basic trigger, this fires on a specific beat number.

Modulus Beat
Very similar to the absolute beat trigger, this trigger fires for each beat number computed modulus a certain divisor. In other words, it may fire every fourth beat, or every other beat, etc.

More on triggers later.

Thursday, February 19, 2009

Markov Drumming Ideas

The Problem: Given a Markov analysis module, particularly for percussion, analyzing pieces of different styles (or even a single piece with slight variations in style) and averaging them into a primary statistics file would cause the file to become a "soup" of conflicting styles. This mushy average would turn into a rather nasty output. It's like taking vibrant blue and green, both very nice colors when taken separately, and combining them to get a nasty brown.

Possible Solution: When analyzing pieces, create statistical profiles of each segment (on an individual measure, or maybe a 4-measure basis) and compare the divergence of the statistics. If the divergence measure surpasses a certain threshold (which the user may set), then the segments are treated as separate styles that use separate statistical profiles. If they don't diverge by much, then the statistics can safely be averaged and saved to the main statistical profile for that style. It's like averaging all the shades of red and all the shades of green separately, so as to avoid mixing to get brown. Furthermore, an overarching statistical profile for style transitions could be made so that the drummer knows how often Style A moves to Style B and when, based on the segmented analysis of each piece. In this way, the analysis could conceivable decipher and reproduce an entire sequence of Intro, Verse, Chorus, Verse, etc. without actually understanding what each part means, just knowing that the statistical profiles for each diverge and transition into each other in certain parts of the composition.

Tuesday, February 17, 2009

Music, the Brain, and Ecstasy

Music, the Brain, and Ecstasy
Chapter 4: Harmony
  • Harmony is essentially the "depth" dimension of music
  • Dissonance is greater between lower notes and it is for this reason that bass notes should be well-spaced
  • The effect or "feeling" of a chord depends mostly on its context within a larger harmonic progression, not solely upon the identity of the chord
  • Notes 1, 3, 4, and 5 in a scale tend to appear more in musical phrases, especially in positions of emphasis
  • "Triads are at the heart of the harmony we are accustomed to"
  • "Western music is built on variations of a few dozen standard progressions"
  • Harmony is all about offsets - relations between notes and chords, not the identity of individual members
  • A group of composers known as serialists, including Arnold Schoenberg, Béla Bartók, and Igor Stravinsky tried discarding tonal centers (thus effectively removing harmony), resulting in a kind of atonal music that few listeners could grasp and even fewer enjoyed
These constitute the notes I took on my reading today.

Monday, February 16, 2009

Saving & Loading

Although it might not seem likes a big step, the ability to save and load project data is pivotal in almost every program. Imagine having to always type an essay in a single sitting, and then having to retype it when you want to make corrections. We simple wouldn't be able to function efficiently without the ability to save and load.

Today I finished a basic saving/loading system that allows my program to save project files and load them again when the user wants. The system saves all instrument and composition data, including all the individual plugin configurations. It wasn't an easy task, but it works, and it makes testing everything a lot easier, since now I can simply make a save file that contains the configurations for all my testing instruments and use it over and over again.

Progress!

Sunday, February 15, 2009

Progress Report #6

Advisor
On Friday I met with Mr. Taranto, ran my ideas by him and filled him in on everything I was planning to do. He seemed very interested and impressed by the scope of the project. He said he'd gladly be my advisor. So I guess that's pretty much that as far as advisors are concerned! With my programming abilities and his musical expertise, I think we'll make a killer team.

Goals/Deadlines
Since I've not had good luck with setting abstract goals in the past, I'm going to set more manageable, concrete ones this week that I know I can achieve. In particular, I'd like to really get cranking on all the books I've ordered pertaining to my thesis, so that I can absorb knowledge to use in the creation of my program.
  1. Read through page 200 in Music, the Brain, and Ecstasy
  2. Read through page 50 in Sweet Anticipation: Music and the Psychology of Expectation
  3. Again, try to have something audible for the class to listen to for the next status seminar.

Goals 1 and 2 should be met by next Monday, if not earlier. Goal 3 should be met by the next status seminar.

So basically my week is going to involve a LOT of reading and absorbing information. I think they're reasonable goals that will help me move towards a greater understanding of music that will ultimately manifest itself in my programming.

Thursday, February 12, 2009

Contour and Arpeggiation

One of the easiest trends for brains to identify in listening to music is that of contour. Contour is the direction of a certain instrument in terms of pitch. For instant, walking your fingers up the keys of a piano would constitute an unbroken upwards contour. Contour is a simple way to create pleasant-sounding order in music. An arpeggio is the technical term for musically walking along a contour - that is, playing a continuous scale either up or down.

A simple arpeggio, notice the note is just increasing by a step each time. This pattern represents a simple upwards contour

I started writing a new plugin for my program that will focus on the orderly but random manipulation of contours to produce pleasing background patterns for music. I am starting with simple contours: up and down. I plan to experiment with nested contours, that is, contours within contours. It's easier to understand this graphically:

This time, the inner contour is increasing by two steps with each note and goes four notes before resetting. The outer contour, however, increases the starting position of the inner contour by a single step each time the inner contour finishes its four notes, then the whole process repeats with the inner contour's new starting position


Although this seems very simple on sheet music (and would seem even simpler in sound form), contour isn't quite so easy for computers to comprehend, especially with several layers of nested contours. I think, however, that creating a program that understands the manipulation of contours will allow a great variety of generated patterns that please the ear.

Monday, February 9, 2009

Renovations

I've completely deconstructed the program's main GUI over the past few days and am now in the process of redesigning it to be sleeker, more efficient, and more powerful all at the same time. It's starting to actually look like a real piece of software now. One of the problems with algorithmic composition is that it's so darn complicated. If I'm ever to get this thing going mainstream, users will need the interface to be intuitive and easy to use.

Thursday, February 5, 2009

Progress Report #5

Purpose Statement

In a broad exploration of algorithmic composition, I will study the creative aspects of music composition. In doing so, I will attempt to learn more about the human creative process and methods for simulating such creativity. I will demonstrate my findings and show my own creative ability by building a working computer program to generate new, original compositions. I will conclude the demonstration by producing a collection of unique and original music for each and every other seminar member in the form of a CD.

Reflection

My inquiry here is no doubt a vast one. My love of composing, performing, and listening to music coupled with my fascination with computer programming has led me to a deep study of the art of computer-generated music. Formally known as algorithmic composition, the field of computer music demands a high degree of knowledge in all of the aforementioned skills; for this reason, the field continues to progress rather slowly even in an age of technological explosion. In order to algorithmically create music, one must not only attempt to uncover the mechanisms behind the human creative process, but also study the patterns that lie within and constitute enjoyable music as well as pack all of this abstract knowledge into the precise bounds of code and user interfaces.

I find the project of creating my own algorithmic composition program appealing for a number of reasons. As I mentioned above, it involves all of the subjects in which I have a great interest: music, computers, and even math. Since the field has had relatively few breakthroughs and an equally scarce number of working programs that implement algorithmic composition, I imagine that if I put a lot of effort into my work, I can make a serious contribution to the field. I can literally count on one hand the number of decent, working algorithmic composition programs available on the internet. Very few fields boast such a lack of implementation.

The work will be difficult. My ability to endure through the challenges that I will face in trying to emulate the creativity of a true human being will no doubt prove my worth as an honors student, provided I manage to endure said challenges. I also think that my work will show my passion for the subjects involved in this project. I'm excited, nervous, happy, and ready to work, all at the same time.

Sources

Resource Link Lists

Algorithmic.net
The largest bank of resources I have found so far, algorithmic.net contains a huge list of algorithmic composition projects as well as all sorts of literature on the subject.
IBM developerWorks spaces: Music Programming and Algorithmic Composition
Another huge repository of all sorts of resources related to algorithmic composition and general computer programming with relation to music.
The Voice of Al-Kwarismi
A very extensive list of links related to algorithmic composition, music theory, and mathematics in relation to music. Unfortunately the site appears to be extremely old and few of the links have survived the years.
Algorithmic Composition Tools

Bloom
Bloom is a mobile application developed for the iPhone that implements algorithmic composition to compose original ambient tracks with or without user input. Brian Eno, the founder of ambient music, co-developed this project.
cgMusic
A freely-available and unpublicized software, cgMusic is the gem of all algorithmic composition tools that I have found so far. Maciej Biedrzycki created this flexible architecture that uses a scripting engine and expert systems to generate coherent music. A very impressive demonstration of algorithmic composition.
noatikl
A fairly extensive algorithmic composition tool aimed at creating ambient music. Good implementation and interface, though restricted in terms of genre.
Soundtrek Jammer Pro
Though more of an assisted-composition tool rather than a full-out algorithmic composition software, Jammer still qualifies to some extent as random composition. The sound demos are pretty impressive and it looks like a well-made program with a nice interface. I could probably learn a lot if I could get my hands on this program.
Algorithmic Composition Methodology and General Information

Algorithmic Composition
A great overview of the history and methods of algorithmic composition as well as a broad definition.
Algorithmic Composition as a Model of Creativity
Bruce Jacob describes the workings behind his variation system for algorithmic composition. Interestingly, his method involves splitting the work into two distinct parts: the 'composer' module and the 'ear' module. The first generates music, the second evaluates the quality of the generated music and filters it until a piece of adequate quality is found. This sounds like a promising method.
A Brief History of Algorithmic Composition
The title of this one leaves little to be explained. Very concise and well-summarized history of algorithmic composition.
Composing with Genetic Algorithms
Bruce Jacob explains in further detail the method of genetic algorithms, part of the variation algorithmic composition system mentioned above.
The Geometry of Music
Discusses and interesting analysis method developed by Dmitri Tymoczko in which chord progressions are mapped to various mathematical spaces. Provides a look into the math of chord progressions and could be useful for the progression module in my algorithmic composition program.

Wednesday, February 4, 2009

New Resource Links

IBM: Music Programming and Algorithmic Composition
Developer space for a wide variety of algorithmic composition resources, tools, etc. Very extensive!

The Geometry of Music
Excellent article that takes a more mathematical approach to music analysis.

Blog: Ideas and thoughts on computer programs that compose music
Katarina Miljkovic seems to have produced a new electronic piece of music for her blog each day for quite some time in 2007. I haven't yet figured out if these pieces were generated algorithmically or by her, but it seems like a valuable resource.

Music Algorithms
Really cool website dedicated to demonstrating the mathematical relations in music. It has a bunch of fun tools that let you try algorithmic composition yourself!

The Voice of Al-Kwarismi
Great list of sources related to algorithms, music, and math in general.

Generative Art and Music
Yet another helpful list of sources related to generative music.