EvoSpeak - Optimization

Yes, I'm STILL working on getting the analysis part of EvoSpeak working. I now have the structure of the species' brains figured out and I've optimized the analysis engine a LOT, thanks to the new storage method of the analysis filters. So things are looking pretty good and soon enough I should be working within the interface of EvoSpeak instead of grinding around in the code.

I'll admit, progress on mGen is coming slowly. It still looks like I'm going to make the code deadline that I set for Wednesday (17,000 lines), which is encouraging. How is mGen shaping up in the big picture/long run? That's what I'm more worried about. I'm going to have to step back and take a serious look at what's going on after I hit this deadline. I don't even really have anything for Mr. Taranto to help with yet, even though our meeting is in under two weeks. It's time to step up to the plate.

EvoSpeak - Analysis

Work is starting to get pretty messy with EvoSpeak. I'm trying to design a very generalized analysis framework to allow easy analysis of just about any relationship. Doing so is not at all easy. I'm trying to set up a "perception" matrix that simulates the state of any given sample at any given point in time. The idea is that an analysis function can built a filter matrix and then call the perception function, which will then compare the filter matrix to the perception matrix and gather some statistics.

The first analysis I'm designing is, of course, a zeroeth-order Markov. Fancy language aside, what it boils down to is this: did the species use a certain word (melodic or rhythmic) in a certain sample? So a zeroeth-order Markov simply deals with the innate "quality" of certain words over other words, not taking into account ANY contextual variables.

Problems are arising with this general framework. It's very difficult to obtain certain state values to populate the matrix because of the grammar engine design. The melodic and rhythmic data streams are asynchronous, so melodic events don't necessarily line up with rhythmic events, which makes finding synchronous data (like perception data) very difficulty. Apparently I've messed up in trying to separate the streams because some of the preliminary statistics are doing some strange things.

On top of all that, the analysis is a LOT slower than I thought it would be, even after serious reconstruction and optimization. I knew that it would take a lot of loops and recursions to do the analysis...but I thought the computer would just chew through them. Already a simple zeroeth-order Markov analysis on the melody alone costs about 2.6 seconds. Using that number and extrapolating, a second-order Markov analysis would take a whopping sixteen minutes, which is simply unacceptable. And that's only to level-up once. I'm definitely going to have to figure something out there.

While I'm running into some obstacles, EvoSpeak is still advancing steadily and I'm confident that the analysis functions will soon be fully-functional.

EvoSpeak - Getting Closer

I finished the preview builder and now have a working random pattern generator and previewer for EvoSpeak. I still can't submit ratings so species don't gain experience yet, but the hardest work is done...until it comes time to build the "leveling" mechanism (i.e. the Markov analysis tool).

And the results of the initial grammar runs? Good! Overall, I am very satisfied with what I'm hearing. Based off of the twenty-or-so previews that I've listened to so far, the engine is much more interesting than GrammGen. It sounds a lot better.

The thing I really like, however, is that switching languages dramatically changes the previews. Of course the same was true for GrammGen, but I never built a second language for GrammGen because of the relative difficulty of editing the languages. In EvoSpeak there's a built-in language editor. It's as easy as slapping in some pipe-delimited numbers for rhythm and melody and listening to the results.

It took me thirty seconds to build a language that could be used for repetitive arps in the background. So I think I've found my solution for arpeggiation! The simple the language, the more likely it is to repeat words - which is exactly what you want in a background pattern. After listening to some previews of the new language, I'm certain that this will be a very promising and flexible system.

So far EvoSpeak is going very well! The real question, however, has yet to be answered: will the "experience" and analysis system actually allow EvoSpeak to improve the quality of its output? The answer would seem to be a very obvious yes if I do everything right. But at the same time, it's hard to believe that listening to samples and pressing buttons can train a program to make better music. But who knows, I guess I'll just have to find out.

PS - It's worth noting, in case I was never clear about this, that EvoSpeak is NOT a grammatical subdivision engine like GGrewve, rather, it's a grammatical chain engine like GrammGen. Chains are simpler and easier to work with but subdivision is more powerful. And yes, I coined both of those terms, which is why you won't find information on them anywhere else :)

Slow

Work's going a little slow this week as I have a lot on my plate, especially starting my internship (32 hour week).  I'm brushing up WordSmith here and there, trying to improve the dictionary analysis quality so renders choose more appropriate words. What worked well for drums is taking quite a bit more messing with to work well for other instruments.

Unfortunately I'm having doubts concerning the viability of grammatical subdividion with non-percussive instruments.  While it may be possible, I think it's lacking in structure.  Perhaps there also needs to be a higher-order grammatical structure to guide motifs so that they do not lack direction.

The question is, then, how does one classify a higher-order grammar?  What symbols make up such a grammar?  Perhaps phrases could be classified as "suspenseful," "tense," or "declining."  Of course there could be loads of other adjectives.  The point is that sentences.  Sense do not make?  When words a sense of direction lack!  Yes, that was a bit of a punny example of how grammar can go wrong.

Rough days ahead for algorithmic composition.  Fasten your seatbelts and keep a pale ready for sea-sickness.  We will be experiencing some turbulent, nonsensical grammar.

WordSmith - Generalized Grammar

Today I began construction of a more generalized grammar framework - very similar to the one upon which GGrewve runs, except extended to work for melodic instruments as well as percussive ones.  I began a crucial part of this rewriting today with WordSmith - the new equivalent to GGrewveWriter.

WordSmith is a generalized grammar/dictionary synthesizer capable of taking a "training file" in MIDI form and constructing a dictionary from the file.  WordSmith, unlike GGrewveWriter, actually has a very flexible interface and of course will work for any instrument, not just drums.  In addition to the usual features of the last grammar analysis tool, WordSmith will also include extended features such as time compression and maybe even synthesis tools like hybridization and mutation for more random possibilities.

As a minor under-the-hood tweak, WordSmith will perform extensive quantitative abstract analysis on individual words (categorizing intensity and such) so that plugins like GGrewve don't have to waste valuable processing time doing analysis that can be done once up front instead of once every run.  This will also greatly increase the possibilites for such analysis, since a one-time processing cost will allow for much more extensive and in-depth analysis of the grammar. Users won't mind having to wait half a minute to create a dictionary, as long as it doesn't take the same amount of time every time the dictionary is used by a plugin.  Which it won't.  It'll only take milliseconds for the plugins to access that analysis data that will have already been created by WordSmith.

Grammatical subdivision is going full speed ahead, and I hope this promising method won't let me down.

GGrewve Fill Dictionaries

Today was yet another huge leap forward for GGrewve.  I coded a second dictionary loading into the program so that it now keeps two dictionaries in memory: one for the main style, and one for fills.  Basically, a fill dictionary looks and functions exactly like a normal dictionary.  I wrote a small MIDI file that contained several nice fills and then passed it through GGrewveWriter, yielding a fill dictionary.  Along the way I had to do a lot of bugfixing on GGrewveWriter, which threw a fit because the fill patterns were a bit different from normal patterns.

At any rate, after everything was said and done, GGrewve was intelligently using a fill dictionary to make entrances as well as highlight fourth-measures.  The result?  Awesome.  Simply awesome.  It sounds great - it sounds real.  Even more impressive, GGrewve displays great inventiveness in using the fills.  As I explained the process of grammatical subdivision before, GGrewve basically chops up the MIDI files I give it and recombines them using intelligent analysis.  Not only were many of the fills very inventive (I wouldn't have thought to make them), they were coherent and well-placed!  I wasn't just hearing a repeat of the file I had fed into GGrewveWriter.  I was hearing an elaboration on those files - the same style, but not the same pattern.  GGrewve was not mimicking my speech, it was mimicking my ideas with its own speech (to use a linguistic metaphor).

GGrewve is by far my most impressive plugin so far, and with a solid drum plugin to depend on, I know it can only get better from here.

GGrewve Intelligent Decision Making

Major progress has been made with the GGrewve drum plugin. It now has the ability to determine which words are more intense based on an extremely scalable and versatile analysis engine. Using this knowledge, GGrewve is able to place words more appropriately than before. Without even needing to look at the context of the words in the original MIDI file from which they were extracted, GGrewve uses the words to simulate the way a real drummer plays. Accents and fills will occur more during even measures, with a greater weight every fourth measure. In this way, GGrewve allows listeners to more effectively keep track of the measure, which gives the composition a much greater overall coherency.

The intelligent GGrewve engine is a step forward in autonomous music generation!

GGrewveWriter Analysis Tool

The development of GGrewve is going well, but rather slowly. This is partly because of the difficulty in expanding the dictionary. This is not an unfamiliar problem: I ran into the same issues with Variating Drummer. It's generally difficult to write drum patterns in anything short of a full-blown sequencer. Since writing a sequencer is beyond my capabilities, I have developed a pretty cool alternative.

Enter GGrewveWriter, the intelligent drum pattern analysis tool that disects a drum beat and packages it up for use with GGrewve.

In a nutshell, GGrewveWriter takes a MIDI file that contains a drum beat, splits the beat up into sections of an appropriate length, and derives "words" from these small sections. GGrewveWriter pays close attention to the placement of words within the original beat and the subtleties of wordly interactions. GGrewveWriter then creates a compressed dictionary file that contains all the grammatical data about the drum beat found in the MIDI file. This compressed dictionary can be read directly by GGrewve, and is actually more efficient than the uncompressed dictionaries used previously.

The whole thing is very impressive in practice.

It took me no more than a few minutes to write eight measures of a neat little drum beat in a sequencer and export it to a MIDI file. After exporting, I let GGrewveWriter go to work with the file. After only a few seconds, I had a fresh compressed dictionary sitting in the folder. I loaded the dictionary into GGrewve and let the generation begin. Sure enough, out came forty measures (I had requested this much) of well-placed, humanized drum beat. I had actually made the original MIDI beat sound very stale (I did not vary the volume levels at ALL, so it sounded very robotic) to see how well the humanization routines within Grewve were working. Sure enough, the computer's output sounded a heck of a lot more human than the beat that I, a human, had created. The output was varied and clearly derived from my beat, but not exactly a copy and pleasantly unpredecatable.

Writing the beat, creating the dictionary, and generating took about seven minutes. Had I tried to achieve the same thing with the Variating Drummer, it would have taken hours and hours and the final result would not have been as impressive.

This is a very successful test for GGrewve, and I look forward to more impressive results in the future (we may finally be ready for drum fills!)

GGrewve

GGrewve generated output for the first time today. The dictionary right now is very small and the subdivision rules are very strict, so there's really no variety in the drumming yet, but that was intentional so that I could test the engine. Everything worked very nicely.

One of the cooler parts of the engine is the "humanization" function that varies the velocities to simulate a "groove" as well as add ghost notes to the snare drum like professional drummers do automatically. Even with a tiny dictionary (right now it has only five simple words that correspond to four-beat patterns) the drumming quality already surpasses the Variating Drummer, so I think I'm on the right track with this plugin.

I've still got obstacles to conquer. Most notably, I have to devise a way for GGrewve to determine which words are "fancier" and "heavier," so that it knows in what context to use the words. For example, a fill should obviously not come on the first beat of the first measure of a section...it should be saved for a turning point or a transition into another section. I could just include some quantitative specifications in the word files to let GGrewve know where they should be used, but I think that's taking too much of the computer work out of it. I'll write the words but GGrewve has to figure out how to use them. This will, of course, be a lot more work that just telling it when to use words, but I think the results will be more satisfying. Expanding the dictionary will also be a lot easier if I don't have to take the time to think about when each word should be used.

In conclusion, I'm excited about where the drumming side of mGen is going and I think GGrewve has a bright future.

GGrewve

I'm still in the process of implementing the grammatical subdivision idea in a drum plugin. So far everything is going smooth and it's shaping up to be a really nice module! It's called GGrewve (grammatical groove).

I hope to have some simple output in a few days. The idea is a LOT easier to work with than that of the Variating Drummer, which has become so convoluted that even I have trouble figuring out how to build a drum pattern (which shouldn't happen, considering the computer is supposed to be the one doing the work).

At any rate, more to come on grammatical subdivision and GGrewve in the near future.

More on Grammatical Subdivision

Here's a rough diagram of the process with the entities shown on the left and the process shown on the right.

Grammatical Subdivision

I had somewhat of a small breakthrough several minutes ago. Grammatical percussion is really all about subdividing time. It's about taking a given chunk of time and doing something with it. Unlike the melodic grammatical system, the percussive system should have "words" that correspond to different ways to subdivide a set amount of time. This way, one does not have to deal with overlapping words and other problems that arise when the length of a word isn't absolutely defined.

The process of subdivision is recursive, so the process might start at the level of a measure and ask, "ok, how do I want to subdivide this measure?" The response may be, "I want to equal subdivisions." The process will then make another decision: "Do I want to go ahead and replace this chunk of time (half a measure) with an 8-beat word? Or do I want to subdivide it further?" Let's say the computer decides to split the first chunk of time into two quarter notes (4 beats, also 1/4 of a measure). It decides to replace the second chunk with a word.

So the system really operates around two entities and one process: time "chunks ," words, and subdivision, respectively.

Conceptually this is a big step forward for grammatical drumming. With any luck it will make the process a lot less painful.

Representing Drums With Grammar

I'm still trying to resolve the grammatical method to work with drums and other percussive instruments. The problem is that, while melodic instruments only require notes (and details such as velocity, panning, etc.), drums require notes that correspond to different drums. Thus, the notes are not related in the same way that melodic notes are related in. Drums have no "key" and do not play contoured riffs.

Representing drum patterns within a grammar will require a more flexible grammar that allows for multiple notes on the same beat. I'm thinking of using a very low-level grammar to describe individual hit patterns on the different drums, then combining these in higher-level grammars that describe riffs, fills, and grooves that finally combine into the highest-level grammar that describes an entire style of playing.

Grammar: Success?

Well, after coming back from a long break, I have some good progress to report.

I tried implementing a grammatical system for algorithmic composition over the break and had a good deal of success in my endeavors. Although I created only a rudimentary composition language and a very basic phrase generator, the results seem more natural and interesting than any other method explored thus far, which means progress!

After a great deal of thinking on the subject, I've decided that a grammatical system might be the key I've been looking for to a successful path to my goals. The trick is that I can use a grammatical system as the underlying paradigm for other methods. In other words, I could have an evolutionary model that uses an underlying higher-level grammar to generate phrases. In theory, this idea is really no different than using an evolutionary model to generate an equally-abstract number that corresponds to a certain pitch (a MIDI note event). I could do the same thing with Markov chains and state-transition matrices.

There are a lot of places to go with grammatical algorithmic composition. I have a feeling I'm just scraping the surface of something big. Let's hope I'm not let down.

Grammar System Ideas