EvoSpeak - Progress and Ideas

I'm still working on EvoSpeak, getting the engine all set up. I finished the random generating algorithms that will provide training material from which EvoSpeak will "learn." They also define the basis of the new grammar system, whose syntax is simpler even than that of GrammGen, but whose power is much greater.

Next I need to create the functions that will analyze the training material to figure out what attributes they have in terms of melody and rhythm. All of this analysis data will be stored in a training file that will also indicate how well the user likes the material. After a certain number of training pieces have been graded by the user, EvoSpeak will dig up all the analysis data and perform an extensive statistical analysis on it to try find correlations and develop a "brain," so to speak, that will allow the program to function as an output device.

I'm still trying to figure out exactly what variables/attributes should be part of the "brain." This has always been my problem with statistical models; I've never known exactly what variables to draw statistics from. Now I've got to tackle the issue. I'll start simple - state variables (such as what beat the rhythmic or melodic object falls on) and first-order memory variables (what the last rhythmic or melodic object was) should work fine for the first version.

I plan to have EvoSpeak set up in an intuitive "leveling" kind of way that reflects a simple game. Before EvoSpeak will work, the user must first create a new "creature" that speaks a certain "language." At first the creature will have no idea how to speak the language; like a child, the creature must be shown how to use words to make sentences. The user "trains" the creature by listening to samples and rating them on a scale of 1 (strong dislike) to 5 (strong like). The creature gains XP (experience points) when the user listens to samples and submits ratings. When the creature has enough XP, it can "level up." During the leveling-up process (unbeknownst to the user), the creature actually goes back and analyzes all of the samples and ratings and essentially "learns" from the previous batch of material. The leveling system is good because it will ensure that correlations are relatively strong before they will be used to generate (i.e. the creature won't work without the user having trained it to a certain level).

At higher levels, creatures may learn the ability to analyze deeper variables other than states and first-order memories. Perhaps the creature gains more memory with each level (this is equivalent to increasing the order of the Markov chain analysis). Or perhaps the creature starts analyzing surface contours (3-variable functions) instead of 2-dimensional dependencies.

These are pretty abstract and crazy ideas, but I think they make sense, and I think they will provide a refreshing and intuitive break from the usual grind of KBSs. I'm interested to start training my first creature! And if the leveling system actually makes the music sound better (as intended)...well...I think I could spend all day leveling my creatures (is this starting to sound like Pokemon? That's neither my intent nor my inspiration).

More on Grammatical Subdivision

Here's a rough diagram of the process with the entities shown on the left and the process shown on the right.

Representing Drums With Grammar

I'm still trying to resolve the grammatical method to work with drums and other percussive instruments. The problem is that, while melodic instruments only require notes (and details such as velocity, panning, etc.), drums require notes that correspond to different drums. Thus, the notes are not related in the same way that melodic notes are related in. Drums have no "key" and do not play contoured riffs.

Representing drum patterns within a grammar will require a more flexible grammar that allows for multiple notes on the same beat. I'm thinking of using a very low-level grammar to describe individual hit patterns on the different drums, then combining these in higher-level grammars that describe riffs, fills, and grooves that finally combine into the highest-level grammar that describes an entire style of playing.

Grammar System Ideas

Consciousness Model Ideas

In order to create a sense of flow and retained motifs yet at the same time stylistic diversity, a generative module could at all times retain several streams of 'subconscious ideas' that represent viable solutions to the composition. Each idea uses a different set of parameter values to create an independent behavior for each idea. In this sense, each idea possesses its own personality of sorts. The module's 'conscious' thought starts on any given idea stream. The conscious thought represents the output of the module.

At any given time, however, the module may switch to another idea stream. In this way, the style changes yet the other styles (and motifs) are preserved, because the module continues to process them 'unconsciously.' Effective stream transitions could be made when two streams 'intersect,' so to speak, at a given point. In other words, if the contents of stream A at a time t are similar to stream B at the same time, consciousness may effectively change from stream A to stream B smoothly. The idea stream will then continue off in another direction, until it intersects with another stream at another point.

Here's a diagram of these ideas:
As the diagram indicates, this module has three streams in its subconscious. The streams are being processed independently (thus they have individual characteristic behaviors) but in parallel. The box of consciousness starts on a given idea stream and uses the stream to determine the module's output. The box can switch, however, to a new stream based on switch conditions (such as instantaneous stream similarity).

I think that this idea represents a step forward in both coherence and creativity. I'll try to implement it soon so I can see how the theory plays out in practice.

Ideas on the Quantification of Style

Today I worked on developing a structural outline for a system capable of analyzing and quantifying the essence of musical "style." The analysis will consist primarily of Markov chains with criteria automatically developed and analyzed by the program in the style of a nodal or neural network. In this way, the program will learn autonomously what criteria best 'define' a style and thus learn to reproduce new music in this style.

Markov Drumming Ideas

I need to create a list of statistical attributes that can be analyzed and linked to form a statistical Markov profile for drum styles. I also need to establish how attributes will be stored. Here are some ideas:

Format for statistical attribute: Node-(Trigger:Boolean Operator:Trigger...)=[Correlation],[Strength]

Examples
Snare-(Beat5:AND:CHatLast1)=26.73,2
Snare-(Beat5:AND:!CHatLast1:AND:ChatLast2)=53.9,4

The above attributes specify a few things. First, there is a 26.73% chance that a snare hit will directly follow a closed hi-hat hit on beat 5 (meaning the hi-hat hit falls on beat 4 and the snare hit falls on beat 5). Second, there is a 53.9% chance that a snare hit will follow a closed hi-hat hit by two beats on beat 5 provided that it does not directly follow a closed hi-hat hit (meaning the hi-hat hit falls on beat 3, beat 4 must not be filled by a hi-hat hit, and the snare hit falls on beat 5).

I think that these kinds of logical combinations will allow a thorough analysis of percussive styles.

Absolute Beat
The most basic trigger, this fires on a specific beat number.

Modulus Beat
Very similar to the absolute beat trigger, this trigger fires for each beat number computed modulus a certain divisor. In other words, it may fire every fourth beat, or every other beat, etc.

More on triggers later.

Markov Drumming Ideas

The Problem: Given a Markov analysis module, particularly for percussion, analyzing pieces of different styles (or even a single piece with slight variations in style) and averaging them into a primary statistics file would cause the file to become a "soup" of conflicting styles. This mushy average would turn into a rather nasty output. It's like taking vibrant blue and green, both very nice colors when taken separately, and combining them to get a nasty brown.

Possible Solution: When analyzing pieces, create statistical profiles of each segment (on an individual measure, or maybe a 4-measure basis) and compare the divergence of the statistics. If the divergence measure surpasses a certain threshold (which the user may set), then the segments are treated as separate styles that use separate statistical profiles. If they don't diverge by much, then the statistics can safely be averaged and saved to the main statistical profile for that style. It's like averaging all the shades of red and all the shades of green separately, so as to avoid mixing to get brown. Furthermore, an overarching statistical profile for style transitions could be made so that the drummer knows how often Style A moves to Style B and when, based on the segmented analysis of each piece. In this way, the analysis could conceivable decipher and reproduce an entire sequence of Intro, Verse, Chorus, Verse, etc. without actually understanding what each part means, just knowing that the statistical profiles for each diverge and transition into each other in certain parts of the composition.