Computer Models of Musical Creativity

Computer Models of Musical Creativity
Chapter 3: Current Models of Musical Creativity

• Although randomness often competes with creativity in terms of surprise, it is no substitute for a creative process
• Most random processes are simply too complex to predict
• Randomness arises from a lack of predictability using logic, not a lack of determinism
• Good creativity simply requires good algorithms
• Some models of creativity include cellular automata, mathematical models, fuzzy logic, neural networks, and Markov chains
• Using Markov chains, one can analyze a piece of music and produce new music in roughly the same style
• Genetic algorithms operate on the principle of natural selection
• Genetic algorithms and cellular automata can both generate very complex output
  
• In rule-based programs, creativity really belongs to the programmer, not the program
• Neural networks use "hidden unit" networks to simulate the output of a given situation based on a sample input and output
• Neural networks simulate the workings of the human brain
• Mathematical formulas can be used to produce quasi randomness
• 
  
• "Randomness is not an engaging mystery, but a simple reflection of ignorance"
  
• "Randomness refers to behavior that is either too complex, too patternless, or too irrelevant to make prediction possible"
• "For those believing that using algorithms to create music somehow removes imagination, inspiration, and intuition from the composing process, know that defining a good algorithm requires as much imagination, inspiration, and intuition as does composing a good melody or harmony"
• "Neither good algorithms nor good musical ideas grow on trees"
• "Integrating association-based procedures with data-driven processes increases the creative potential of this approach to music composition"
• "GAs typically involve DNA-like inheritance of characteristics as well as crossover and mutation techniques to develop new traits"
• "Neural networks then cycle through a series of forward or back propagations that compare output with input and alter hidden unit values, until the output values match or approximate the relationships of the input and output data upon which they were trained"
• "Sandwiched between these nodes are variable numbers of layers of hidden units, as well as variable numbers of connections between these inputs, outputs, and hidden units, making the training process extremely complex"
• "We should not overestimate the abilities of neural networks or let comtivity mask a lack of true creativity"
• "Mathematical origins for algorithmic music, while occasionally producing interesting results, in no way indicate the presence of creativity"
• "Computer programs must be sufficiently independent of their programmers and users in order to qualify as truly creative. Most apparently creative algorithmic composing programs either produce enormous output from which users make preferential choices or invoke so many programmer-defined rules that the software only proves the creativity of the programmer"
  

These constitute the notes I took on my reading today.

Bland Statistics

Unfortunately, the analysis tool used to create a permutation of Super Mario has ultimately led me to a dead-end. While such analytic methods might help for basic applications such as chord progressions, they clearly don't work as well when applied to individual notes, since duration, velocity, and padding must also be taken into account.

If I had the time to develop a more comprehensive analysis tool that used more than just a first-order transitional Markov matrix, I might extract better results from this analytic method. I feel, however, that this would simply take too long. Such intense analysis would merit an entire thesis all of its own.

Super Mario Statistics

Today I created a program to statistically analyze midi files and use the collected data to create a new, random 'permutation' of the file using the same statistics. I used the super mario brothers game theme song as my source midi file and the result was surprisingly interesting. Considering the simplicity of the statistics collected, I think the results demonstrate the feasability of a stylistic permutation tool as discussed in a previous post.

Shifted, Narrower Focus

I need to stop fumbling around with stuff for which I don't have a really solid goal. I need to start thinking in terms of concrete genres and concrete goals instead of trying to construct some kind of amorphous compisition from myriad modules of different production methods. I said in the beginning I wanted coherence. I've diverged a bit too much.

I'm thinking simple rock. The majority of voters in my music pole said they preferred rock. I like rock too. Plus, rock is not that hard of a genre to replicate in my opinion. Now I need to start looking at actual songs and thinking about how I would reconstruct them algorithmically, then generalize the processes.

Three Lines

Three Lines
Will define
An ardent mind's
Expense of time

They bring about
No change at all
But in and out

They are three lines.

(This poem is about the project's current code count: 9997, three lines short of a huge landmark)

Stuck

For some reason I'm just not feeling much inspiration lately. I've been pushing into the conscious model and it's a lot of work and, while the results are cool, I'm ready to hear something really inspiring. At almost ten thousand lines of code, the program is huge and I feel like I'm not getting my money's worth, so to speak. I want to hear something awesome. Guess I'll just have to keep going.

Consciousness Model Ideas

In order to create a sense of flow and retained motifs yet at the same time stylistic diversity, a generative module could at all times retain several streams of 'subconscious ideas' that represent viable solutions to the composition. Each idea uses a different set of parameter values to create an independent behavior for each idea. In this sense, each idea possesses its own personality of sorts. The module's 'conscious' thought starts on any given idea stream. The conscious thought represents the output of the module.

At any given time, however, the module may switch to another idea stream. In this way, the style changes yet the other styles (and motifs) are preserved, because the module continues to process them 'unconsciously.' Effective stream transitions could be made when two streams 'intersect,' so to speak, at a given point. In other words, if the contents of stream A at a time t are similar to stream B at the same time, consciousness may effectively change from stream A to stream B smoothly. The idea stream will then continue off in another direction, until it intersects with another stream at another point.

Here's a diagram of these ideas:
As the diagram indicates, this module has three streams in its subconscious. The streams are being processed independently (thus they have individual characteristic behaviors) but in parallel. The box of consciousness starts on a given idea stream and uses the stream to determine the module's output. The box can switch, however, to a new stream based on switch conditions (such as instantaneous stream similarity).

I think that this idea represents a step forward in both coherence and creativity. I'll try to implement it soon so I can see how the theory plays out in practice.

Computer Models of Musical Creativity

Computer Models of Musical Creativity
Chapter 2: Background

• Surprise is a necessary ingredient in creativity
• Associationism - a view that creativity stems of unconscious or subconscious associations - linking thoughts, ideas, experiences, and images
• Letter Spirit, a program used to creatively make new fonts, operates with decision waves - unifying operations that consist of "the gradual, slow-but-sure tightening up of internal consistency all across the structure under construction"
• Cope feels that focusing on details is not the important part of a creative work
• The Turing test may also be an adequate gauge of creativity
• Creativity is not an output, rather, it is a function
• Creativity is not necessarily audible - it is only perceptible when the method of creation is known
  
• "Most recent research in artificial intelligence and creativity tends to focus on connectionism, geneticism, and expertism"
  
• "Full-scale creativity consists in having a keen sense for what is interesting, following it recursively, applying it at the meta-level, and modifying it accordingly" - Hofstadter
• "I do not believe ... that all humans share a common aesthetic"
• "It is the processes, rather than their output, that best identify creativity"
• "The results of creativity should be different from the results of other processes"
• "Creativity is a process, not the result of a process"
  

These constitute the notes I took on my reading today.

Computer Models of Musical Creativity

Computer Models of Musical Creativity
Chapter 1: Definitions

• Many see creativity as restricted to humans
• If humans can't create creative machines, then are they really creative in the first place?
• Two important questions when considering creativity: must the creator know what he or she is creating and must they appreciate his or her own creation?
• Creativity is directly related to intelligence
• Intelligence consists of the following abilities: to learn, to remember, to infer, to analogize, and to create
• Combinatorial accidents can be powerful creative tools and also produce striking originality
  
• "Originality must be useful"
• "Cross-wiring can produce interesting, unique, and important creative connections"
• "Vertically thinking is selective and analytical, while lateral thinking is generative and instigative"
• Cope's definition of creativity: "The initialization of connections between two or more multifaceted things, ideas, or phenomena hitherto not otherwise considered actively connected"
• "I do not feel that creativity requires intelligence, nor does it consequently require life"
• "Creativity should never be confused with arbitrary or convenient contrivances that simply take any road untried for the sake of novelty"
  

These constitute the notes I took on my reading today.

mGen Sample 2

Believe it or not, the program has undergone many changes since the last sample. I created a brand new structure plugin based around the idea of a smooth, contoured intensity structure. I also injected a few lines into most of the other plugins to give them a primitive obey-the-structure rule. Most of the instruments now actually pay attention when the structure module tells them to play loud versus when to play soft. Some, however, still don't have this capability, and most lack any real sense of volume other than the most basic loud and soft abilities.

Still, these improvements constitute serious advancements in mGen. I find this sample to be moderately pleasing. Although it gets repetitive in some places, I'm very pleased with the dynamics. I'm aware that they are still laughably lacking, but at least there ARE dynamics in this sample, unlike the last.

And yes, I know that it sounds very similar. That's due to the lack of rendering options that I have given the program to this point. I'm more concerned with the scoring than the rendering at the moment, but I'll get around to expanding the instrument catalog eventually.

Oh, and a little side note - the drum "solo" at the end was actually a very unintentional and very strange glitch. I've no idea why the drums played a whole extra measure...I'm still trying to figure it out. Three months in and the program already has a mind of its own - I guess I should be proud and worried at the same time!

Music, the Brain, and Ecstasy

Music, the Brain, and Ecstasy
Chapter 6: Composition

• Imagery describes what perception cannot
  
• The brain always works through abstract hierarchies, connecting ideas with a web of relations rather than storing a single data bank for a specific subject
  
• Composers' brains group notes, chords, and progressions with such relations
  
• [My own deduction] "Categorization circuitry" is essentially what is needed to build a computer capable of composition
  
• A soloist draws upon a vocabulary of sounds a phrases that comprise a musical language
  
• In terms of improvisation, a brain can't generate complex, far-reaching structures enough to improvise a piece of significant depth [but perhaps a computer can?]
  
• Mozart mapped out the structure of his compositions before filling in the details (this is great news, because it's how my program is designed! So at least I know this method is workable!)
• Scores can limit the capacity of composers (this again is good news: my program is not bounded by traditional scoring)
• Intelligence comes from neural networks
  
• "It is memory that is the composer's workshop"
  
• "The phenomenon of musical ideas arriving full-blown in the composer's mind is called inspiration"
• "[The soloist] works from a vocabulary of sounds, and a kind of grammar for juxtaposing them, that has become his musical language"
• "Musicians work through a hierarchy of ready-made movements. Thousands of patterns of scales and arpeggios and chord progressions are deeply channeled in their nervous system"
• "[Mozart] began by mapping a composition's structure. Only later did he go back to fill in supporting voices and embellishments that could be written in any number of ways without changing the basic character of the piece"
• "Music notation tends to discourage complex melodies and rhythms"
• "By promoting abstracted, hierarchical thinking, the score can seduce composers into a theoretical, unmusical approach to composition"
• "In the end, intellectual power of any kind arises from the laborious creation of networks of neurons"
  

These constitute the notes I took on my reading today.

Ideas on the Quantification of Style

Today I worked on developing a structural outline for a system capable of analyzing and quantifying the essence of musical "style." The analysis will consist primarily of Markov chains with criteria automatically developed and analyzed by the program in the style of a nodal or neural network. In this way, the program will learn autonomously what criteria best 'define' a style and thus learn to reproduce new music in this style.

mGen Sample 1

Well, here it is (the clip should be playing when you load the blog).

Eighty-nine hundred lines worth of code, embodied in a single audio clip. Yes, I know it's bad, probably pretty much intolerable by the standards of any of you guys. So am I wasting my time? Thousands of lines and all you get is an overly simplistic, predictable drum beat and uninspiring arpeggiations layered on top of overly simplistic chord progressions? No, not at all. Most of the coding so far has been dedicated to creating the framework for mGen, not the actual generative modules. The real work so far was getting a progression, getting arpeggiations, getting a drum beat, and pulling them all together into the same mp3 file and rendering them all automatically (everything in this sample was done automatically, I was literally one button click away from my rendered mp3 file).

I'd say I'm pretty much on target for my goal. For starters I've already achieved the one-click philosophy cited in my proposal. My system has now effectively demonstrated its ability to go from nothing to a finished mp3 file in only a single button click. Also, when one considers briefly the complexity of computer music, the fact that mGen's first sample sounds even remotely close to rhythmically and harmonically sound is quite an impressive feat. There are no chords that sound bad, most of the progressions actually sound good. The arpeggiations sound relatively good too. Both of those items were non-deterministic, meaning mGen completely determined each chord in the progression and each note in the arpeggios (actually it also individually determined the notes in each chord of the progression).

So, in defense of my somewhat-lacking first sample, I have clearly shown that what I am wanting to do is possible. I have not spend too much time coding the actual generative part of my program, so I didn't expect a diamond on my first try. I poured in a lot of effort, and I think I reaped a fair reward.

From here, onward to better-sounding samples.

Music, the Brain, and Ecstasy

Music, the Brain, and Ecstasy
Chapter 5: Rhythm

• Meter vs. Phrasing
• Phrasing is a higher-level unit 0f perception than meter, and encompasses harmonic tension, contour, and dynamics
  
• Pulse lies at the core of meter
  
• Perception of meter is based on prime numbers
• Polyrhythms are made by playing more than one meter at a time
• Syncopation is created when beats are accentuated apart from the regular metrical pattern; often the offbeats are regular enough to anticipate
• Memory vs. Anticipation
• Memory recalls what has already happened, anticipation draws on memory to predict notes to come (usually only a beat or two in the future)
• Importance of tempo: if music moves too slowly, the relations are not close enough to be intelligible; if music moves too quickly, the brain cannot keep up with the relation modeling and has to move to shallower relations, missing the nuances of the piece
• Music needs some gradual changes in tempo; sounds unnatural without them
• Harmonic complexity vs. Metrical Complexity have an inverse relationship
• Most listeners and composers alike now opt for harmonic complexity since harmony information is parallel, while metric information is serial, thus more harmony information can be modeled in a shorter time
  
• "Memory is music's canvas"
  
• "In music, it is phrasing that reaches farthest across time to encompass the deepest relations"
• "Composers gain maximum effect by interweaving the tensions created by music's various aspects"
• "Tempo matters because the mechanics of music perception are exceedingly sensitive to the rate at which musical structures are presented to the brain"
• "Most tempo fluctuations are made intentionally. Music just doesn't sound right without them"
• "The more harmony wanders from its tonal center, the more it requires rhythmic buttressing"
  

These constitute the notes I took on my reading today.

Notes from Meeting with Machover

Although I was unable to write down many exact quotes from Professor Machover (thoughts were rolling quite fast), I snatched the general idea of a few of his important thoughts:

• "I think music is more than entertainment"
• Music should be made to touch people
• Melodies have order but they are diverse, probably the most important part of a piece (relatively speaking)
  
• Nobody has figured out a way to design a composition system that can determine the difference between pretty good and really good

Tod also directed me to the following sources:

• Morwaread Mary Farbood
Her publications on music, especially dealing with tension.
• A Generative Theory of Tonal Music
• David Cope
Pioneer in algorithmic composition

Progress Report #7

A Quick Off-Topic Note
I spent the passed week in Boston visiting MIT (for those who aren't sure what that is, it's the Massachusetts Institute of Technology, widely known as the best engineering/mathematics university in the world, check it out at mit.edu). It was everything I expected and more...I'm simply in love with it. I met with many people there, including a professor of computer science and contributor to the field of cryptography working closely with the National Security Agency, a physics graduate student helping design a massive laser to detect and prove the existence of gravitational waves as postulated by Einstein's General Theory of Relativity, a famous composer and opera writer, and a programmer responsible for an innovative music composition tool. What a week. Now the only problem: 10% admissions rate. Ouch. Wish me luck.

Reflection
I actually did a lot this week. I took my laptop with me to Boston and worked on the program every night. I finished a progression module that works based on the interval observations I posted in my blog a few days ago. It turned out quite nicely, and I'm continuing to refine it.

By far the most productive thing I did this week, however, was meet with Tod Machover (or here), a well-known composer, opera writer, digital composition innovator, and just about everything else you can imagine. After getting over my initial feeling of awe when I met him, I managed to talk with him for a good while and told him about my project. He was genuinely interested and gave me some suggestions but overall seemed to think I was heading down a very promising path. He also introduced me to Peter Torpey, one of the programmers that helped make the Hyperscore software a reality. Peter also took a great interest in my project after I explained it in a fair amount of detail to him. He told me to keep in touch and let him know how the project goes. So I may have found some BETA-testers for my project! (That's a technical term for the people that test a software before it gets released) Peter also told me that a modularized structure (like the one I currently have built in my program) is definitely the way to go. He also said creative interfaces are important for making things intuitive to the public.

Goals

1. Render a music sample
   Obviously this is a recurring goal for me. I have yet to achieve it, just because I don't want the first music sample to be embarrassingly bad. I know it won't be great, but I want it to at least sound somewhat like music. I think I'm getting pretty darn close to achieving this goal.


2. Read more of Music, the Brain, and Ecstasy
   This is an important part of my research. I'm doing heavy note taking and annotating on this book.


3. Organize everything into Noodlenotes
   In order to prepare for my research portfolio, I'll need to organize what I have into NoodeNotes. I should also start printing screen captures of the program's interface so that I can visually prove some of the work I've been doing. Ideally I could even have a sound clip as part of my portfolio.


4. Expand the progression data structure
   This is a pretty specific goal. Right now the progression data structure stores everything in terms of which notes will play and how long they will play. I should at least separate the chord from the bass note so that I can have interesting splits between bass and treble.


5. Add capabilities for post-processing
   I need to make the program capable of running post-processing modules after it finishes running generative modules. An example of a plugin that would fall under this category would be one that applies a swing groove to the piece, or perhaps various playing styles. This is done after the actual notes of the piece have been generated.