I built a machine that cranks these out by the dozen. This one is interesting. I also implemented a few new routines, one that flips sections horizontally, and another that tiles sections. The former reverses the direction of a short segment, the latter repeats a section over and over. With all the masking going on, the results are sometimes subtle.
This is another fantasia on the output of the coconet Deep Neural Network Model, scored for a primarily percussion ensemble. I fed a real chorale into the model, with the exception of one of four voices, and it predicts the missing voice. I repeat the process until I have 4 chorales all made up by coconet. I then manipulate the chorale using python functions.
An important factor in the way coconet processes the input data, is that is starts with a MIDI file, but immediately translates it into a piano roll structure. The piano roll consists of 32 data points representing notes that are played by each of four voices during a four measure, 32 1/16 note intervals. If a note lasts longer than a 1/16th note, it appears in the next time step. The note ends when a different note appears or a zero appears in a subsequent time step.
For example, here is the first four notes of Bach’s chorale, “O Haupt voll Blut und Wunden”. This is used in several sections of his St. Matthew Passion. I know it in English as “O Sacred Head Now Wounded”.
Each time step is equal to a 1/16th note. The first is the representation in MIDI note numbers, the second is translated into note names.
[66 66 66 66 71 71 71 71 69 69 69 69 67 67 67 67]
[62 62 62 62 62 62 62 62 62 62 62 62 62 62 64 64]
[57 57 57 57 62 62 62 62 57 57 57 57 59 59 57 57]
[50 50 50 50 55 55 55 55 54 54 54 54 47 47 49 49]
['F♯' 'F♯' 'F♯' 'F♯' 'B♮' 'B♮' 'B♮' 'B♮' 'A♮' 'A♮' 'A♮' 'A♮' 'G♮' 'G♮' 'G♮' 'G♮']
['D♮' 'D♮' 'D♮' 'D♮' 'D♮' 'D♮' 'D♮' 'D♮' 'D♮' 'D♮' 'D♮' 'D♮' 'D♮' 'D♮' 'E♮' 'E♮']
['A♮' 'A♮' 'A♮' 'A♮' 'D♮' 'D♮' 'D♮' 'D♮' 'A♮' 'A♮' 'A♮' 'A♮' 'B♮' 'B♮' 'A♮' 'A♮']
['D♮' 'D♮' 'D♮' 'D♮' 'G♮' 'G♮' 'G♮' 'G♮' 'F♯' 'F♯' 'F♯' 'F♯' 'B♮' 'B♮' 'C♯' 'C♯']
So even though a note appears in the time step, it may not be played. Instead the machine knows that if the note in a voice doesn’t change, it holds the note instead of striking it again. The convention in piano rolls is that if a time step in a voice is 0 then that ends the note. If I pass a mask of zeros at locations in a matrix against the piano roll, it turns held notes into arpeggios. I discovered that by mistake as I was playing with the data. But I soon began exploring what matrices could produce the most interesting results.
As before, I searched for those time steps that contained notes not in the root scale of D major, and then extended their duration by 5, 10, or 15 times so that they were heard much longer than the time steps that only contained notes in the root scale. This had the interesting effect of lingering on the passages in the chorale that contained a different key, suspensions, or diminished chords.
This version uses some of my favorite instruments from the music I was making five years ago: harp, classical guitar, marimba, double bass martelé, xylophone, piano, vibes, and bass finger piano. These eight instruments start all playing, then over time they form groups of four, five, or six instruments, until they all come back together at different times.
The tuning is Victorian Rational Well Temperament in C, which seems to work well with the D major that this chorale is written.
I fed another Bach chorale through the coconet model. This one is Wachet doch, erwacht, ihr Schläfer (BWV 78.7, K 188, R 297), which is the same as BWV 353 Jesu, der du meine Seele. I call it “Wake Up, Wake Up, you Sleepers”.
I removed one of the SATB voices, and had the model recreate that voice using the deep learning model in coconet. Then removed a different voice, over and over, until I had 16 voices all created by the model. I then ran those artificial chorales through some conventional algorithmic music functions that I’ve built in python. If this piece could be divided up into four parts, the first one uses the first four synthesized voices, the next uses the next four, and so on until all 16 have been revealed. I played around with a lot of different modifications. More here.
I’ve been working on the algorithmic aspect of this music making enterprise for a while. What I’m doing is not strictly Deep Learning, and not just algorithmic music either. It’s more of a hybrid of the two. In this case, I took one of the 16 voice chorales made with the coconet deep learning model, and then serially modifying each of the four component chorales through a set of algorithms, sort of like the layers in a deep learning model, but really more of simple modifications based on information about the notes. I start out by evaluating all the simultaneous notes to find those that are not in the scale that the chorale is in. These are lengthened up to 5 times their original length. At a later step those steps get a gradual crescendo applied to them. Steps that only use the notes in the key, G minor, get a decrescendo. I arpeggiate it in another step. I then double the voices from 4 to eight. Finally it is sent through a function that shifts the octaves of some notes up. I’ve set that up so that I can specify the percent of notes to have their octaves increase, and then clump that percent of notes into strings of 33 and 45 notes at a time. The effect is sort of shimmering, as some of the notes are higher, but then they are also played at their normal octave at the same time. That’s a filter that I created today.
Here’s the python code for those interested in that sort of thing.
file_stub = 'chorale_HP80063.npy' # one of about a hundred artificial chorales that the model created
dirname = 'wake_up_chorales' # the original chorale is called Wake Up, Wake Up, You Sleepers by J.S. Bach.
chorale = np.load(os.path.join(dirname,file_stub)) + 30
chorale = s.expand_and_concatenate(chorale, expand_codes)
chorale = s.transpose_up_segment(chorale,root) # restore the original key of G minor
masks = np.array([[16,18,2,6,3],[16,18,4,4,2],[16,20,1,4,3],[16,20,2,2,2]]) # arpeggiation mask
concat_chorale = np.empty((4,0),dtype=int)
concat_challenging_steps = np.empty((0,4),dtype=int)
previous_size = 0
for i in range(4): # for each of the four chorales stacked in a 16 voice array, take them one at a time.
sub_chorale = chorale[i*4:(i+1)*4,:]
scores = s.assign_scores_to_time_steps(sub_chorale, root, mode)
range_of_steps = s.find_challenging_time_steps(scores, sub_chorale)
range_in_tune = s.find_in_tune_time_steps(sub_chorale, range_of_steps)
sub_chorale, challenging_steps, in_tune_steps = \
s.expand_challenging_time_steps(sub_chorale, range_of_steps, range_in_tune, high = 6)
sub_chorale = s.arpeggiate(sub_chorale,s.shaded_mask((masks[i,0],masks[i,1]),masks[i,2],masks[i,3]),masks[i,4])
concat_chorale = np.concatenate((concat_chorale, sub_chorale),axis = 1) # add this chorale to the chain of chorales
challenging_steps += previous_size
previous_size += sub_chorale.shape
concat_challenging_steps = np.concatenate((concat_challenging_steps,challenging_steps),axis=0)
chorale = np.concatenate((concat_chorale,concat_chorale),axis = 0) # double the voices from 4 to 8.
# 15% of the time increase the octave of a note, clump the changes in groups of 33 and 45
chorale = s.octave_shift(s.octave_shift(chorale, 0.15, 45), 0.15, 33)
This one is based on synthetic chorale #90, which is the one with the lowest note entropy. I take voices 9 through 15, which equates to A, T, B, S, A, T, B, where we have two altos, two tenors, two basses and a soprano voice. With more notes, there are more wrong notes, so I have more parts to repeat until they make sense. I can make more.
I want to stress up front that this is an experiment in making music. I’m fascinated by deep learning in all its magical incantations, but I’m using it as just one of many tools in my musical toolbox. At present, I’m using deep learning techniques to generate artificial chorales based on Bach using the coconet AI model. But then I’m using my conventional algorithmic musical techniques to turn those chorales into musical expressions.
Today’s addition uses the generated chorale number 44. I search through the chorale for interesting sections to stretch out, in some cases making those sections four to six times longer than the original generated chorale. I then put them through other modifications to arpeggiate in different ways, using a random mask to turn notes on and off. I let the algorithm seek out parts of the chorale that are harmonically using a measure of the percentage of notes that are not in the root key of F major. That’s pretty arbitrary, but it’s very fast, and finds interesting parts to concentrate on in the chorale.
Today’s version fixes a few mistakes:
- Increased the use of louder and softer samples by increasing the velocity variability
- Added a copy of the 4 voices to make 8 voices in total, then modified each to improve the arpeggiation variability
- Added some octave variability in new places in the flow
- Changed the base fantasy chorale from #90 to #35
- Played around with the repetition choices
It has than lonely piano player feel, kind of like the acoustic guitar solo they put in 80’s rock albums to show how sensitive the players were when they weren’t plugged into 1000 watt Marshall amplifiers.
This is a piece that originated as Bach’s BWV 180 Schmücke dich, o liebe Seele, reprocessed by the coconet deep learning model. I split the chorale up into 32 1/16th note segments, which is what the model was built to handle. I zero’d out one of the voices, and had the model remake that voice to create a four voice chorale similar to the Bach. Then I did it again, masking a different voice each time.
First, I searched for generated chorales that the highest pitch entropy using the muspy framework. Muspy is a library that reads a variety of musical structures, and includes many functions that help convert music files to other formats. It includes a set of metrics that measure a variety of qualities of the music. In my case, I searched the 100 16 voice chorales for those that had the most entropy, which in my case was the 12th through 15th voices of chorale #90.
Then I took that four voice chorale and stretched it out to three times its length. Next, I looked for parts of the longer chorale that had interesting segments. This was done using a measure of pitches outside the root F major scale, but on small subsets of the chorale. I stretched the most interesting segments by up to 8 times their original length. This was done in order to linger on the leading tones, suspensions, and other Bach tricks to add suspense and interest.
Next, I arpeggiated the entire chorale using a mask, not unlike the deep learning technique of convolution, but in my case just masking parts of notes, so that it created an arpeggio effect.
Finally, I rendered the chorale using Csound and my microtonal slide Bosendorfer in George Secor’s Victorian Rational Well Temperament. I added a convolution with an impulse response file from Teatro Alcorcon in Madrid from Angelo Farina. I think it sounds sweet, kind of like a finger-picking guitar.
It’s not Bach. But then, neither am I.
This one was done a different way than the previous versions. Previously I had created 16 voice chorales by preserving just the bass part and letting the model figure out the other notes. What I created were four different versions of harmonizing a bass line. Each knew nothing of what the other ones had created, which led to chaos. I had to surreptitiously go back and blot out the notes that were in foreign keys. This was a rather crude way to compensate. Anyway, I though of a different way.
In this version, I went through the process in a very methodical way. I found a way to mask one voice and keep the others, letting the model use it’s own judgement for what that new voice should be. I then went through all the voices in the four part chorale until they had all been replaced. I kept doing that until I had a total of 16 voices. The new voice is usually pretty close to the original, matching 80% of the original notes.
Imagine a chorale with voices S A T B. The first time through it S becomes S prime, which creates a chorale with S’ A T B. Then it makes S’ A’ T B, followed by S’ A’ T’ B, finally making S’ A’ T’ B’. It saves that generated chorale (4,32) in a slot in a (4,4,32) array. Then it does it again, gradually shifting from the original chorale to one that includes some odd notes. S’ A’ T’ B’ becomes S” A’ T’ B’, then S” A” T’ B’.
Each is stored in the (4,4,32) array. At the end, it reshapes that into a (16,32) array and returns it to the calling program.
I end up with a 16 voice chorale. The first four are very close to the original chorale (Schmucke by Bach). They are in the far left of the audio stereo field. The ones towards the right are mutations of that, until on the far right it has gone into strange areas. The ear can’t really separate them out, so you end up with a strange mess of notes that include many that don’t belong. Maybe if I slow it down it would make more sense.
I should mention that I’ve been using the George Secor’s Victorian rational well-temperament (based on Ellis #2) in F for these realizations. It does a pretty good job in that key.
George Secor's Victorian rational well-temperament (based on Ellis #2) on F