Blog

This is based on the coconet model transforming another Bach chorale. Ach Gott, vom Himmel sieh darein (Oh God, Look Down From Heaven). This chorale uses a lot of notes outide the primary key of D minor. Coconet did his best to harmonize it. I scored it for some samples that I made myself, and don’t have any licensing issues. The instruments are two different finger pianos, one full, and another just for bass notes, plus some Ernie Ball guitar strings, and other assorted strings sounds.

I used the same basic manipulation techniques on this one: stretch out the interesting parts, and repeat them in different ways. This version includes code to randomly choose among several different manipulations:

if final_length > 10: probability = ([0.1, 0.05, 0.05, 0.1, 0.1, 0.05, 0.15, 0.05, 0.15, 0.2])
else: probability = ([0.2, 0.1, 0.06, 0.1, 0.1, 0.15, 0.04, 0.05, 0.1, 0.1])

if type == 'any':
type = rng.choice(['tile', 'repeat', 'reverse', 'tile_and_reverse', 'reverse_and_repeat',
'shuffle_and_tile', 'tile_and_roll', 'tile_and_repeat', 'repeat_and_tile',
'tile_reverse_odd'], p = probability)
print(f'time_steps: {clseg.shape[1] = }, {factor = }, {type = }')

So I control the likelihood of picking different techniques for longer repetitions, favoring ’tile_and_roll’ and ‘repeat_and_tile’. The former tiles the section, basically repeating it note for note, but each time it rolls the notes, so it starts at a different point in the array. Repeat and tile takes half the voices and tiles them, and the other half it just makes them longer. It all works out in the end.

This one was an experiment with a tuning that works well in the primary keys of the chorale, but goes into strange territory with notes outside those triads. It’s designed to sound good with the following triads: D major, A major, G major, B minor, F# major, and E minor. But there is more to the chorale than some triads. The scala file is here:

! adams_on_D.scl
!
Tuning for O Sacred Head Now Wounded: works well on D maj, A maj 7, B min, E min, G major, F# maj
12
!
15/14
8/7
32/27
9/7
4/3
10/7
32/21
11/7
12/7
25/14
40/21
2/1

And the result is here:

Here’s another:

I built a machine that cranks these out by the dozen. This one is interesting. I also implemented a few new routines, one that flips sections horizontally, and another that tiles sections. The former reverses the direction of a short segment, the latter repeats a section over and over. With all the masking going on, the results are sometimes subtle.

This is another fantasia on the output of the coconet Deep Neural Network Model, scored for a primarily percussion ensemble. I fed a real chorale into the model, with the exception of one of four voices, and it predicts the missing voice. I repeat the process until I have 4 chorales all made up by coconet. I then manipulate the chorale using python functions.

An important factor in the way coconet processes the input data, is that is starts with a MIDI file, but immediately translates it into a piano roll structure. The piano roll consists of 32 data points representing notes that are played by each of four voices during a four measure, 32 1/16 note intervals. If a note lasts longer than a 1/16th note, it appears in the next time step. The note ends when a different note appears or a zero appears in a subsequent time step.

For example, here is the first four notes of Bach’s chorale, “O Haupt voll Blut und Wunden”. This is used in several sections of his St. Matthew Passion. I know it in English as “O Sacred Head Now Wounded”.

Each time step is equal to a 1/16th note. The first is the representation in MIDI note numbers, the second is translated into note names.


MIDI:
[66 66 66 66 71 71 71 71 69 69 69 69 67 67 67 67]
[62 62 62 62 62 62 62 62 62 62 62 62 62 62 64 64]
[57 57 57 57 62 62 62 62 57 57 57 57 59 59 57 57]
[50 50 50 50 55 55 55 55 54 54 54 54 47 47 49 49]
Note Names:
['F♯' 'F♯' 'F♯' 'F♯' 'B♮' 'B♮' 'B♮' 'B♮' 'A♮' 'A♮' 'A♮' 'A♮' 'G♮' 'G♮' 'G♮' 'G♮']
['D♮' 'D♮' 'D♮' 'D♮' 'D♮' 'D♮' 'D♮' 'D♮' 'D♮' 'D♮' 'D♮' 'D♮' 'D♮' 'D♮' 'E♮' 'E♮']
['A♮' 'A♮' 'A♮' 'A♮' 'D♮' 'D♮' 'D♮' 'D♮' 'A♮' 'A♮' 'A♮' 'A♮' 'B♮' 'B♮' 'A♮' 'A♮']
['D♮' 'D♮' 'D♮' 'D♮' 'G♮' 'G♮' 'G♮' 'G♮' 'F♯' 'F♯' 'F♯' 'F♯' 'B♮' 'B♮' 'C♯' 'C♯']


So even though a note appears in the time step, it may not be played. Instead the machine knows that if the note in a voice doesn’t change, it holds the note instead of striking it again. The convention in piano rolls is that if a time step in a voice is 0 then that ends the note. If I pass a mask of zeros at locations in a matrix against the piano roll, it turns held notes into arpeggios. I discovered that by mistake as I was playing with the data. But I soon began exploring what matrices could produce the most interesting results.

As before, I searched for those time steps that contained notes not in the root scale of D major, and then extended their duration by 5, 10, or 15 times so that they were heard much longer than the time steps that only contained notes in the root scale. This had the interesting effect of lingering on the passages in the chorale that contained a different key, suspensions, or diminished chords.

This version uses some of my favorite instruments from the music I was making five years ago: harp, classical guitar, marimba, double bass martelé, xylophone, piano, vibes, and bass finger piano. These eight instruments start all playing, then over time they form groups of four, five, or six instruments, until they all come back together at different times.

The tuning is Victorian Rational Well Temperament in C, which seems to work well with the D major that this chorale is written.

I fed another Bach chorale through the coconet model. This one is Wachet doch, erwacht, ihr Schläfer (BWV 78.7, K 188, R 297), which is the same as BWV 353 Jesu, der du meine Seele. I call it “Wake Up, Wake Up, you Sleepers”.

I removed one of the SATB voices, and had the model recreate that voice using the deep learning model in coconet. Then removed a different voice, over and over, until I had 16 voices all created by the model. I then ran those artificial chorales through some conventional algorithmic music functions that I’ve built in python. If this piece could be divided up into four parts, the first one uses the first four synthesized voices, the next uses the next four, and so on until all 16 have been revealed. I played around with a lot of different modifications. More here.

I’ve been working on the algorithmic aspect of this music making enterprise for a while. What I’m doing is not strictly Deep Learning, and not just algorithmic music either. It’s more of a hybrid of the two. In this case, I took one of the 16 voice chorales made with the coconet deep learning model, and then serially modifying each of the four component chorales through a set of algorithms, sort of like the layers in a deep learning model, but really more of simple modifications based on information about the notes. I start out by evaluating all the simultaneous notes to find those that are not in the scale that the chorale is in. These are lengthened up to 5 times their original length. At a later step those steps get a gradual crescendo applied to them. Steps that only use the notes in the key, G minor, get a decrescendo. I arpeggiate it in another step. I then double the voices from 4 to eight. Finally it is sent through a function that shifts the octaves of some notes up. I’ve set that up so that I can specify the percent of notes to have their octaves increase, and then clump that percent of notes into strings of 33 and 45 notes at a time. The effect is sort of shimmering, as some of the notes are higher, but then they are also played at their normal octave at the same time. That’s a filter that I created today.

Here’s the python code for those interested in that sort of thing.


file_stub = 'chorale_HP80063.npy' # one of about a hundred artificial chorales that the model created
dirname = 'wake_up_chorales' # the original chorale is called Wake Up, Wake Up, You Sleepers by J.S. Bach.
chorale = np.load(os.path.join(dirname,file_stub)) + 30
chorale = s.expand_and_concatenate(chorale, expand_codes)
chorale = s.transpose_up_segment(chorale,root) # restore the original key of G minor

masks = np.array([[16,18,2,6,3],[16,18,4,4,2],[16,20,1,4,3],[16,20,2,2,2]]) # arpeggiation mask

concat_chorale = np.empty((4,0),dtype=int)
concat_challenging_steps = np.empty((0,4),dtype=int)
previous_size = 0
for i in range(4): # for each of the four chorales stacked in a 16 voice array, take them one at a time.
sub_chorale = chorale[i*4:(i+1)*4,:]
scores = s.assign_scores_to_time_steps(sub_chorale, root, mode)
range_of_steps = s.find_challenging_time_steps(scores, sub_chorale)
range_in_tune = s.find_in_tune_time_steps(sub_chorale, range_of_steps)
sub_chorale, challenging_steps, in_tune_steps = \
s.expand_challenging_time_steps(sub_chorale, range_of_steps, range_in_tune, high = 6)
sub_chorale = s.arpeggiate(sub_chorale,s.shaded_mask((masks[i,0],masks[i,1]),masks[i,2],masks[i,3]),masks[i,4])
concat_chorale = np.concatenate((concat_chorale, sub_chorale),axis = 1) # add this chorale to the chain of chorales
challenging_steps += previous_size
previous_size += sub_chorale.shape[1]
concat_challenging_steps = np.concatenate((concat_challenging_steps,challenging_steps),axis=0)

chorale = np.concatenate((concat_chorale,concat_chorale),axis = 0) # double the voices from 4 to 8.
# 15% of the time increase the octave of a note, clump the changes in groups of 33 and 45
chorale = s.octave_shift(s.octave_shift(chorale, 0.15, 45), 0.15, 33)


This one is based on synthetic chorale #90, which is the one with the lowest note entropy. I take voices 9 through 15, which equates to A, T, B, S, A, T, B, where we have two altos, two tenors, two basses and a soprano voice. With more notes, there are more wrong notes, so I have more parts to repeat until they make sense. I can make more.

I want to stress up front that this is an experiment in making music. I’m fascinated by deep learning in all its magical incantations, but I’m using it as just one of many tools in my musical toolbox. At present, I’m using deep learning techniques to generate artificial chorales based on Bach using the coconet AI model. But then I’m using my conventional algorithmic musical techniques to turn those chorales into musical expressions.

Today’s addition uses the generated chorale number 44. I search through the chorale for interesting sections to stretch out, in some cases making those sections four to six times longer than the original generated chorale. I then put them through other modifications to arpeggiate in different ways, using a random mask to turn notes on and off. I let the algorithm seek out parts of the chorale that are harmonically using a measure of the percentage of notes that are not in the root key of F major. That’s pretty arbitrary, but it’s very fast, and finds interesting parts to concentrate on in the chorale.

Today’s version fixes a few mistakes:

1. Increased the use of louder and softer samples by increasing the velocity variability
2. Added a copy of the 4 voices to make 8 voices in total, then modified each to improve the arpeggiation variability
3. Added some octave variability in new places in the flow
4. Changed the base fantasy chorale from #90 to #35
5. Played around with the repetition choices

It has than lonely piano player feel, kind of like the acoustic guitar solo they put in 80’s rock albums to show how sensitive the players were when they weren’t plugged into 1000 watt Marshall amplifiers.