Chordination

Pitch Class Profile (PCP)

The PCP is the heart of our chord detection algorithm. The PCP is effectively a system of binning any frequency spectrum into distinct notes. First, N(w) is generated. N(w) is shown below - steps are centered around the frequencies of notes( A = 110, 220, 440... Hz).

An integer value is assigned to each note: A = 0, A# = 1, B = 2, etc. We are given the FFT of the input signal X(w). Then, for any w such that N(w) = n, we sum over X(w) only on those values of w. This sum is the nth entry of the PCP vector.

The following diagram shows the output of the PCP algorithm when input a B-flat major chord.

The output vector has large magnitudes for A#, D, and F, the notes in the detected chord.

We eventually realized that the N(w) given by [1] is only correct in theory and not in practice. The given N(w) treats every frequency in the FFT as an equal contributor to a note - the magnitude X(440Hz) is weighted the same as the magnitude X(453Hz). This is unintuitive and musically misleading - 453 Hz is midway between A and A# and its magnitude should not be added to either note. Even a low resolution FFT in our processing scheme would not shift the 440 Hz frequency component to 453 Hz due to quantization error of the DFT. A further inspection of several FFTs of real songs indicated that the peaks of the frequency spectra fell within 5 Hz of a theoretical note frequency more than 95% of the time. Thus we generated a new N(w) that discards any frequency further than 10 Hz away from a theoretical note frequency. For example, frequencies between 430 and 450 Hz and between 456 and 476 Hz are summed in the PCP - they are within 10 Hz of A and A# (at 440 and 460 Hz respectively). The frequency component at 453 Hz is dropped.

Our improved PCP was implemented in our final algorithm execution and is successful at reducing noise in the output PCP vectors, thus increasing the accuracy of the Chord Detector

Smoothing Convolution

We often found that the detected notes were “spiky”; certain notes would be detected in certain windows but not in adjacent ones. This would cause the detected chords to vary widely from window to window, resulting in an erratic output. To solve this problem, we implemented a smoothing convolution. We created a vector whose length was three times number of overlaps and set the first half of its elements to 0s and the second half to 1s. For example, if the number of overlaps were 4, the smoothing vector would look be [0 0 0 0 0 0 1 1 1 1 1 1]. We then convolved this vector with each row of notes. The following diagrams show the output of the PCP algorithm with a C major chord, before smoothing:

And after smoothing:

The smoothing removes the higher order variations from the correct notes as well as forces the erroneous notes closer to 0.

ChordFinder

Chord Dictionary

The final task for our algorithm is to use the detected notes output from the smoothed PCP to determine which chords are being played. This is accomplished using our ChordFinder, a function which searches a dictionary of chords to find a closest match for the detected notes.

We discovered an online database of 660 chord names and their corresponding notes. Using C, we parsed this file into two parts: a string array containing the 660 chord names (“Cmaj”, “Cmin”, …) and a parallel array containing 660 1x12 binary vectors. In these vectors, each element corresponds to a note in the chromatic scale (A A# B C C# D D# E F F# G G#), and each 1 corresponds to a note present in the chord. For example, the vector [1 0 0 0 1 0 0 1 0 0 0 0] represents an A major chord (which contains the notes A, C#, and E). We then converted the binary vectors to decimal numbers (for example, [1 0 0 0 1 0 0 1 0 0 0 0] became 2^11 + 2^7 + 2^4 = 2192) and saved the arrays in .csv files. Finally, we created a map in MATLAB with the chord names as values and the decimal values as keys.

Through testing our algorithm, we discovered that the algorithm was often likely to detect erroneous notes and consequently choose very complicated and unusual chords from the dictionary. These chords, such as F#5(add13), are very unlikely to occur in real music, especially popular music. Therefore, we decided to create a smaller chord dictionary containing 64 of the most common chords in popular music: the major chords, the minor chords, the seventh chords, and the minor-seventh chords. Using this simpler dictionary made our algorithm much more robust to noisy data, at the expense of losing the ability to detect unusual chords.

ChordFinder Algorithm

The ChordFinder algorithm first removes notes in the PCP vector below a certain threshold to clean the data. It also “rewards” notes above this threshold by boosting their levels closer to 1. It then computes the differences between the PCP vector and each binary vector in the keys of our chord map. It computes this difference as the sum of the absolute value of the differences between each element of the vectors. The “closer” the PCP vector is to a given chord key, the smaller this difference will be. Finally, it chooses the key with the smallest difference and uses this key to decide which chord is being played. We also created a version of this algorithm which took the dot product of the vectors rather than their difference; this version tended to be accurate with our simplified dictionary but would err towards complicated chords with our full dictionary.

Final Output

The most basic output of our algorithm is a string vector containing the detected chords in each window. Additionally, we created two optional outputs to give the user more insight into how our algorithm functions:

LiveDisplay: If this option is chosen, once Chordination has completed detecting chords it will play the song and simultaneously output the corresponding detected chords in the command window. This allows the user to hear the audio and see whether the detected chords match in real time.

PlotDisplay: If this option is chosen, Chordination will output a plot showing the strength of the notes detected in each window. This gives the user a good visual representation of Chordination’s output as well as an insight into how accurately notes are detected. Here is an example of the plot for the first verse of the Beatle’s “Hey Jude”:

Benchmarking

We wanted a quantifiable metric for how well our algorithm performed. We created a simple chord progression with seven chords:

This chord progression is good for testing because it contains six unique chords, and each chord lasts for exactly one second, so we know exactly what chords should be chosen in each window. We created 8 different audio versions of this chord progression, 5 synthetic and 3 real:

• 1. A basic synthetic piano version
• 2. An arpeggiated (broken chord) synthetic piano version
• 3. A “short” synthetic piano version (each chord plays for half a second rather than a second)
• 4. A synthetic version without the bass notes in the chords
• 5. A synthetic organ version
• 6. A basic real guitar version (a recording of Phil strumming these chords)
• 7. An arpeggiated real guitar version
• 8. A “short” guitar version

We then created an “answer key” in MATLAB, a string vector consisting of the correct chords for each windowed time interval. We ran the algorithm with each of the 8 versions on our chord progression: for each version, we compared the detected chords to the answer key and gave a score to the algorithm based on the number of chords it correctly named. These scores gave us an insight into how well our algorithm performed, and allowed us to see the impact of changing our algorithm parameters.

The Baseline, (Case 1) is the final algorithm as presented at the end of the term, and Cases 2-6 show different variants of the algorithm.