A device that shifts the pitch of an input sound signal is presented in detail. The pitch shifter uses a simple algorithm of stretching or squeezing a sample in time to perform the desired pitch shift. The system's Analog to Digital and Digital to Analog mechanisms are examined in detail. The system also includes a Micro-Programmable Control Unit, whose instruction format and operation are explained. A Storage Unit for addressing the necessary buffers and an Accumulator for mixing the original and shifted signals are also presented. A major problem in the Accumulator that was encountered is described and a work around of the problem is presented. Finally, some reflections on lessons learned are discussed.
For the third laboratory experience in 6.111, a sound pitch shifter was built. The pitch shifter takes as an input an analog voltage representing a sound wave, and produces as an output an analog voltage representing a pitch shifted version of that sound. The shifted signal can have either higher or lower pitch than the original signal. The output can be either one or an arithmetic average of the original signal and the pitch shifted signal.
To produce the pitch shift, short samples of the input are recorded. The samples are then played back either faster or slower than they were recorded. By playing the sample back slower than it was recorded, the perceived pitch is shifted down. By playing the sample back faster that it was recorded, the perceived pitch is shifted up. To stretch out the sample, the information at the end of the sample is discarded. To squish the sample, some of the sample's information is played twice. Figure 1 illustrates how a particular sample's pitch can be modified by stretching or squishing the original waveform.
By removing the analog signals from the block diagram, the design of the pitch shifter can be done exclusively in the digital domain. The separation of the analog components from the digital components is shown in Figure 3.
The A/D is an Analog to Digital converter. The A/D is a single chip which converts an instantaneous analog voltage on its input line into an 8 bit digital value on its output lines. The time at which the A/D samples the input voltage is determined by control signals generated by the digital pitch shifter.
The D/A block is a Digital to Analog converter. The D/A is a single chip which converts a 8 bit digital value on its input lines into an analog voltage on its output lines. In this implementation of the pitch shifter, the D/A is a part of the Accumulator, and hence the digital part of the pitch shifter only deals with the D/A indirectly through the Accumulator interface.
Figure 4 illustrates the further decomposition of the pitch shifter into its component modules. The heart of the pitch shifter is a Micro-Programmed Control Unit (MCU). An MCU is similar to the CPU found in personal computers, except that the MCU is specialized to a particular task. The MCU has specialized output for controlling the A/D, the Storage Unit, and the Accumulator. The MCU is programmed using an assembly language created specifically for the pitch shifter system and that is described in detail below.
In addition, the pitch shifter also contains a Timing Unit which provides both the system clock and the sample clock. The sample clock determines the rate at which the pitch shifter samples the input signal. The frequency of the sample clock depends on the current setting of the sample_frequency input switch.
The Storage Unit is responsible for maintaining two buffers using a Random Access Memory (RAM). The sample buffer is filled with the current sample. The shift buffer contains the last sample recorded. By reading the shift buffer either more or less quickly than the sample buffer, the pitch of the stored sample can either be increased or decreased.
The Accumulator allows a mixture of the original and the shifted signal to be outputted at the same time. The Accumulator can average its current value with a new value to easily mix the two signals together.
The system clock is created by dividing the crystal clock in half by using a 74LS393 counter. The crystal oscillator is fed into the clock input of the 74LS939 and the second least significant bit of the output is used as a system clock. The output of the 74LS393 is buffered using a 74LS04 inverter to avoid clock skew. Clock skew is a potential problem because the system clock is widely distributed around the pitch shifter.
The sample clock is generated by using a 22V10 Programmable Array Logic (PAL) programmed with the VHDL code shown in Figure 7. Depending on the value of the sample_frequency input, the 22V10 produces a sample clock of 9600 Hz or 19200 Hz clock on its output. It is important to note that the sample clock is much longer than the system clock. The relative length of the sample clock to the system clock is shown in Figure 6.
The conversion of the input analog signal to a digital signal that the pitch shifter can manipulate is done using the AD670 8-bit ADC made by Analog Devices Corporation. Once the AD670 has been signaled to sample, it takes approximately 10 microseconds to perform a conversion. After 10 microseconds, the data representing the sample from the AD670 can be written to the output pins by sending the AD670 appropriate control signals. A timing diagram for the conversion cycle used in the pitch shifter is shown in Figure 8.
The conversion cycle is started by pulsing both the R/n_W and the n_CE/n_CS lines low. The conversion is finished a maximum of 10 microseconds later. The output pins are in a high impedance state until the n_CE/n_CS line is brought low. A few nanoseconds later, the AD670 drives the output pins with an 8 bit value representing the input voltage.
The MCU controls the operation of the pitch shifter by asserting various control signals to the other modules. The MCU is created from two 28F256A (8 bit address, 8 bit data word) Programmable Read Only Memories (PROMs). When composed together, the two PROMs create 16 bit instructions at each 8 bit address.
If the instruction at the current address asserts control signals, the MCU will execute the instruction at the next address on the following clock cycle. If the instruction at the current address is a jump, the MCU executes the instruction at an address specified in the current instructions data. Jumps may be conditional (CJMP) or unconditional (JMP). Conditional jumps only occur if a certain condition (such as the sample clock currently being high) is true. Unconditional jumps always force the MCU to jump to a new instruction because their condition (0b111) is wired high and hence always true.
Figure 9 shows the instruction format of each 16 bit MCU instruction. The most significant bit of the instruction is the opcode. If the opcode is a 1, the instruction is called an ASSERT instruction. In each ASSERT instruction, the low 8 bits of the instruction control signals to rest of the pitch shifter. More than one control signal may be asserted in each ASSERT instruction. If the opcode is a 0, the instruction called a CJMP instruction. When executing a CJMP instruction, the MCU compares the value at the conditional address specified by bits 14 through 12 of the instruction. If the condition value is high (logical true) the next instruction that the MCU executes is the address specified in the lowest 8 bits of the instruction. If the conditional value is false, the MCU executes the instruction at the next address.
The hardware to implement the MCU is shown in Figure 10. The two 74LS163 counters keep track of the current instruction being executed. If the opcode is a 0b0, then the conditional mux is enabled. The input that the conditional mux selects is specified by the conditional address, located in bits 14 through 12 in the instruction. If the condition selected is high, the 74LS163s are loaded with the value contained in the last 8 bits of the instruction.
Figure 11 shows a flow chart representing the MCU program in the pitch shifter. Figure 12 shows the assembly specification file for the MCU, and Figure 13 shows the actual assembly which implements the pitch shifter.
The MCU is in an infinite loop which begins by looking for the falling edge of the sample clock. When the falling edge of the sample clock occurs, the MCU signals the A/D to take a sample and the Storage Unit to increment the current count.
The A/D can't be read from until it has finished converting the sample. Instead of querying the A/D, the MCU assumes that the sample clock is so long compared to the A/D conversion time by the rising edge of the sample clock, the sample may be read with impunity.
The RAM used in the storage unit is sensitive to changes on its address lines. Even worse, the address lines are controlled by a Cypress 374I CPLD (Complex Programmable Logic Device) which has glitchy outputs. To avoid errors due to changing address lines during a read or write to or from the RAM, the address count is incremented a long time before any RAM access is attempted. The MCU then waits for the rising edge of the sample clock to proceed.
On the rising edge of the sample clock, the current value is read from the A/D into the sample buffer of the RAM. If the user wishes the original signal to appear on the output, the MCU loads the accumulator with the newly written contents of the sample buffer. If the user wishes the shifted signal to appear on the output, the MCU tells the Storage Unit to address the shift buffer, and the MCU loads the accumulator with the data from the shift buffer.
The Storage Unit is responsible for generating the RAM addresses of the current sample and shift buffers. The Storage Unit addresses the sample buffer by default, but when the shift_buff signal is asserted, the Storage Unit addresses the Shift Buffer. The MCU does not worry about keeping track of buffers, maximum buffer sizes, or swapping the buffers. All of the book keeping is done internally by the Storage Unit. A block diagram of the Storage Unit is shown in Figure 14.
The sample buffer and the shift buffer are each comprised of 2048 (11 bits of address) locations with 8 bits of data each. A twelfth address bit is used to distinguish between the two buffers. Hence, a total of 4096 locations out of a possible 8192 in the RAM are used. When the sample buffer is full, the roles of the two buffers are swapped by reversing the twelfth address bit.
The pitch multiplier counter's 6 bit increment is used to increment the shift buffer's address. The increment is added to an internal 15 bit counter, and the top 11 bits of this internal counter are used as the shift buffer address. Pressing the shift_up button causes the increment to be increased, and pressing the shift_down button causes the increment to be decreased.
When the increment is 0b010000, the shift buffer address is the same as the sample buffer address (except for the twelfth bit) and the shifted signal is the same as the sampled signal. When the increment is greater than 0b010000, some shift buffer addresses are skipped. By skipping shift buffer addresses, the signal is squished in time and the pitch is raised. When the increment is less than 0b010000, some shift buffer addresses are repeated, causing the sample to be stretched out and lowering the pitch. Figure 15 shows the relation of the internal counter to the shift buffer address.
If the user wishes to use smaller buffers, the buff_size counter can be set to something other than 1111. When the top four sample buffer address bits are equal to the buffer size, the internal shift and sample counters are reset to zero, and their roles are reversed.
The VHDL code to implement the Storage Unit is shown below. Figure 16 shows the code for the pitch multiplier counter module responsible for generating the increment signal. Figure 17 shows the code for the sequencer module which is responsible for storing and incrementing the internal shift and sample addresses, as well as detecting when the buffers are full and swapping their roles. Figure 18 is the entity which combines both the pitch multiplier counter and the sequencer into a CPLD and assigns pin numbers to the input and output signals.
The Accumulator was the most difficult part of the pitch shifter to implement. The Accumulator is necessary to allow a mixture of both the original and the shifted signals to be outputted at the same time. The original design of the pitch shifter is shown in the left of Figure 19. As implemented, the original design could not be made to work. The right of Figure 19 shows the accumulator that was actually built and demonstrated.
The original Accumulator design called for two independently loadable registers, whose output was averaged and sent to the D/A converter as shown in Figure 20. The original design was implemented in VHDL and synthesized on a Cypress 374I CPLD. The VHDL code is shown in Figure 21.
Despite a large amount of time spent trying to get the Accumulator to work, its behavior never matched the specification given in by the VHDL code. The reason for the discrepancy between description and operation was never ascertained. The discrepancy was the source of much confusion and anguish in the debugging process.
When it became clear that the original design could not be made to work, an alternate approach was undertaken. The alternate approach approximates the original accumulator design. The alternate Accumulator only has a single load signal which averages the current output with the current input to yield the new output. The output of the original design does not depend not on past signals as does the output of the alternate design, therefore the original design was a better. However, the approximate design could be implemented. The implemented Accumulator design is shown in Figure 22, and the VHDL code for the 22V10 8 bit register is shown in Figure 23.
The external control signals from the MCU to the RAM, to the Accumulator, and to the A/D were synchronized using the scheme shown in Figure 24. The assertion logic presented in Figure 24 does not assert the output signals if the opcode is 0 (jumps) and it holds the asserted signal for the whole of the following clock cycle
The extra synchronization circuitry was necessary for two reasons. First, both the output from the program counters (74LS163, see Figure 10) and the output from the PROMs is glitchy. To avoid problems with glitches on the control signals, the PROMs are only enabled for the last half of the clock cycle. The rest of the pitch shifter only looks at the values of the control lines at the rising edge of the next clock cycle, so the synchronization circuitry holds the asserted signal for the next clock cycle.
The second reason that the circuitry in Figure 24 is necessary is that if the MCU is jumping to a new address, the address is contained in the same bits of the 16 bit instruction as the assert signals. The synchronization circuit forces the output signals to 0 if the opcode indicates that a jump is occurring (opcode = 0) by tieing the n_reset pit to the opcode.
Although the 6264 RAM is advertised as a static RAM, if the address lines change while the RAM is active (CS1 high and n_CS2 low), the I/O pins suddenly drive the bus regardless of the state of the output enable (n_WE) signal. Originally unaware of this subtly, the design called for the RAM to always be enabled by wiring the CS1 pin high and the N_CS2 pin low. After observing unaccounted behavior from the RAM and re-reading the lab 2 handout, the n_CS2 pin was wired to the clock. The RAM therefore was active only for the last half of each clock cycle. Enabling the RAM on the last half of the clock cycle gave the address lines time to stabilize before enabling the RAM. Figure 25 shows the eventual RAM timing that was used.
Implementing lab 3 involved two painful but separate experiences. Both experiences could have been significantly more painful had the design been more complex. In addition, debugging was aided by implementing the pitch shifter one module at a time. Each module was thoroughly tested before the next module was implemented. By building one module at a time, the possible source of errors were kept to a minimum.
Provided with lab 3 was a simple MCU program and assertion logic which caused lights to blink in a particular order. This program was supplied to give an indication that the MCU was working properly before the other modules of the pitch shifter were implemented. Figure 26 shows the specification file for this test program. Figure 27 shows the assembly code for the test program, and Figure 28 shows the VHDL code for the assertion logic. All three files were provided by the 6.111 staff.
The MCU was built without the aid of detailed wiring diagrams with pin number. While the MCU eventually worked, several wiring mistakes were made that were hard to track down. When implementing the rest of the system, detailed diagrams were made before wiring. Because of these wiring diagrams (attached), very few wiring mistakes were made in the rest of the pitch shifter.
The VHDL code provided for the original Accumulator design is very simple and should have worked. Instead, it cost a day of debugging time to realize that it did not. It is suspected that the problem arose because the Accumulator and Storage Unit were placed on separate CPLDs and a signal conflict occurred. The suspicion has not been verified, and is suspect itself because The Storage Unit worked correctly. After checking all of the other modules comprising the pitch shifter, the Accumulator VHDL code was compiled with different options. The Accumulator still did not work as intended, but it behaved differently, leading to the conclusion that something was wrong in the process between the VHDL implementation and hardware implementation. After trying to make the CPLD do what was intended for half a day, the alternate (and less optimal) approach was adopted. The lesson learned from this debacle was that misbehaving CPLDs are very hard to debug because the internal signals are not visible, and hence can not help with debugging.
The pitch shifter is a very complex device. Even though thoughtful care was taken in the original design, problems inevitably arose. By keeping the device as simple as possible and still retaining the desired functionality, time was saved in the debugging process. The majority of the time spent implementing the pitch shifter was in the debugging process, an all too common occurrence. Detailed wiring diagrams cut down the debugging time as did careful wiring. It took a major effort to determine that the Accumulator was malfunctioning. Once the malfunction was isolated, however, only a few hours were necessary to create an alternate design and produce a working Accumulator. Overall, simplicity and a good debugging strategy are essential to the design and implementation of a complex system.