Week Seven: OOP Asteroids and tone detection

Working (but slow) Sound Controlled “Asteroids” with PDM mic on Raspberry Pi Pico by Digital Maker CIC

Using Martin’s Ada Fruit FFT experiment (week 5) and Phil’s OOP Asteroids, we started to get some sound controlling the Spaceship!

This week has proven to be quite a challenge.

We wanted to combine the knowledge gained previously from Phil’s Asteroids and Martin’s FFT code to make a game controlled by sound. Phil continued his work on the Asteroids and control of a small rocket capable of firing projectiles.
Martin looked at isolating individual tones, these could then be used for the control of the rocket. Three tones were needed for, boost, rotate and fire. He chose an online tone generator https://www.szynalski.com/tone-generator/ to generate the three required tones.

A tutorial https://learn.adafruit.com/light-up-reactive-ukulele/software found online was particularly helpful. This led to the following CircuitPython code:

"""FFT Tone detection
to work with ulab on Raspberry Pi Pico"""

from time import sleep
import audiobusio
import board

from ulab import numpy as np
from ulab.scipy.signal import spectrogram
import array

#---| User Configuration |---------------------------
SAMPLERATE = 16000
SAMPS = 256

fft_size = 256

# ============== make the PDM mic object =============== #

mic = audiobusio.PDMIn(board.GP27_A1, board.GP0,
                                sample_rate=SAMPLERATE, bit_depth=16, startup_delay=0.05, mono=True)

#use some extra sample to account for the mic startup
samples_bit = array.array('H', [0] * (fft_size+3))

# Main Loop
def main():
    max_all = 10
    while True:
        mic.record(samples_bit, len(samples_bit))
        samples = np.array(samples_bit[3:])
        spectrogram1 = spectrogram(samples)
        # spectrum() is always nonnegative, but add a tiny value
        # to change any zeros to nonzero numbers
        spectrogram1 = np.log(spectrogram1 + 1e-7)
        spectrogram1 = spectrogram1[1:(fft_size//2)-1]
        peak_idx = np.argmax(spectrogram1)
        peak_freq = peak_idx * 16000 / 256 / 2
        #print((peak_idx, peak_freq, spectrogram1[peak_idx]))
        
        if peak_freq > 1000 and peak_freq < 1150:
            print("Fire")
        elif peak_freq > 500 and peak_freq < 550:
            print("Rotate")
        elif peak_freq > 225 and peak_freq < 275:
            print("boost")
        else:
            print(peak_freq)
        sleep(0.1)

main()

The code worked reasonably well so the next task was to incorporate it into Phil’s asteroid code.

This proved quite challenging as once the two were combined together. The frequency returned by the microphone was always 0.0.

Various methods were tried, adding the Mic as object, calling it as a function and directly adding it to the main code.py. Adding it to directly to code.py was the final stage but still wasn’t providing the desired results.

Next we resorted to commenting out elements of code, the final conclusion was that some of the print statements were causing timing errors that caused the spectrogram to return values of 0.0. We aren’t quite sure why but once these were reduced to a minimum the code started working.

The final code with libraries is available as a zip file here.

It’s quite slow – the FFT calculations might be pushing the Pico to the edge of its computing power! Phil is exploring how to hone the OOP too, to see if he can eek out a few more milliseconds

Week 6: OOP asteroids

With the use of the FFT code to recognise sounds / frequencies, we had an idea that a game could be controlled with specific sounds

With the use of the FFT code to recognise sounds / frequencies, we had an idea that a game could be controlled with specific sounds… Phil started to look at OOP again (Object Orientate Programming) & wanted to create a game of “Asteroids” but instead of pressing a button to “shoot” , making a laser “pew pew” sound would activate the shooting … Ambitious? has it been done before?

Phil loves a bit of OOP, and recently bought the fantastic book by Dusty Phillips and with the previous lessons in creating vectorIO polygons, he started out making a large polygon with randomly spread out “nodes” (thinking about large “initial” asteroids and how they would break down & split into smaller objects, with their own properties of speed, direction (vectors), edges etc).Phil’s 1st attempt at creating Asteroids

There are a few bugs, mostly from edge detection / re-draw – but it’s looking promising!

Week 5: FFT Waterfall

Martin has been following the #AdaFruit example of FFT

Martin has been following the #AdaFruit example of FFT ( fast Fourier transform) to convert the microphone input into a visual representation (spectrum) with great results! The code for this can be downloaded here