Using Martin’s Ada Fruit FFT experiment (week 5) and Phil’s OOP Asteroids, we started to get some sound controlling the Spaceship!
… getting there… #oop #circuitPython with #pdm mic listening for specific frequencies to control a game on the #raspberryPi #pico . again, Phil apologises for his “singing”. Hz control rotation, boost & firing. pic.twitter.com/cdSTmsGFgH
— Digital Maker CIC (@digitalMakerCIC) April 19, 2022
This week has proven to be quite a challenge.
We wanted to combine the knowledge gained previously from Phil’s Asteroids and Martin’s FFT code to make a game controlled by sound. Phil continued his work on the Asteroids and control of a small rocket capable of firing projectiles.
Martin looked at isolating individual tones, these could then be used for the control of the rocket. Three tones were needed for, boost, rotate and fire. He chose an online tone generator https://www.szynalski.com/tone-generator/ to generate the three required tones.
A tutorial https://learn.adafruit.com/light-up-reactive-ukulele/software found online was particularly helpful. This led to the following CircuitPython code:
"""FFT Tone detection
to work with ulab on Raspberry Pi Pico"""
from time import sleep
import audiobusio
import board
from ulab import numpy as np
from ulab.scipy.signal import spectrogram
import array
#---| User Configuration |---------------------------
SAMPLERATE = 16000
SAMPS = 256
fft_size = 256
# ============== make the PDM mic object =============== #
mic = audiobusio.PDMIn(board.GP27_A1, board.GP0,
sample_rate=SAMPLERATE, bit_depth=16, startup_delay=0.05, mono=True)
#use some extra sample to account for the mic startup
samples_bit = array.array('H', [0] * (fft_size+3))
# Main Loop
def main():
max_all = 10
while True:
mic.record(samples_bit, len(samples_bit))
samples = np.array(samples_bit[3:])
spectrogram1 = spectrogram(samples)
# spectrum() is always nonnegative, but add a tiny value
# to change any zeros to nonzero numbers
spectrogram1 = np.log(spectrogram1 + 1e-7)
spectrogram1 = spectrogram1[1:(fft_size//2)-1]
peak_idx = np.argmax(spectrogram1)
peak_freq = peak_idx * 16000 / 256 / 2
#print((peak_idx, peak_freq, spectrogram1[peak_idx]))
if peak_freq > 1000 and peak_freq < 1150:
print("Fire")
elif peak_freq > 500 and peak_freq < 550:
print("Rotate")
elif peak_freq > 225 and peak_freq < 275:
print("boost")
else:
print(peak_freq)
sleep(0.1)
main()The code worked reasonably well so the next task was to incorporate it into Phil’s asteroid code.
This proved quite challenging as once the two were combined together. The frequency returned by the microphone was always 0.0.
Various methods were tried, adding the Mic as object, calling it as a function and directly adding it to the main code.py. Adding it to directly to code.py was the final stage but still wasn’t providing the desired results.
Next we resorted to commenting out elements of code, the final conclusion was that some of the print statements were causing timing errors that caused the spectrogram to return values of 0.0. We aren’t quite sure why but once these were reduced to a minimum the code started working.
The final code with libraries is available as a zip file here.
It’s quite slow – the FFT calculations might be pushing the Pico to the edge of its computing power! Phil is exploring how to hone the OOP too, to see if he can eek out a few more milliseconds
