Adafruit PyPortal UK tide viewer

A little while ago Adafruit produced a learning guide for the PyPortal that displayed tidal data https://learn.adafruit.com/pyportal-tides-viewer/overview unfortunately this was only suitable for use in the USA. I decided to see what changes were required to make a version that could use UK tidal data. In the first instance have a look at the guide and download and play with the code provided

Uk tidal data

A quick search on duck duck go revealed that the Admiralty Maritime Data Solutions, part of the UK Hydrographic office, have a freely available API.

You can register for an account here https://admiraltyapi.portal.azure-api.net/ Once signed up and logged in there is plenty of information about their API and a couple of tools to play with. As part of the process you will receive a couple of keys needed to authenticate you when requesting data.

Details of high and low tide, time and height can be obtained for between one and seven days.

PyPortal

I initially looked at just displaying todays high and low tide, the same as Adafruit. Thinking it would just be a simple matter of plugging in a new API address. A number of days later!!! I had a solution. One major issue was the issue of Authentication. All examples provided by Adafruit passed any authentication strings in the URL Datasource querystring. The Admiralty use an Azure server that is set to receive authentication in a header and not the querystring. A quick check revealed that the default PyPortal library did not support this, although it was supported in libraries further down the stack.

I implemented the functionality in a forked version of PyPortal, this has now been added to PyPortal and is available in all new releases.

Code changes

A copy of my code is provided below. The STATION_ID used is for Peterhead, my nearest tidal station. You will also need to add two keys to the secrets.py file

'Ocp-Apim-Subscription-Key' : 'primary key',
'tidal_secondary' : 'secondary key',

Additional changes are in the pyportal call where we add a headers field.

headers={"Ocp-Apim-Subscription-Key":secrets['Ocp-Apim-Subscription-Key']},

The JSON data returned is in the form of an array so needed to be converted to a DICT format

raw_info = json.loads(raw_info)

The times returned also needed to be reformatted

for i, hi_time in enumerate(tide_info["HighWater"]):
      HI_LABELS[i].text = '{:.5}'.format(hi_time)
for i, lo_time in enumerate(tide_info["LowWater"]):
      LO_LABELS[i].text = '{:.5}'.format(lo_time)

Most of the existing code was reused

Completed code

import time
import board
from adafruit_pyportal import PyPortal
from adafruit_bitmap_font import bitmap_font
from adafruit_display_text.label import Label
import json
from secrets import secrets

#--| USER CONFIG |--------------------------
STATION_ID = "0245"   # tide location, find yours from admiralty website/
HI_COLOR   = 0x00FF00    # high tide times color
LO_COLOR   = 0x11FFFF    # low tide times color
DATE_COLOR = 0xFFFFFF    # date and time color
#-------------------------------------------

# pylint: disable=line-too-long
DATA_SOURCE = "https://admiraltyapi.azure-api.net/uktidalapi/api/V1/Stations/" + STATION_ID + "/TidalEvents?duration=1"
DATA_LOCATION = []

# determine the current working directory needed so we know where to find files
cwd = ("/"+__file__).rsplit('/', 1)[0]
pyportal = PyPortal(url=DATA_SOURCE,
                    headers={"Ocp-Apim-Subscription-Key":secrets['Ocp-Apim-Subscription-Key']},
                    json_path=DATA_LOCATION,
                    status_neopixel=board.NEOPIXEL,
                    default_bg=cwd+"/tides_bg.bmp")

# Connect to the internet and get local time
pyportal.get_local_time()

# Setup tide times font
tide_font = bitmap_font.load_font(cwd+"/fonts/cq-mono-30.bdf")
tide_font.load_glyphs(b'1234567890:')

# Setup date and time font
date_font = bitmap_font.load_font(cwd+"/fonts/Arial-12.bdf")
date_font.load_glyphs(b'1234567890-')

# Labels setup
HI_LABELS = [ Label(tide_font, text="00:00", color=HI_COLOR, x= 40, y= 80) ,
              Label(tide_font, text="00:00", color=HI_COLOR, x= 40, y=165) ]
LO_LABELS = [ Label(tide_font, text="00:00", color=LO_COLOR, x=180, y= 80) ,
              Label(tide_font, text="00:00", color=LO_COLOR, x=180, y=165) ]
DATE_LABEL = Label(date_font, text="0000-00-00 00:00:00", color=DATE_COLOR, x=75, y=228)

# Add all the labels to the display
for label in HI_LABELS + LO_LABELS + [DATE_LABEL]:
    pyportal.splash.append(label)

def get_tide_info():
    """Fetch JSON tide time info and return it."""

    # Get raw JSON data
    raw_info = pyportal.fetch()
    raw_info = json.loads(raw_info)
    # Return will be a dictionary of lists containing tide times
    new_tide_info = {"HighWater":[], "LowWater":[]}
    # Parse out the tide time info
    for info in raw_info:
        tide_type = info['EventType']
        tide_time = info['DateTime'].split("T")[1]
        new_tide_info[tide_type].append(tide_time)

    return new_tide_info

def update_display(time_info, update_tides=False):
    """Update the display with current info."""

    # Tide time info
    if update_tides:
        # out with the old
        for tide_label in HI_LABELS + LO_LABELS:
            tide_label.text = ""
        # in with the new
        for i, hi_time in enumerate(tide_info["HighWater"]):
            HI_LABELS[i].text = '{:.5}'.format(hi_time)
        for i, lo_time in enumerate(tide_info["LowWater"]):
            LO_LABELS[i].text = '{:.5}'.format(lo_time)
    # Date and time
    DATE_LABEL.text = "{:04}-{:02}-{:02} {:02}:{:02}:{:02}".format(time_info.tm_year,
                                                                   time_info.tm_mon,
                                                                   time_info.tm_mday,
                                                                   time_info.tm_hour,
                                                                   time_info.tm_min,
                                                                   time_info.tm_sec)

    board.DISPLAY.refresh_soon()

# First run update
tide_info = get_tide_info()
current_time = time.localtime()
update_display(current_time, True)
current_yday = current_time.tm_yday

# Update daily
while True:
    current_time = time.localtime()
    new_tides = False
    if current_time.tm_yday != current_yday:
        # new day, time to update
        tide_info = get_tide_info()
        new_tides = True
        current_yday = current_time.tm_yday
    update_display(current_time, new_tides)
    time.sleep(0.5)
UK tidal data

Making it graphical

The next stage of the learning guide shows the tidal data plotted over a 24 hour period with tidal times, heights provided every six minutes. The Admiralty doesn’t provide this level of detail. They only provide the high and low water times. I decided to plot them anyway. The code is provided below.

Completed code

import time
import board
import displayio
import json
from secrets import secrets
from adafruit_pyportal import PyPortal
from adafruit_bitmap_font import bitmap_font
from adafruit_display_text.label import Label

#--| USER CONFIG |--------------------------
STATION_ID = "0245"   # tide location, find yours from admiralty website
PLOT_SIZE = 2            # tide plot thickness
PLOT_COLOR = 0x00FF55    # tide plot color
MARK_SIZE = 6            # current time marker size
MARK_COLOR = 0xFF0000    # current time marker color
DATE_COLOR = 0xE0CD1A    # date text color
TIME_COLOR = 0xE0CD1A    # time text color
VSCALE = 20              # vertical plot scale
#-------------------------------------------

# pylint: disable=line-too-long
DATA_SOURCE = "https://admiraltyapi.azure-api.net/uktidalapi/api/V1/Stations/" + STATION_ID + "/TidalEvents?duration=1"
DATA_LOCATION = []

WIDTH = board.DISPLAY.width
HEIGHT = board.DISPLAY.height

# determine the current working directory needed so we know where to find files
cwd = ("/"+__file__).rsplit('/', 1)[0]
pyportal = PyPortal(url=DATA_SOURCE,
                    headers={"Ocp-Apim-Subscription-Key":secrets['Ocp-Apim-Subscription-Key']},
                    json_path=DATA_LOCATION,
                    status_neopixel=board.NEOPIXEL,
                    default_bg=cwd+"/tides_bg_graph.bmp")

# Connect to the internet and get local time
pyportal.get_local_time()

# Setup palette used for plot
palette = displayio.Palette(3)
palette[0] = 0x0
palette[1] = PLOT_COLOR
palette[2] = MARK_COLOR
palette.make_transparent(0)

# Setup tide plot bitmap
tide_plot = displayio.Bitmap(WIDTH, HEIGHT, 3)
pyportal.splash.append(displayio.TileGrid(tide_plot, pixel_shader=palette))

# Setup font used for date and time
date_font = bitmap_font.load_font(cwd+"/fonts/mono-bold-8.bdf")
date_font.load_glyphs(b'1234567890-')

# Setup date label
date_label = Label(date_font, text="0000-00-00", color=DATE_COLOR, x=7, y=14)
pyportal.splash.append(date_label)

# Setup time label
time_label = Label(date_font, text="00:00:00", color=TIME_COLOR, x=234, y=14)
pyportal.splash.append(time_label)

# Setup current time marker
time_marker_bitmap = displayio.Bitmap(MARK_SIZE, MARK_SIZE, 3)
for pixel in range(MARK_SIZE * MARK_SIZE):
    time_marker_bitmap[pixel] = 2
time_marker = displayio.TileGrid(time_marker_bitmap, pixel_shader=palette, x=-MARK_SIZE, y=-MARK_SIZE)
pyportal.splash.append(time_marker)

def get_tide_data():
    """Fetch JSON tide data and return parsed results in a list."""

    # Get raw JSON data
    raw_data = pyportal.fetch()
    raw_data = json.loads(raw_data)
    # Results will be stored in a list that is display WIDTH long
    new_tide_data = [None]*WIDTH

    # Convert raw data to display coordinates
    for data in raw_data:
        _, t = data["DateTime"].split("T") # date and time
        t = '{:.5}'.format(t)
        h, m = t.split(":")         # hours and minutes
        v = data["Height"]               # water level
        x = round( (WIDTH - 1) * (60 * float(h) + float(m)) / 1440 )
        y = (HEIGHT // 2) - round(VSCALE * float(v))
        y = 0 if y < 0 else y
        y = HEIGHT-1 if y >= HEIGHT else y
        new_tide_data[x] = y

    return new_tide_data

def draw_data_point(x, y, size=PLOT_SIZE, color=1):
    """Draw data point on to the tide plot bitmap at (x,y)."""
    if y is None:
        return
    offset = size // 2
    for xx in range(x-offset, x+offset+1):
        for yy in range(y-offset, y+offset+1):
            try:
                tide_plot[xx, yy] = color
            except IndexError:
                pass

def draw_time_marker(time_info):
    """Draw a marker on the tide plot for the current time."""
    h = time_info.tm_hour
    m = time_info.tm_min
    x = round( (WIDTH - 1) * (60 * float(h) + float(m)) / 1440 )
    y = tide_data[x]
    if y is not None:
        x -= MARK_SIZE // 2
        y -= MARK_SIZE // 2
        time_marker.x = x
        time_marker.y = y

def update_display(time_info, update_tides=False):
    """Update the display with current info."""

    # Tide data plot
    if update_tides:
        # out with the old
        for i in range(WIDTH * HEIGHT):
            tide_plot[i] = 0
        # in with the new
        for x in range(WIDTH):
            draw_data_point(x, tide_data[x])

    # Current location marker
    draw_time_marker(time_info)

    # Date and time
    date_label.text = "{:04}-{:02}-{:02}".format(time_info.tm_year,
                                                 time_info.tm_mon,
                                                 time_info.tm_mday)
    time_label.text = "{:02}:{:02}:{:02}".format(time_info.tm_hour,
                                                 time_info.tm_min,
                                                 time_info.tm_sec)

    board.DISPLAY.refresh_soon()

# First run update
tide_data = get_tide_data()
current_time = time.localtime()
update_display(current_time, True)
current_yday = current_time.tm_yday

# Run forever
while True:
    current_time = time.localtime()
    new_tides = False
    if current_time.tm_yday != current_yday:
        # new day, time to update
        tide_data = get_tide_data()
        new_tides = True
        current_yday = current_time.tm_yday
    update_display(current_time, new_tides)
    time.sleep(0.5)

Adafruit PyPortal : CircuitPython Powered Internet Display

I’ve been playing with the Adafruit PyPortal recently, a 3.5 inch touchscreen display with in built functionality to quickly build Internet of Things, IoT, projects.

The first thing I built was a viewer for my Luftdaten Air Quality Sensor. I had connected my sensor to opensensemap as I liked the way that they displayed data. They also provide a really nice API interface.

The code I used to access my sensor is provided below

"""
This example will access the opensensemap API, grab the air quality values
pm10, pm2.5, temperature... and display it on a screen!
if you can find something that spits out JSON data, we can display it
"""
import time
import board
from adafruit_pyportal import PyPortal

# Set up where we'll be fetching data from
DATA_SOURCE = "https://api.opensensemap.org/boxes/5c6ec1ae15451500198f5abe"   # pylint: disable=line-too-long
PM10 = ["sensors",0, "lastMeasurement","value"]
PM25 = ["sensors",1, "lastMeasurement","value"]
TEMP = ["sensors",2, "lastMeasurement","value"]

# the current working directory (where this file is)
cwd = ("/"+__file__).rsplit('/', 1)[0]
# Initialize the pyportal object and let us know what data to fetch and where
# to display it
pyportal = PyPortal(url=DATA_SOURCE,
                    json_path=(PM10, PM25, TEMP),
                    status_neopixel=board.NEOPIXEL,
                    default_bg=cwd+"/IMG_6562.bmp",
                    text_font=cwd+"/fonts/Arial-ItalicMT-17.bdf",
                    text_position=((50, 225), (100, 225), (160, 225)),
                    text_color=(0x00FF00,0x00FF00,0x00FF00),
                    caption_text="PM10, PM2.5, Temperature",
                    caption_font=cwd+"/fonts/Arial-ItalicMT-17.bdf",
                    caption_position=(50,200),
                    caption_color=0x00FF00)

# track the last value so we can play a sound when it updates

while True:
    try:
        value = pyportal.fetch()
        print("Response is", value)
    except (ValueError, RuntimeError) as e:
        print("Some error occured, retrying! -", e)

    time.sleep(180)  # wait a minute before getting again

It’s all done with CircuitPython.

I quite liked the result shown below. The picture in the background is of a local lake.

Big Lottery funded DM CIC workshops start in Peterhead

We have been fortunate to be awarded a Big Lottery grant for 5 Driverless Car workshops in the North East of Scotland! We have arranged with Peterhead Academy & Inverurie Academy to start this term, Peterhead have an Afterschool Club for S2+ and We’ll be working with the Inverurie S3 Engineering class.

We’re running the Driverless Car workshop we piloted with Northfield Academy & Transition Extreme after school clubs (funded by ACC’s U-decide (participatory Budgeting)) – so a five week course, introducing the concepts of Machine Learning & Driverless Car technology.

Week one is always relaxed, fun & creative, with the teams creating their car chassis, camera mounts and raspberry pi placements. The Peterhead pupils were fantastic, engaged, entertained, interested, enthusiastic and inquisitive, all the attributes we love to see in innovators of tomorrow!

Below are a few photos from tonight’s session, we look forward to working with the class for the next 4 weeks!

If you’d like to know more about the Driverless Car workshop, want to get us into your school, or have any other questions about our STEAM activity, do email us! we’re getting busy!

Big thanks to the Big Lottery funding, it’s been invaluable! A lot of kids that wouldn’t usually get this type of STEAM education are now, thanks to this funding.

Driverless Car workshops – a five week course

Digital Maker CIC recently completed two 5-week driverless car workshops in Aberdeen. Northfield academy’s Science Club & Transition Extreme After School Club each had an exciting and challenging set of workshops for children aged 12+. Digital Maker CIC believes that these workshops are a “first” for the UK. Both projects were funded by Aberdeen City Council’s “U-decide” (participatory budgeting project).

The participants learned about Machine Learning, Engineering & design of the 1:16th scale cars, raspberry Pi terminal commands, to interface with the cars, computers    & cloud computing interfaces and driving the remote-controlled cars.

Working in teams of four, the pupils were given tasks of constructing & managing their cars, “training” and improving their driving skills, as the better the car is driven, the better the Machine Learning model will be. The pupils could quickly train & then produce ML models (via cloud computing), giving exciting results, as the cars drove themselves around the 4m2 tracks we use. The pupils quickly learned that the better they drove the cars, the automated driving improved.

The teams also tried to add obstacles (orange cones) into the modelling, creating various results (success & failure), but, with more training & time, the pupils understood that their cars would “get better” with time.

We’d like to thank Northfield Academy & Transition Extreme Sports Ltd for their support & use of space, in order that we could run the workshops. We are working on extending the workshops to Aberdeenshire & eventually have a North East interschools competition for the Driverless Car technology.

If you’d like more information, or want your school to participate in this project, please do get in touch!

AwesomeTech@digital-maker.co.uk

Digital Maker CIC – CPD for the College Development Network

Digital Maker CIC had the privilege of being asked to run a robotics challenge workshop for the CDN (College Development Network Scotland) in Dundee & Angus College’s Gardyne campus on Friday 28th of September for over 20 College Lecturers.

Teams from all over Scotland joined us for a fun day, exploring, building & programming robots in order to compete in various challenges once the teams were ready. We gave each team a CamJam robotics kit with minimal worksheets / instructions and spent around 30 mins building the kits, using raspberry Pi Zeros & piTop Ceeds (to control / interface with the robot rover via  5 meter USB cable). the Edinburgh team took the initiative to make their rover wirelessly controlled, downloading & installing software through their own phone (extra points there!)

Once everyone had built & played with their rovers, controlling it via simple on-screen button & programmable interfaces (Custom Adapted Blockly by Digital Maker CIC), we then added a line detection sensor, giving the participants experience in electronics & GPIO use on the Pi Zero. The challenge was to create a line following algorithm using only 1 sensor & the Blockly interface, to get the team’s rover around a line path in the quickest time. There were a lot of different approaches, using repeat loops, “if statements” and more. One of the most elegant solutions was a simple algorithm by West Lothian College.

All teams produced line following algorithms with varying degrees of success, but, all teams were resilient & spent time honing their algorithms, trying various tweaks & fixes based on their observations, critical thinking & experimentation, which was great to see. The fastest line follower was under 1 minute, the longest (complete circuit) was just under 3 minutes.

Some teams managed to start exploring adding an ultrasonic distance sensor (HC SR04) to tackle the minimal maze we had brought along too, but, time had got the better of us, and after 6 hours of playing, experimenting, trying and learning, time was up!

We’d like to thank Kenji Lamb for organising the event & asking Digital Maker CIC to facilitate the workshop. We are planning to run a 2nd CDN Robotics CPD challenge with Kenji in 2019… so if you are interested, or want more information, please do get in touch!

College Development Network – DM CIC Robotics workshop

We were asked by the College Development Network to put on a robotics workshop for their CPD provision, so we gladly said YES!

Here’s a link to the course in Dundee & Angus College. They have kindly agreed to host the event at their Gardyne campus, and CDN are looking for college staff keen to exercise their engineering, programming and design skills on the 28th of September. 

Full details & links are here : https://www.events.cdn.ac.uk/ehome/362505?&t=30f2d8676e7ed69370facb7507bd3a4e

First Class for Driverless Car workshop in Northfeld Academy

Digital Maker’s Martin & Phil rolled out the first class of the five week Driverless Car workshop in Northfield Academy yesterday, with 20 pupils ranging from S1 – S6.

An introduction to the areas covered & straight onto building the 1/16th scale cars that are used with the Raspberry Pi & Camera. Everyone was fantastic, really productive, inquisitive, collaborative, it was a real pleasure to work with the Pupils. Also a special mention to science teacher, Mr Hunter, his help & assistance during the 2 hour class was brilliant, thank you so much for a great start to what we hope is a very rewarding & challenging class, which we believe is a first for Aberdeen, and Scotland!

Driverless Car Workshop – Call to Participate.

Digital Maker CIC are looking for 20 young people (11-18 years old) to join us for a five week course at Transition Extreme to build, maintain, program and use driverless cars!

Working in teams, we will build a state of the art 1/16th scale car, learn about the technology involved in running an A.I (Artificial Intelligence) robot in order to train & race them!

The 20 spaces are free to join (thanks to the U-Decide funding from Aberdeen City Council), so be quick! This is a first for Aberdeen & Scotland!

The sessions will start on Friday, September 7th & end on October 5th 2018 (4.30-6.30pm)

Participants must be:

  • Over 11 years of age
  • Live in Community Council areas of:
    • Aberdeen City Centre
    • Castlehill & Pittodrie
    • George Street

If you want to be in with a chance of joining us, send us an email to AwesomeTech@digital-maker.co.uk First Come, First Served.

youTube video uploads for Driverless Car R&D

We’ve uploaded 2 videos of the work we’ve done so far on preparing for the Driverless Car workshops we’re running soon.

We’ve documented the first “track” (Masking tape on the floor)

and then the black paper track + 2 different types of tape (for higher contrast).

Driverless Car Workshop development

We’ve been working in Aberdeen’s Transition Extreme to develop & test our Driverless car workshop, with great success. We’re learning a lot, honing the car, coding, track & driving techniques…

Here’s a little bit of footage from the car on our last test. We’d developed a more contrasted track with black paper & coloured tapes, which is providing excellent images for the processor / AI.