Skip to content
Rex St. John

Get in touch

A Big Month For AI, Voice and FPGAs

I have added you to this list because we met in person and I felt strongly that you might appreciate hearing insights and developments from the “front lines” of the internet of things.

Originally published on the Taroko blog

A Big Month For AI, Voice and FPGAs
Tags
  • newsletter
  • iot

Originally published at the Taroko newsletter on May 8, 2019.

Hello Taroko newcomers,

I have added you to this list because we met in person and I felt strongly that you might appreciate hearing insights and developments from the “front lines” of the internet of things. I try to keep these sparse yet at a regular cadence and only when I have a critical mass of interesting news.

Please feel free to share any comments, suggestions or feedback about what you think or want to hear more about.

Today’s Topic: Recent Trends in AI, Voice and FPGA

Image from the original post

It is only Wednesday and the volume of developments focused around voice, AI and FPGA technology I have witnessed in the last two weeks is surprisingly high. I wanted to take a moment to talk about some of these developments, why they are of interest and how they fit into major trends occurring at the edge in AI.

Recap of Hardware Pioneers “Building Smarter IoT Products With Edge AI” Event

Recently, I attended an interesting event organized by Hardware Pioneers in Seattle focused on AI at the edge here in Seattle. We had speakers from Microchip, XNOR and Swim.ai talking about developments in AI hardware and software technologies. I am borrowing a couple of the slides here to talk about what I felt were the highlights.

Trend #1: FPGA and MCU Hybrids

Image from the original post

Image from the original post

Bob Martin was the speaker on behalf of Microchip, where he is acting Senior Staff Engineer, and covers the Arduino relationship (as well as other wizardry). Bob had some simply wonderful insights based on his experience that are worth mentioning.

One topic I have seen a lot of movement in is low-power, low-cost and open-source FPGA tooling for edge AI and voice. Bob shared a “Thumb-stick” format board which combined an FPGA and MCU together to provide super low-power AI-like processing capabilities for non-camera related activities.

I liked this quite a lot and thing we will be seeing more things like this. The combination of an MCU plus an FPGA has some benefits and flexibility over an ASIC which the industry seems to want to explore further.

Trend #2: AI != Cameras and Object Detection

Image from the original post

This is another big one: AI is not Object Detection. I remember watching how the term “IoT” came to mean “Home Automation” in the minds of most developers and end users. It makes sense - The only time most people are consciously aware that they are interacting with an IoT system is in their homes. As a result, the term “IoT” gradually warped to mean “Home Automation” since turning lights on and off and flushing toilets was more relevant and tangible to most users than anything happening in some remote factory or hidden in a street light.

The same has happened to the term “AI.” Detecting Objects is only part of the story, there are many, many hidden use cases where AI makes sense that doesn’t rely on camera input or data. Bob mentioned that TensorFlow and other approaches have been built specifically from the ground up to be based on 2D camera data, which doesn’t make sense for some of the myriad operations one might want to perform, such as detecting a vibration signaling that an engine needs maintenance before it fails.

Trend #3: XNOR and their Software Enhancements (Software Makes AI Better!)

Image from the original post

XNOR, an AI startup incubating in the Paul Allen AI accelerator, also spoke. They have been busy, and demonstrated videos of their software running, capable of recognizing and rapidly picking out objects by the dozen in real time without the need for much fancy acceleration.

If you can take anything away from XNOR and their work, it is that an awful lot can be done just by optimizing software without even needing to get to custom hardware (I think you can get even further by combining both approaches!).

They also have announced in the past a super low-cost FPGA-based AI camera that can run on solar power.

Image from the original post

XNOR have been in the news also recently with some benchmarks that have been performed that show quite favorable performance for object detection on a Raspberry Pi over some of the accelerators on the market: <https://medium.freecodecamp.org/perf-machine-learning-on-rasp-pi-51101d03dba2?gi=242f821ca43a>.

This is interesting news because it is another demonstration of the performance in AI that can be wrangled out of cheap hardware with some software magic.

Swim.ai and Simon Crosby

Image from the original post

Now for the most famous, intelligent and wise speaker of the event, Simon Crosby who is CTO of Swim.ai. I am also fairly certain he is in this mailing list somewhere.

The gist of the Swim.ai talk is that sending data to the cloud to process it is a near total waste of time and compute resources (am I being too kind?). Giving some examples of work Swim.ai and his team have done with street light and traffic data in Palo Alto, Simon demonstrated some benchmarks which indicated that many of the things you would want to “do AI on” and process can happen in a fraction of the time at the edge, even on the most cheap-o hardware imaginable.

The data he showed was quite alarming if you think that a lot of this workload is going to the cloud. It seems to be quite the opposite. I think there needs to be a new business model: Price Per Node, or something like that, in the IoT space to support all of this.

The other point Simon made is that there are so many devices, no mass of humans can ever be expected to write all of the necessary software that is required. Swim are working on ways to automatically generate the needed solutions at large scale, I have to agree this is sounding like the best way to go.

Image from the original post

In an example, Simon showed that it took 30ms to send traffic data to the cloud for processing and back. That 30ms delay was both wasteful and unnecessary (as well as extremely costly!) compared to just doing your AI locally using a network of dirt-cheap computing devices.

This is largely consistent with gut feelings I have had for a long time. I have sat through so many slide decks and presentations ranting about how every connected car and jet engine is going to stream 4 terabytes of data to the cloud every second that I have become deeply suspicious that (a) anyone wants this data and (b) it isn’t a better idea to just discard 99.9999999% of it out of hand. Who would pay for all this useless data?

Image from the original post

Today, Swim also posted some tweets. You can now view Swim traffic data in your Tesla car! Read the tweet storm to learn more.

AIoT in Anchorage, Alaska - SIGKDD, August 5th

Image from the original post

An announcement was made that a super awesome looking event focused on AI for IoT is coming in August in Anchorage Alaska. The URL is: aiotworkshop.github.io.

I would go but I have to be at Arm in Cambridge at this time. You should go, and get there by taking the cruise up the Alaska coastline, you deserve it. Event is August 5th. Sign up now!

Hardware Pioneers June Seattle Robotics Events

Final note on the Hardware Pioneers event, they are returning to Seattle in June to talk about Robotics. I can’t go because my wife is giving birth that week.

Other Interesting News

Ok, as we draw towards the end, we have some announcements and other interesting things to talk about.

Image from the original post

The first interesting, relevant and amazing thing is this incredible benchmark article produced by Hackster.io author Alasdair Allan. It is the most comprehensive comparison between Google Coral, Intel Neural Compute Stick, NVIDIA Jetson, Raspberry Pi, Google TPU ever produced. It really nicely shows the performance differences between these devices at various tasks, using various inference libraries.

Highly recommend reading this and following Alasdair and Hackster for more like this. Share it out! Support great work!

Matrix Labs Open Sources their FPGA DSP Voice Libraries

Image from the original post

Our friends at Matrix Labs have open sourced their FPGA code for Xilinx to do voice processing on their boards. This is a nice gesture by Matrix Labs which I hope will get more people doing low-power voice on the edge. I believe there is a lot of unexplored territory in Voice at the edge with devices like Matrix Creator.

I would watch Voice FPGA, Open Source FPGA and AI space closely, it seems lots of neat stuff happening.

Final note: Arm uTensor + Google TensorFlow Lite Merge

Image from the original post

One last thing, it was announced today that the Arm uTensor and Google’s TensorFlow Lite projects are going to merge. Another snippet from Alasdair:

“The ability to deploy machine learning models on edge devices opens doors for smarter devices that are power- and bandwidth-efficient. We believe this will usher in an era of TinyML battery-powered devices that can take advantage of low-power wireless communication”

This is a nice synchronization between two complimentary projects and will definitely help more developers do more low-power AI on cheap silicon.

Signing Off

Follow me on Twitter: <https://twitter.com/rexstjohn>.

Please share this link to get more people to sign up if you find this interesting: <https://www.rexstjohn.com/welcome>.