COMP544 - TinyML Wake-Word: UP and DOWN detection

Goal

Since I am curious about how digital assistants work, such as Google Assistant, Apple's Siri, and Amazon Alexa, I choose Wake-Word detection as my TinyML project. In this way, I could understand why my iPhone can always magically detect "Hey Siri." My goal is to train a new model that can recognize "UP" and "DOWN, " I will apply it to my Arduino device. Then, when I say "UP, " the device can turn on the green light; when I say "DOWN, " the device can turn on the red light, and if I say other words, it can turn on the blue light. The most amazing part is that I can just use this little embedded device combined with Tensorflow lite so that my device can run machine learning that detects the words I say.

Approach

To achieve my goal, I breakdown the project into several steps.

1. Training a new model with "UP" and "DOWN" words on Google Colab

1.1. Modify WANTED_WORDS in the configuration

1.2. download the dataset and begin training

1.3. Test the accuracy of the new model, and my model's accuracy is around 88.2%

1.4. Successfully generate a new model to identify "UP" and "DOWN" words.

2. Deploy the new generated C array to my Arduino Nano 33 Sense board.

2.1. First of all, we need to modify kCategoryLabels in micro_features_micro_model_settings.cpp

const char* kCategoryLabels[kCategoryCount] = {
    "silence",
    "unknown",
    "up",
    "down",
};

2.2. Secondly, we need to modify the condition for verifying which command is being detected in arduino_command_responder.cpp

if (found_command[0] == 'u' && found_command[1] == 'p') {
      last_command_time = current_time;
      digitalWrite(LEDG, LOW);  // Green for up
    }

if (found_command[0] == 'd' && found_command[1] == 'o') {
      last_command_time = current_time;
      digitalWrite(LEDR, LOW);  // Red for down
    }

if (found_command[0] == 'u' && found_command[1] == 'n') {
      last_command_time = current_time;
      digitalWrite(LEDB, LOW);  // Blue for unknown
    }

2.3 Lastly, we need to use our new generated array to replace the old array in

micro_features_model.cpp

2.4 Click upload on Arduino IDE!

Results

• When I say "UP", the green light will flash and the serial monitor will display a "Heard up" message.
• When I say "DOWN", the red light will flash and the serial monitor will display a "Heard down" message.
• When I say words that the board cannot recognize, the blue light will flash and the serial monitor will display a "Heard unknown" message.

Demo

The video is a demo of my TinyML wake-word project. You can see when I say "UP" and "DOWN", the device is recognized these two words. Also, the serial monitor shows it Heard what kinds of words.