Idea
How Does it Work
Device
Data Collecting
Collected data
Adding data to get a better result
Training Model
Testing Model
Video of the Project

•

Rabia Batool

Published November 11, 2022 © GPL3+

Control Laptop using keyword spotting using Edge Impulse

Learn how to create a voice control of laptop using Arduino ble 33 sense and edge impulse using keyword spotting

IntermediateProtip10 days249

Control Laptop using keyword spotting using Edge Impulse

Things used in this project

Hardware components

Arduino Nano 33 BLE Sense

Software apps and online services

Edge Impulse Studio

Arduino IDE

Jupyter Notebook

Story

It seems distractive in an office environment or university auditorium when a person is giving a presentation and moving towards the dice and switching the slides one after the other.

It is considered obligatory in our university presentations to designate a person to manage the display and switch slides. Thence I decided to resolve the problem by "Keyword Spotting"

Whether we are in the working place or university, the person will say start the display of a meeting, forward or go to next slide. Voice control makes this process more intuitive, by allowing users through voices to say, “start the Presentation” to get started. Then, when a presentation ends a presenter can say “, Stop the video” to turn attention away from the display and onto the discussion.

Idea

How Does it Work?

Collecting data and train model using an NN classifier made in Edge Impulse Studio, I can classify keywords said by different people. The device I used is small memory footprint and has low computational cost along with high precision.

Device

Data Collecting

I spent days and nights collecting data from multiple people, family, friends and from universities to make model more robust I also took some commands from siri by saying these keywords.

Collected data:

Adding data to get a better result

Since the result does not satisfy me, I added the data using my smartphone and micro controller

Training Model

Testing Model

After that, I applied Edge Impulse's EON Tuner, which produced perfect test accuracy of about 91%, which is far better than it would have been without it. Making the optimal model for the datasets was made simple and convenient by using Edge Impulse Studio, which also gave the tools for data to model deployment and inference. On Edge Impulse, you can examine how the model and dataset were trained. https://studio.edgeimpulse.com/public/128033/latest

Video of the Project

Python Code

import serial #Serial imported for Serial communication
import time #Required to use delay functions
import pyautogui

ArduinoSerial = serial.Serial('com3',9600) #Create Serial port object called arduinoSerialData
time.sleep(2) #wait for 2 seconds for the communication to get established

while 1:
    incoming = str (ArduinoSerial.readline()) #read the serial data and print it as line
    print (incoming)
    
    if 'Play' in in coming:
        pyautogui.typewrite(['space'], 0.2)

    if 'Stop' in incoming:
        pyautogui.typewrite(['space'], 0.2)

    if 'Backward' in incoming:
        pyautogui.hotkey('ctrl', 'left')  

    if 'Forward' in incoming:
        pyautogui.hotkey('ctrl', 'right') 

    if 'Vup' in incoming:
        pyautogui.hotkey('ctrl', 'down')
        
    if 'Vdown' in incoming:
        pyautogui.hotkey('ctrl', 'up')

    if 'Next' in incoming:
        pyautogui.hotkey('n')
 
    incoming = "";

Credits

Shanzay Jamil

1 project • 0 followers

Rabia Batool

0 projects • 0 followers

Thanks to Rabia Batool.

Control Laptop using keyword spotting using Edge Impulse

Things used in this project

Hardware components

Software apps and online services

Story

Idea

How Does it Work?

Device

Data Collecting

Collected data:

Adding data to get a better result

Training Model

Testing Model

Video of the Project

Code

Python Code

Building a Voice Command control of Screen using Keyword Spotting

Credits

Shanzay Jamil

Rabia Batool

Comments

Embed the widget on your own site

Control Laptop using keyword spotting using Edge Impulse

Control Laptop using keyword spotting using Edge Impulse

Things used in this project

Hardware components

Software apps and online services

Story

Idea

How Does it Work?

Device

Data Collecting

Collected data:

Adding data to get a better result

Training Model

Testing Model

Video of the Project

Code

Python Code

Building a Voice Command control of Screen using Keyword Spotting

Credits

Shanzay Jamil

Rabia Batool

Comments

Related channels and tags