Custom TinyML model for SenseCap Watcher

Introduction to SenseCAP Watcher

The SenseCAP Watcher represents a significant leap forward in intelligent monitoring systems for smart homes and buildings. As the world's first AI-powered physical watcher, it is designed to offer advanced, context-aware monitoring and management capabilities. The Watcher seamlessly integrates voice command functionality with sophisticated sensing technologies, allowing it to execute complex tasks based on real-time analysis of its environment.

SenseCAP Watcher is not just another sensor—it's a comprehensive AI agent that combines multiple functionalities into a single, user-friendly device. It is equipped with a camera, microphone, speaker, and touchscreen, enabling it to see, hear, speak, and display various emotions. Powered by a high-performance ESP32S3 main controller and the efficient Himax WiseEye2 HX6538 AI chip, Watcher excels in image and vector data processing.

The device offers versatile deployment options, from desktop to wall mounting, and includes a battery for flexible installation. Its open-source software infrastructure, provided by the SenseCraft suite, supports on-premise deployment and customization, ensuring that users have full control over their data and system operations.

With the ability to integrate over 300 Grove sensors and actuators, SenseCAP Watcher excels in creating a comprehensive environmental analysis, making it ideal for various applications such as automated home management, security monitoring, and more. Whether you're looking to protect privacy by controlling curtains or manage complex workflows with voice commands, Watcher adapts to meet your needs with high efficiency and privacy assurance.

Technical Specifications

• Main Controller: ESP32S3
• AI Subprocessor: Himax WiseEye2 HX6538 (ARM M55 + U55)
• Camera: Integrated, high-resolution
• Microphone:Built-in for audio capture
• Speaker: Built-in for audio output
• Touchscreen: Integrated for user interaction
• Connectivity:
• Seeed Grove Interface (supports over 300 Grove sensors and actuators)
• I2C Interface
• Power Supply: Battery-powered for flexible installation
• Deployment Options: Desktop and wall mounting
• Software Suite: SenseCraft (open source, supports on-premise deployment)

SenseCAP Watcher combines cutting-edge technology with user-centric design to create a versatile, intelligent monitoring solution that adapts to your specific needs. Whether for enhancing security, managing home environments, or integrating with various sensors, Watcher provides a robust platform for creating smarter spaces

Building a Custom TinyML Model for SenseCAP Watcher

Introduction

This tutorial will guide you through the process of creating a custom TinyML model using Seeed Studio's SenseCraft AI platform and deploying it on the SenseCAP Watcher. The Watcher is an AI-powered device designed for real-time edge inference, supporting models trained on the Grove Vision AI V2, which shares the same Arm Cortex-M55 processor.

Prerequisites

• A SenseCAP Watcher device
• A Grove Vision AI V2 for data collection (recommended)
• A computer with a camera (optional for data collection)
• An internet connection

Step 1: Accessing SenseCraft AI

• Open your browser and navigate to the SenseCraft AI website.
• Click on the Products menu and select SenseCraft AI.

Step 2: Setting Up Model Training

• Choose the type of training you need. It is recommended to use Grove Vision AI V2 for data collection since it shares the same processor as the Watcher.
• If you do not have the Grove Vision AI V2, you can use a computer camera for data collection.

Step 3: Collecting Training Data

• Create and name different categories for classification (e.g., "Car" and "Doll").
• Modify the category tags for better organization.
• Click Hold to Record within the selected category to start capturing images.
• Choose Grove Vision AI V2 as the data collection device.
• Gather as much high-quality data as possible to improve model accuracy.
• Once data collection is complete, proceed to training.

Step 4: Training the Model

• Click on Start Training under the collected classification data.
• Wait for the model to process and generate results.
• Once the training is complete, select the model file.
• Click Save to SenseCraft to store the trained model.

Step 5: Defining Model Details

• Provide a Model Name and Description.
• Write an Introduction and describe the Deployment Process.
• Select SenseCAP Watcher and Grove Vision AI V2 under Supported Devices.

Step 6: Configuring Model Parameters

• Choose the AI Framework based on your model requirements.
• Set the Confidence Threshold (e.g., 80% means the model is 80% confident in predictions).
• Adjust the IoU Threshold (Intersection over Union) to determine bounding box accuracy.
• Click Confirm to finalize settings.
• Navigate to Pretrained Models > My Own Models to view the saved model.

Step 7: Deploying the Model on SenseCAP Watcher

• Open the SenseCraft App and select your Watcher device.
• Locate the Manually Set Up Task option.
• Enable the Use TinyML Model setting.
• Find and select your deployed AI model.
• Adjust the Confidence Threshold and Detection Conditions as needed.
• Assign a Task Name for easy identification.
• Wait for the Watcher to download and deploy the model.

Troubleshooting

• If the model fails to download, reselect it, adjust parameters, and retry.
• Ensure that you have a stable internet connection during the process.

Conclusion

By following these steps, you can create, train, and deploy a TinyML model on the SenseCAP Watcher using the SenseCraft AI platform. This enables real-time AI inference for various applications, from object detection to environmental monitoring. Happy coding!