Hardware components | ||||||
 |
| × | 1 | |||
Industrial automation is rapidly evolving, with manufacturing companies increasingly adopting Industry 4.0 practices to become smarter and more efficient. Simultaneously, advancements in hardware have made computational power more accessible and compact.
Leveraging both trends, we have developed a real-time bolt detection and counting system using the NVIDIA Jetson Nano. This solution not only involves the development of robust machine learning models but also their optimization and deployment directly on edge devices like the Jetson Nano - enabling transformative automation in industrial processes.
Execution StepsDataset Collection: The dataset was collected using the Mobile camera and USB camera, capturing images of industrial bolts arranged in various positions. The bolts were placed at different angles and under multiple lighting conditions to simulate real-world industrial environments. This diversity ensures that the model can generalize well and make accurate predictions even when external conditions vary, such as in low light or from unusual viewing angles.
Data Annotation: As the objective is to detect and count industrial bolts, we used Edge Impulse application to create annotations to mark each bolt in the image. Annotation was done manually for higher accuracy using rectangular boxes around each bolt. Proper labeling is critical to train an object detection model that can identify and differentiate multiple bolts in a single frame.
Feature Extraction: We performed feature extraction to convert raw image data into meaningful features. This step includes normalization, resizing, and conversion to RGB channels depending on the selected model pipeline. These features are critical inputs for training the object detection model.
ML Model: We trained the model using MobileNetV2 SSD FPN-Lite 320x320, a highly efficient object detection architecture tailored for edge devices. This model architecture is ideal for detecting small objects like industrial bolts in complex scenes.
- MobileNetV2 serves as the feature extractor, offering a lightweight and fast backbone.
- SSD (Single Shot Multibox Detector) allows for real-time object detection by predicting bounding boxes and class scores in a single forward pass.
- FPN-Lite (Feature Pyramid Network Lite) improves detection of small-scale objects by aggregating features from multiple levels of the network.
- The input image resolution of 320x320 balances detection accuracy and computation efficiency for edge deployment.
Training Configuration:
- Number of training cycles: 25
- Learning rate: 0.15
- Batch size: 32
- Training processor: CPU
- Validation set size: 20% of total dataset
- Data split method: Randomized split
- Quantization: INT8
Post-training, the model was profiled to run efficiently on edge hardware using INT8 quantization, further reducing its memory footprint and inference latency. This configuration allowed us to create a compact yet accurate model capable of running in real-time on resource-constrained devices like the NVIDIA Jetson Nano.
ML Model Conversion: After successful training, the model was converted into a format compatible with edge devices. Trained ML Model is converted to TensorFlow Lite and custom C++ library formats, depending on the deployment target. In this case, the model was converted to TensorRT-compatible format for optimized performance on the Jetson Nano.
ML Model Deployment on Nvidia Jetson Nano: The deployment environment was prepared by flashing Ubuntu 18.04 onto the NVIDIA Jetson Nano using NVIDIA SDK Manager. This ensured a stable and compatible base system for AI model inference.
After the OS setup, Python 3.6 was installed and configured to support the model's runtime requirements. The converted model was integrated with custom C and Python scripts to enable real-time inference directly on the device.
The model was optimized using TensorRT, NVIDIA’s high-performance deep learning inference library. This allowed the Jetson Nano to leverage its onboard GPU for accelerated processing. A lightweight inference application was created to handle camera input, image preprocessing, model execution, and output visualization - all in real-time.
This deployment setup ensures that the system runs autonomously at the edge, delivering low-latency detection and bolt counting without needing an internet connection.
ML Model Inference: Once deployed, the model performs real-time inference on video frames. It detects and counts bolts present in the field of view and overlays bounding boxes on them. The output includes the count of bolts and their locations, which can further be used for quality control, automated inspection, or operational analytics in an industrial setting.
Comments