Visual Perception Engine: Modular, GPU-Accelerated Framework for Real-Time Visual Inference in Robotics

About this technology

Overview

The Visual Perception Engine is a modular, GPU-accelerated framework designed for real-time visual inference tasks in robotic applications. It integrates a vision foundation model (DINOv2) with multiple model heads to perform monocular depth estimation, semantic segmentation, and object detection at 30Hz. The system employs separate processes for the foundation model and each model head, communicating through custom GPU tensor queues to ensure efficient data exchange. Supporting both TensorRT and PyTorch models, the engine offers flexibility in deployment and performance optimization. Its runtime configurability allows users to adjust model parameters and inference rates dynamically. Additionally, the provided ROS2 (Humble) node enables straightforward integration into existing robotic systems, making the Visual Perception Engine a versatile tool for enhancing robotic perception capabilities.