AI Project
Open-Vocabulary Segmentation Multi-Object Tracking & Counting TensorRT FP16

Open-Vocabulary Video Analytics System: Segmentation, Tracking, & Counting

Author: Ching-Chi Yeung

End-to-end open-vocabulary video analytics combining YOLOE-26 (YOLOE-26l variant) zero-shot segmentation, multi-object tracking, and virtual tripwire line-crossing object counting from text prompts. Supports switchable PyTorch, ONNX GPU FP16, and TensorRT FP16 backends with an interactive tuning UI.

Interface

Interactive control panel for backend selection, open-vocabulary text prompts, confidence thresholding, and tripwire coordinate configuration.

Gradio Interactive Interface for Open-Vocabulary Video Analytics System

Results

Qualitative comparisons across input video and text-prompted segmentation, tracking, and line-crossing counts.

01 • Input Video original
02 • Text Prompt "car, van, truck"
03 • Text Prompt "heavy truck"
04 • Text Prompt "red car"

Performance

Inference latency and throughput benchmark for YOLOE-26l across deployment runtimes (NVIDIA T4 GPU).

Inference Engine Precision Avg. Latency (ms) Throughput (FPS) Speedup
PyTorch FP32 29.2 34.3 1.00x (Baseline)
ONNX GPU FP16 23.2 43.2 1.26x
TensorRT Optimal FP16 14.3 69.8 2.04x

Architecture & Tech Stack

End-to-end video inference pipeline and deployment components.

01. INPUT
Video Input & Prompt
Video Frames + Text Query
02. SEGMENTATION
YOLOE-26
(Multi-Backend)
Open-Vocab Seg Masks
03. ANALYTICS
Tracking + Tripwire
Track IDs & Line-Crossing Counts
04. FRONTEND
Interactive Control UI
Annotated Video & Count Stats

Model & Inference Acceleration

YOLOE-26l TensorRT FP16 ONNX GPU FP16 PyTorch