The release of the Python framework Savant 0.2.7 has been published, simplifying the use of NVIDIA DeepStream for machine learning tasks. The framework handles all the complex work with GStreamer or FFmpeg, allowing users to focus on building optimized output pipelines using a declarative syntax (YAML) and Python functions. Savant enables the creation of pipelines that work equally well on accelerators in data centers (NVIDIA Turing, Ampere, Hopper) as well as on edge devices (NVIDIA Jetson NX, AGX Xavier, Orin NX, AGX Orin, New Nano). With Savant, you can easily process multiple video streams simultaneously and quickly create production-ready video analytics pipelines using NVIDIA TensorRT. The project code is released under the Apache 2.0 license.
Savant 0.2.7 is the latest release with functional changes in the 0.2.X branch. Future releases in the 0.2.X branch will include only bug fixes. Development of new features will occur in the 0.3.X branch, which is based on DeepStream 6.4. This branch will not support the Jetson Xavier family of devices, as NVIDIA does not support them in DS 6.4.
Key innovations:
- New use cases:
- Example of working with the RT-DETR transformer-based detection model;
- CUDA post-processing with CuPy for YOLOV8-Seg;
- Example of integrating PyTorch CUDA into the Savant pipeline;
- Demonstration of working with oriented objects.
- New features:
- Integration with Prometheus. The pipeline can export performance metrics to Prometheus and Grafana for monitoring and tracking performance. Developers can declare custom metrics that are exported along with system metrics.
- Buffer adapter — implements a persistent transactional buffer on disk for data moving between adapters and modules. It can be used to develop high-load pipelines that unpredictably consume resources and handle traffic spikes. The adapter exports its data about items and sizes to Prometheus.
- Model compilation mode. Modules can now compile their models into TensorRT without running the pipeline.
- Shutdown event handler in PyFunc. This new API allows for proper handling of pipeline shutdown operations, releasing resources and notifying external systems of the shutdown.
- Frame filtering on input and output. By default, the pipeline accepts all frames containing video data. With input and output filtering, developers can filter data to exclude it from processing.
- Post-processing model on GPU. With the new feature, developers can access model output tensors directly from GPU memory without loading them into CPU memory and process them with CuPy, TorchVision, or OpenCV CUDA.
- GPU memory representation functions. In this release, we provided functions to convert memory buffers between OpenCV GpuMat, PyTorch GPU tensors, and CuPy tensors.
- API access to pipeline queue usage statistics. Savant allows adding queues between PyFunc to implement parallel processing and buffering of processing. The added API provides developers access to the queues deployed in the pipeline and allows querying their usage.
The next release (0.3.7) is planned to transition to DeepStream 6.4 without any functionality expansion. The idea is to get a release that is fully compatible with 0.2.7, but based on DeepStream 6.4 and improved technology, while maintaining API compatibility.
Source: opennet.ru
