Welcome to trtutils

trtutils is a high-level Python interface for TensorRT inference, providing a simple and unified way to run arbitrary TensorRT engines. This library abstracts away the complexity of CUDA memory management, binding management, and engine execution.

Features

  • Simple, high-level interface for TensorRT inference

  • Automatic CUDA memory management

  • Support for arbitrary TensorRT engines

  • Built-in preprocessing and postprocessing capabilities

  • Comprehensive type hints and documentation

  • Support for both basic engine execution and end-to-end model inference

  • Specialized support for YOLO models

  • Performance benchmarking and monitoring

Quick Start

from trtutils import TRTEngine

# Load your TensorRT engine
engine = TRTEngine("path_to_engine")

# Get input specifications
print(engine.input_shapes)  # Expected input shapes
print(engine.input_dtypes)  # Expected input data types

# Run inference
inputs = read_your_data()
outputs = engine.execute(inputs)

Documentation

Contents:

Indices and Tables

Getting Help