trtutils.inspect package

Module contents

Submodule containing tools for inspecting TensorRT engines.

Classes

LayerInfo

Detailed information about a single layer in a TensorRT network.

Functions

get_engine_names()

Get the input/output names of a TensorRT engine in order.

get_tensor_size()

Calculate the size of a TensorRT tensor in bytes.

inspect_engine()

Inspect a TensorRT engine.

inspect_onnx_layers()

Inspect the layers of an ONNX model.

class trtutils.inspect.LayerInfo(index: int, name: str, layer_type: str, precision: trt.DataType, input_tensor_size: int, output_tensor_size: int, dla_compatible: bool)

Bases: object

Information about a single layer in a TensorRT network.

index

The layer index in the network.

Type:

int

name

The name of the layer.

Type:

str

layer_type

The type of the layer (e.g., "CONVOLUTION", "POOLING").

Type:

str

precision

The precision of the layer.

Type:

trt.DataType

input_tensor_size

Total size of all input tensors in bytes.

Type:

int

output_tensor_size

Total size of all output tensors in bytes.

Type:

int

dla_compatible

Whether the layer can run on a DLA accelerator.

Type:

bool

index: int

name: str

layer_type: str

precision: trt.DataType

input_tensor_size: int

output_tensor_size: int

dla_compatible: bool

trtutils.inspect.get_engine_names(engine: TRTEngine | Path | str) → tuple[list[str], list[str]]

Get the input/output names of a TensorRT engine in order.

Parameters:

engine (Path | str | trt.ICudaEngine) – Path to the TensorRT engine file or an already loaded engine

Returns:

The input and output tensors in order of enumeration.

Return type:

tuple[list[str], list[str]]

trtutils.inspect.get_tensor_size(tensor: ITensor) → int

Calculate the size of a tensor in bytes.

Computes the total memory footprint by multiplying the number of elements (derived from the tensor shape) by the per-element byte size of the dtype. Dynamic dimensions (-1) are treated as 1.

Parameters:

tensor (trt.ITensor) – The TensorRT tensor.

Returns:

Size in bytes.

Return type:

int

trtutils.inspect.inspect_engine(engine: Path | str | ICudaEngine, *, verbose: bool | None = None) → tuple[int, int, list[tuple[str, tuple[int, ...], DataType, TensorFormat]], list[tuple[str, tuple[int, ...], DataType, TensorFormat]]]

Inspect a TensorRT engine.

Parameters:

• engine (Path | str | trt.ICudaEngine) – Path to the TensorRT engine file or an already loaded engine

• verbose (bool | None, optional) – Whether to print verbose output, by default None

Returns:

The size in bytes of the engine, the max batch size, and two lists of input and output tensors

Return type:

tuple[int, int, list[tuple[str, tuple[int, …], trt.DataType, trt.TensorFormat]], list[tuple[str, tuple[int, …], trt.DataType, trt.TensorFormat]]]

trtutils.inspect.inspect_onnx_layers(onnx: trt.INetworkDefinition | Path | str, config: trt.IBuilderConfig | None = None, *, verbose: bool | None = None) → list[LayerInfo]

Inspect the layers that TensorRT would create from an ONNX model.

Returns detailed per-layer information including tensor sizes, precision, and DLA compatibility. On non-DLA systems, dla_compatible is always False.

Parameters:

• onnx (Path | str | trt.INetworkDefinition) – Path to the ONNX model or an already constructed TensorRT network.

• config (trt.IBuilderConfig | None, optional) – The TensorRT builder config. Required if onnx is a pre-built network and DLA compatibility checking is desired.

• verbose (bool | None, optional) – When True, logs detailed layer information.

Returns:

Per-layer information.

Return type:

list[LayerInfo]

Raises:

ValueError – If onnx is a network and config is not provided.