trtutils.jetson package

Module contents

A submodule implementing additional tools for Jetson devices.

Classes

JetsonBenchmarkResult

The results of benchmarking a TRTEngine on a Jetson device.

Functions

benchmark_engine()

A mirror of trtutils.benchmark_engine, but also measures energy usage.

benchmark_engines()

A mirror of trtutils.benchmark_engines, but also measures energy usage.

class trtutils.jetson.JetsonBenchmarkResult(latency: 'Metric', power_draw: 'Metric', energy: 'Metric')[source]

Bases: object

latency: Metric
power_draw: Metric
energy: Metric
trtutils.jetson.benchmark_engine(engine: TRTEngine | Path | str, iterations: int = 1000, warmup_iterations: int = 50, tegra_interval: int = 5, dla_core: int | None = None, *, warmup: bool | None = None, verbose: bool | None = None) → JetsonBenchmarkResult[source]

Benchmark a TensorRT engine on a Jetson device.

Parameters:

  • engine (TRTEngine | Path | str) – The engine to benchmark. Either a TRTEngine object or path to the engine file. If a path is given, then a TRTEngine will be created automatically.

  • iterations (int, optional) – The number of iterations to run the benchmark for, by default 1000.

  • warmup_iterations (int, optional) – The number of warmup iterations to run before the benchmark, by default 50.

  • tegra_interval (int, optional) – The number of milliseconds between each tegrastats sampling. The smaller the number, the more samples per second are generated. By default 5 milliseconds between samples.

  • dla_core (int, optional) – The DLA core to assign DLA layers of the engine to. Default is None. If None, any DLA layers will be assigned to DLA core 0.

  • warmup (bool, optional) – Whether to do warmup iterations, by default None If None, warmup will be set to True.

  • verbose (bool, optional) – Whether ot not to output additional information to stdout. Default None/False.

Returns:

A dataclass containing the results of the benchmark.

Return type:

BenchmarkResult

trtutils.jetson.benchmark_engines(engines: Sequence[TRTEngine | Path | str | tuple[TRTEngine | Path | str, int]], iterations: int = 1000, warmup_iterations: int = 50, tegra_interval: int = 5, *, warmup: bool | None = None, parallel: bool | None = None, verbose: bool | None = None) → list[JetsonBenchmarkResult][source]

Benchmark a TensorRT engine.

Parameters:

  • engines (Sequence[TRTEngine | Path | str | tuple[TRTEngine | Path | str, int]]) – The engines to benchmark as paths to the engine files.

  • iterations (int, optional) – The number of iterations to run the benchmark for, by default 1000.

  • warmup_iterations (int, optional) – The number of warmup iterations to run before the benchmark, by default 50.

  • tegra_interval (int, optional) – The number of milliseconds between each tegrastats sampling. The smaller the number, the more samples per second are generated. By default 5 milliseconds between samples.

  • warmup (bool, optional) – Whether to do warmup iterations, by default None If None, warmup will be set to True.

  • parallel (bool, optional) – Whether or not to process the engines in parallel. Useful for assessing concurrent execution performance. Will execute the engines in lockstep. If None, will benchmark each engine individually.

  • verbose (bool, optional) – Whether ot not to output additional information to stdout. Default None/False.

Returns:

A list of dataclasses containing the results of the benchmark. If parallel was True, will only contain one item.

Return type:

list[JetsonBenchmarkResult]