trtutils.core.cache module

Tools for managing the trtutils TensorRT engine cache.

Useful for quickly recalling pre-compiled TRT engines, without having to implement your own caching mechanisms. Used in the TRTPreprocessor to manage compiled engines for different image sizes.

Functions

get_cache_dir()

Gets the cache directory inside of the trtutils install.

clear_cache()

Clears the cache directory.

query_cache()

Queries the cache to see if an engine with that name already exists.

store_in_cache()

Stores a compiled TensorRT engine in the cache.

trtutils.core.cache.get_cache_dir() → Path

Get the location of the trtutils engine cache directory.

Returns:

The trtutils engine cache directory Path

Return type:

Path

trtutils.core.cache.clear_cache(*, no_warn: bool | None = None) → None

Use to clear the cache folder for the trtutils engines.

Parameters:

no_warn (bool, optional) – Whether or not to issue a warning that the cache directory is being cleared.

trtutils.core.cache.query_cache(filename: str) → tuple[bool, Path]

Check if the engine filename is present in the cache.

Parameters:

filename (str) – The filename to check for without a suffix.

Returns:

Whether or not the file exists and its Path (whether or not it exists)

Return type:

tuple[bool, Path]

trtutils.core.cache.store_in_cache(filepath: Path, *, overwrite: bool = False, clear_old: bool = False) → Path

Store an engine file in the trtutils engine cache.

Parameters:

• filepath (Path) – The path to the engine file to store in the cache.

• overwrite (bool, optional) – Whether or not to overwrite an existing file with the same name. By default False, will keep the older version.

• clear_old (bool, optional) – Whether or not to automatically clear the original file. By default, False, will not remove original (old) file.

Returns:

The new path of the file in the cache.

Return type:

Path