trtutils.core.cache module¶

Tools for managing the trtutils TensorRT engine cache.

Useful for quickly recalling pre-compiled TRT engines, without having to implement your own caching mechanisms. Used in the TRTPreprocessor to manage compiled engines for different image sizes.

Functions¶

get_cache_dir(): Gets the cache directory inside of the trtutils install.
clear(): Clears the cache directory.
query(): Queries the cache to see if an engine with that name already exists.
store(): Stores a compiled TensorRT engine in the cache.
remove(): Removes an engine file from the cache.
query_file(): Queries the cache for a file with a specific extension.
store_file(): Stores a file in the cache with a specific name.
remove_file(): Removes a file from the cache.
query_timing_cache(): Queries the cache for the global timing cache.
store_timing_cache(): Stores the global timing cache in the cache directory.
save_timing_cache_to_global(): Saves a TensorRT timing cache object directly to the global timing cache.

trtutils.core.cache.get_cache_dir() → Path[source]¶

Get the location of the trtutils engine cache directory.

Returns:: The trtutils engine cache directory Path
Return type:: Path

trtutils.core.cache.clear(*, no_warn: bool | None = None) → None[source]¶

Use to clear the cache folder for the trtutils engines.

Parameters:: no_warn (bool, optional) – Whether or not to issue a warning that the cache directory is being cleared.

trtutils.core.cache.query_file(filename: str, extension: str = 'engine') → tuple[bool, Path][source]¶

Check if a file with the given name and extension is present in the cache.

Parameters:

filename (str) – The filename to check for. Can be with or without extension. If extension is provided in filename, it will be used.
extension (str, optional) – The file extension to use (without the dot). By default, “engine”.

Returns:

Whether or not the file exists and its Path (whether or not it exists)

Return type:

tuple[bool, Path]

trtutils.core.cache.query(filename: str) → tuple[bool, Path][source]¶

Check if the engine filename is present in the cache.

Parameters:: filename (str) – The filename to check for without a suffix.
Returns:: Whether or not the file exists and its Path (whether or not it exists)
Return type:: tuple[bool, Path]

trtutils.core.cache.store_file(filepath: Path, cache_filename: str | None = None, *, overwrite: bool = False, delete_source: bool = False) → Path[source]¶

Store a file in the trtutils cache.

Parameters:

filepath (Path) – The path to the file to store in the cache.
cache_filename (str, optional) – The name to use in the cache. If None, uses the original filename. By default, None.
overwrite (bool, optional) – Whether or not to overwrite an existing file with the same name. By default False, will keep the older version.
delete_source (bool, optional) – Whether or not to delete the source file after storing. By default, False.

Returns:

The new path of the file in the cache.

Return type:

Path

trtutils.core.cache.store(filepath: Path, *, overwrite: bool = False, delete_source: bool = False) → Path[source]¶

Store an engine file in the trtutils engine cache.

Parameters:

filepath (Path) – The path to the engine file to store in the cache.
overwrite (bool, optional) – Whether or not to overwrite an existing file with the same name. By default False, will keep the older version.
delete_source (bool, optional) – Whether or not to delete the source file after storing. By default, False.

Returns:

The new path of the file in the cache.

Return type:

Path

trtutils.core.cache.remove_file(filename: str, extension: str = 'engine') → None[source]¶

Remove a file from the cache.

Parameters:

filename (str) – The filename to remove from the cache. Can be with or without extension. If extension is provided in filename, it will be used.
extension (str, optional) – The file extension to use (without the dot). By default, “engine”.

Raises:

FileNotFoundError – If the file does not exist in the cache.

trtutils.core.cache.remove(filename: str) → None[source]¶

Remove an engine file from the cache.

Parameters:: filename (str) – The filename to remove from the cache.

trtutils.core.cache.query_timing_cache() → tuple[bool, Path][source]¶

Query the cache for the global timing cache.

Returns:: Whether or not the global timing cache exists and its Path.
Return type:: tuple[bool, Path]

trtutils.core.cache.store_timing_cache(filepath: Path, *, overwrite: bool = False, delete_source: bool = False) → Path[source]¶

Store the global timing cache in the cache directory.

Parameters:

filepath (Path) – The path to the timing cache file to store.
overwrite (bool, optional) – Whether or not to overwrite an existing global timing cache. By default False, will keep the older version.
delete_source (bool, optional) – Whether or not to delete the source file after storing. By default, False.

Returns:

The path of the global timing cache in the cache directory.

Return type:

Path

trtutils.core.cache.save_timing_cache_to_global(timing_cache_obj: _TimingCache, *, overwrite: bool = True) → Path[source]¶

Save a TensorRT timing cache object to the global timing cache.

Parameters:

timing_cache_obj – The TensorRT timing cache object (from config.get_timing_cache()).
overwrite (bool, optional) – Whether or not to overwrite an existing global timing cache. By default True.

Returns:

The path of the global timing cache in the cache directory.

Return type:

Path