V.I.P 4.26 4.27 5.0 NNEngine - Neural Network Engine v1.5

Prerequisite to use with CUDA and TensorRT​

To use with CUDA and TensorRT, you need to install the following versions of CUDA, cuDNN, and TensorRT.



Windows:

The versions of cuDNN and TensorRT are different for RTX30** series and others. We only tested GTX1080Ti, RTX2070, RTX3060Ti and RTX3070. Others are not tested.



Versions for other than RTX30** series (RTX20**, GTX10**)

• CUDA: 11.0.3
• cuDNN: 8.0.2 (July 24th, 2020), for CUDA 11.0
• TensorRT: 7.1.3.4 for CUDA 11.0

Versions for RTX30** series

• CUDA: 11.0.3
• cuDNN: 8.0.5 (November 9th, 2020), for CUDA 11.0
• TensorRT: 7.2.2.3 for CUDA 11.0

Ubuntu:

• CUDA: 11.4.2 for Linux x86_64 Ubuntu 18.04
• cuDNN: 8.2.4 (September 2nd, 2021), for CUDA 11.4, Linux x86_64
• TensorRT: 8.2.3.0 (8.2 GA Update 2) for Linux x86_64, CUDA 11.0-11.5

To use with TensorRT, it is recommended to add the following environment variables to cache TensorRT Engine:

• "ORT_TENSORRT_ENGINE_CACHE_ENABLE" and set its value to "1".
• "ORT_TENSORRT_CACHE_PATH" and set its value to any path where you want to save the cache, for example "C:\temp".

Technical Details​

Features:

• Loads ONNX models at runtime. Automatically optimizes the model when loaded.
• Runs ONNX models.
• (On Windows) Supports hardware acceleration by DirectML on DirectX 12 capable GPUs.
• (On Windows and Ubuntu) Supports hardware acceleration by CUDA and TensorRT on supported NVIDIA GPUs.
• (On Windows) Gets a list of GPU names available on the system.
• Processes (resize, crop, rotate) UTexture and converts it to int8 array.

Code Modules:

• OnnxRuntime (Runtime)
• TextureProcessing (Runtime)
• CustomizedOpenCV (Runtime)
• DirectXUtility (Runtime)

Number of Blueprints: 3
Number of C++ Classes: 7+