Error loading script: trt.py, ModuleNotFoundError: No module named 'tensorrt_bindings'

[rlenci99]

Hello, i have fresh installed using installation method provided and the extension will install, but the tab for TensorRT will not show.

*** Error loading script: trt.py
Traceback (most recent call last):
File "E:\Stable diffusion\SD\webui\modules\scripts.py", line 382, in load_scripts
script_module = script_loading.load_module(scriptfile.path)
File "E:\Stable diffusion\SD\webui\modules\script_loading.py", line 10, in load_module
module_spec.loader.exec_module(module)
File "", line 883, in exec_module
File "", line 241, in call_with_frames_removed
File "E:\Stable diffusion\SD\webui\extensions\Stable-Diffusion-WebUI-TensorRT\scripts\trt.py", line 10, in
import ui_trt
File "E:\Stable diffusion\SD\webui\extensions\Stable-Diffusion-WebUI-TensorRT\ui_trt.py", line 10, in
from exporter import export_onnx, export_trt
File "E:\Stable diffusion\SD\webui\extensions\Stable-Diffusion-WebUI-TensorRT\exporter.py", line 10, in
from utilities import Engine
File "E:\Stable diffusion\SD\webui\extensions\Stable-Diffusion-WebUI-TensorRT\utilities.py", line 32, in
import tensorrt as trt
File "E:\Stable diffusion\SD\webui\venv\lib\site-packages\tensorrt_init.py", line 18, in
from tensorrt_bindings import *
ModuleNotFoundError: No module named 'tensorrt_bindings'

[racerx2oo3]

What appears to have worked for others.

From your base SD webui folder: (E:\Stable diffusion\SD\webui\ in your case).

• In the extensions folder delete: stable-diffusion-webui-tensorrt folder if it exists
• Delete the venv folder

Open a command prompt and navigate to the base SD webui folder

• Run webui.bat - this should rebuild the virtual environment venv
• When the WebUI appears close it and close the command prompt

Open a command prompt and navigate to the base SD webui folder

• enter: venv\Scripts\activate.bat
• the command line should now have (venv) shown at the beginning.
• enter the following commands:
• • python.exe -m pip install --upgrade pip
• • python -m pip install nvidia-cudnn-cu11==8.9.4.25 --no-cache-dir
• • python -m pip install --pre --extra-index-url https://pypi.nvidia.com/ tensorrt==9.0.1.post11.dev4 --no-cache-dir
• • python -m pip uninstall -y nvidia-cudnn-cu11
• • venv\Scripts\deactivate.bat
• webui.bat
• Install the TensorRT extension using the Install from URL option
• Once installed, go to the Extensions >> Installed tab and Apply and Restart

[rlenci99]

What appears to have worked for others.

From your base SD webui folder: (E:\Stable diffusion\SD\webui\ in your case).

• In the extensions folder delete: stable-diffusion-webui-tensorrt folder if it exists
• Delete the venv folder

Open a command prompt and navigate to the base SD webui folder

• Run webui.bat - this should rebuild the virtual environment venv
• When the WebUI appears close it and close the command prompt

Open a command prompt and navigate to the base SD webui folder

• enter: venv\Scripts\activate.bat
• the command line should now have (venv) shown at the beginning.
• enter the following commands:
• • python.exe -m pip install --upgrade pip
• • python -m pip install nvidia-cudnn-cu11==8.9.4.25 --no-cache-dir
• • python -m pip install --pre --extra-index-url https://pypi.nvidia.com/ tensorrt==9.0.1.post11.dev4 --no-cache-dir
• • python -m pip uninstall -y nvidia-cudnn-cu11
• • venv\Scripts\deactivate.bat
• webui.bat
• Install the TensorRT extension using the Install from URL option
• Once installed, go to the Extensions >> Installed tab and Apply and Restart

Thanks for the detailed steps, much appreciated. It worked!!

[NicoBerlo]

What appears to have worked for others.
(...)
* Once installed, go to the Extensions >> Installed tab and Apply and Restart

Hi !

I've managed to get it to work with v1-5-pruned-emaonly.safetensors [6ce0161689], but when I try with sd_xl_base_1.0.safetensors [31e35c80fc], I get the following error (I created a profile for the sd_xl_base_1.0 tensors which I selected in the UI, and I'm using a shape that I generated in the profile) :
*** Error completing request *** Arguments: ('task(l8v3an5pj39id83)', 'one robot on a bike', '', [], 80, 'Euler a', 1, 1, 7, 768, 1024, False, 0.7, 2, 'Latent', 0, 0, 0, 'Use same checkpoint', 'Use same sampler', '', '', [], <gradio.routes.Request object at 0x000001EBA30067D0>, 0, False, '', 0.8, 194968604, False, -1, 0, 0, 0, False, False, 'positive', 'comma', 0, False, False, '', 1, '', [], 0, '', [], 0, '', [], True, False, False, False, 0, False) {}

` (...)
File "C:\Users\nicolas\ai\sd.webui\webui\extensions-builtin\Lora\networks.py", line 486, in network_MultiheadAttention_forward
return originals.MultiheadAttention_forward(self, *args, **kwargs)

  File "C:\Users\nicolas\ai\sd.webui\webui\venv\lib\site-packages\torch\nn\modules\activation.py", line 1205, in forward
    attn_output, attn_output_weights = F.multi_head_attention_forward(

  File "C:\Users\nicolas\ai\sd.webui\webui\venv\lib\site-packages\torch\nn\functional.py", line 5373, in multi_head_attention_forward
    attn_output = scaled_dot_product_attention(q, k, v, attn_mask, dropout_p, is_causal)

NameError: name 'scaled_dot_product_attention' is not defined`

actually, when I select the sdxl tensor in the UI, I get this error in the logs :

changing setting sd_model_checkpoint to sd_xl_base_1.0.safetensors [31e35c80fc]: NameError
(...)
File "C:\Users\nicolas\ai\sd.webui\webui\venv\lib\site-packages\torch\nn\functional.py", line 5373, in multi_head_attention_forward
    attn_output = scaled_dot_product_attention(q, k, v, attn_mask, dropout_p, is_causal)
NameError: name 'scaled_dot_product_attention' is not defined

and the tensor isn't shown as selected. I have to select it twice for it to stick

I relaunched the webui.bat and now I'm getting this error with sdxl base :
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument mat1 in method wrapper_CUDA_addmm)

[NicoBerlo]

Ok, si I simply switched to the dev branch and it worked right away !

[BBVamp]

What appears to have worked for others.

From your base SD webui folder: (E:\Stable diffusion\SD\webui\ in your case).

• In the extensions folder delete: stable-diffusion-webui-tensorrt folder if it exists
• Delete the venv folder

Open a command prompt and navigate to the base SD webui folder

• Run webui.bat - this should rebuild the virtual environment venv
• When the WebUI appears close it and close the command prompt

Open a command prompt and navigate to the base SD webui folder

• enter: venv\Scripts\activate.bat
• the command line should now have (venv) shown at the beginning.
• enter the following commands:
• • python.exe -m pip install --upgrade pip
• • python -m pip install nvidia-cudnn-cu11==8.9.4.25 --no-cache-dir
• • python -m pip install --pre --extra-index-url https://pypi.nvidia.com/ tensorrt==9.0.1.post11.dev4 --no-cache-dir
• • python -m pip uninstall -y nvidia-cudnn-cu11
• • venv\Scripts\deactivate.bat
• webui.bat
• Install the TensorRT extension using the Install from URL option
• Once installed, go to the Extensions >> Installed tab and Apply and Restart

Worked like a charm.

[hww44]

What appears to have worked for others.
From your base SD webui folder: (E:\Stable diffusion\SD\webui\ in your case).

• In the extensions folder delete: stable-diffusion-webui-tensorrt folder if it exists
• Delete the venv folder

Open a command prompt and navigate to the base SD webui folder

• Run webui.bat - this should rebuild the virtual environment venv
• When the WebUI appears close it and close the command prompt

Open a command prompt and navigate to the base SD webui folder

• enter: venv\Scripts\activate.bat
• the command line should now have (venv) shown at the beginning.
• enter the following commands:
• • python.exe -m pip install --upgrade pip
• • python -m pip install nvidia-cudnn-cu11==8.9.4.25 --no-cache-dir
• • python -m pip install --pre --extra-index-url https://pypi.nvidia.com/ tensorrt==9.0.1.post11.dev4 --no-cache-dir
• • python -m pip uninstall -y nvidia-cudnn-cu11
• • venv\Scripts\deactivate.bat
• webui.bat
• Install the TensorRT extension using the Install from URL option
• Once installed, go to the Extensions >> Installed tab and Apply and Restart

Thanks for the detailed steps, much appreciated. It worked!!

I can't install tensorrt.It give me a error.

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for tensorrt

My pip is the newest.I don't know what's happen.

[Dacrikka]

Hi, my error is "different"

[Dacrikka]

*** Error loading script: trt.py
Traceback (most recent call last):
File "G:\Edo\Programmi_Installati\Stable\stable-diffusion-webui\modules\scripts.py", line 382, in load_scripts
script_module = script_loading.load_module(scriptfile.path)
File "G:\Edo\Programmi_Installati\Stable\stable-diffusion-webui\modules\script_loading.py", line 10, in load_module
module_spec.loader.exec_module(module)
File "", line 883, in exec_module
File "", line 241, in _call_with_frames_removed
File "G:\Edo\Programmi_Installati\Stable\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\scripts\trt.py", line 10, in
import ui_trt
File "G:\Edo\Programmi_Installati\Stable\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\ui_trt.py", line 10, in
from exporter import export_onnx, export_trt
File "G:\Edo\Programmi_Installati\Stable\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\exporter.py", line 10, in
from utilities import Engine
File "G:\Edo\Programmi_Installati\Stable\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\utilities.py", line 22, in
import onnx_graphsurgeon as gs
ModuleNotFoundError: No module named 'onnx_graphsurgeon'

---

What is "onnx_graphsurgeon"? I installe CUDA 11.8... on a 4060 16gb