Failed to create CUDAExecutionProvider
Hello Everyone,I have been trying to Optimize the model (Text Classification, BERT) for GPU usingORTOptimizerbut first, I got this warning message:[W:onnxruntime:Default, onnxruntime_pybind_state.cc:578  CreateExecutionProviderInstance] Failed to create CUDAExecutionProvider. Please reference https://onnxruntime.ai/docs/reference/execution-providers/CUDA-ExecutionProvider.html#requirements to ensure all dependencies are met. 2022-11-22 13:46:00.800872459 [W:onnxruntime:, inference_session.cc:1458 Initialize] Serializing optimized model with Graph Optimization level greater than ORT_ENABLE_EXTENDED and the NchwcTransformer enabled.  The generated model may contain hardware-specific optimizations, and should only be used in the same environment the model was optimized in.Then I faced an error:assert "CUDAExecutionProvider" in session.get_providers()  # Make sure there is GPU     107 assert os.path.exists(optimized_model_path) and os.path.isfile(optimized_model_path)     108 logger.debug("Save optimized model by onnxruntime to {}".format(optimized_model_path))Environment check:get_available_providers()output:  [‘TensorrtExecutionProvider’, ‘CUDAExecutionProvider’, ‘CPUExecutionProvider’]
2022-11-23T08:37:49Z
Hello@serdarcaglar, ONNX Runtime + GPU can be tricky. Althoughget_available_providers()shows CUDAExecutionProvider available, ONNX Runtime can fail to find CUDA dependencies when initializing the model.I would recommend you to refer toAccelerated inference on NVIDIA GPUs, especially the section “Checking the installation is successful”, to see if your install is good. I would recommend to follow Nvidia documentation to install CUDA and cuDNN, it’s well written.
2022-11-28T14:34:32Z
Thanks. Problem was solved.
2022-12-01T14:58:49Z
Better error message will be present in the next release, to check the CUDA is well installed:Clearer error messages when initilizing the requested ONNX Runtime execution provider fails by fxmarty · Pull Request #514 · huggingface/optimum · GitHub
2023-01-31T05:34:17Z
stackoverflow.comWhy does onnxruntime fail to create CUDAExecutionProvider in Linux(Ubuntu 20)?python, onnxruntimeanswered byOguz Hanogluon12:02PM - 28 Jan 23 UTCsolved my problem.