Pass CPU cores to speed up inference
Hi, I am looking can we speedup inference process by doing multi thread/ multi process on cpu withoptimumonnx runtime
2022-06-14T07:32:02Z
Hello@Talha,By default optimum with onnxruntime will use all available cores. If you want to adjust this, e.g. usemultiprocessingand optimum in parallel you can adjust the used THREADS/CPU cores withonnxruntime.SessionOptionsby configuringinter_op_num_threadsandintra_op_num_threads