How to increase GPU usage using transformer's pipeline?

Beginners
1

This is my first post and I know it is probably simple, but how do I increase my GPU utilization? Below is my memory and utilization for each GPU. The utilization ranges from this to ~40% on average.

image
image302×171 9.13 KB

I am using transformers.pipeline to make my calls with device_map=“auto” to spread the model out over the GPUs as it’s too big to fit on a single GPU (Llama 3.3 70B). I am not sure if I’m just bottlenecked by storage, but I’m very new and almost certain there’s improvements to be made. I’m just doing text-generation and am using the StopStringCriteria. Any tips would be much appreciated.

1 Like
4

Experiencing low GPU utilization can hinder your system’s performance, especially during tasks like gaming or deep learning. To address this, consider the following strategies:

  1. Update Graphics Drivers: Ensure your GPU drivers are current, as outdated drivers can limit performance.
  2. Optimize In-Game Settings: Adjust game settings to better utilize the GPU. For instance, increasing the resolution or disabling VSync can enhance GPU usage
  3. Check for CPU Bottlenecks: A weak CPU can restrict GPU performance. Monitor CPU usage to ensure it isn’t maxed out, which can impede GPU utilization.
  4. Enable High-Performance Mode: Set your system’s power plan to ‘High Performance’ to allow the GPU to operate at full capacity.
  5. Monitor System Temperatures: Overheating can cause the GPU to throttle. Keep an eye on temperatures to prevent thermal throttling.
  6. Ensure Adequate Power Supply: Verify that your PSU delivers sufficient power to the GPU, as inadequate power can limit performance.
  7. Update or Reinstall the Game: Corrupted game files can lead to low GPU usage. Reinstalling the game might resolve such issues.
  8. Disable Background Processes: Close unnecessary applications running in the background that might consume CPU resources, affecting GPU performance.
  9. Check for Malware: Malware can consume system resources. Run a full system scan to ensure your system is clean.
  10. Consider Hardware Upgrades: If persistent low GPU usage continues, it might be time to upgrade components like the CPU or RAM to eliminate bottlenecks.

Implementing these steps can help enhance GPU utilization, leading to improved system performance.
The original answer is here.
[/quote]

1 Like