gpu-unlock

What is it?

GPU Unlock is an open-source tool released by GitHub developer Zagitta, targeting the RTX 30 series based on the Ampere architecture. It raises the throughput of certain operations that are set low only on consumer products, on the Tensor Cores (compute units dedicated to matrix multiplication) that handle AI calculations. The targets are operations that compute in FP16 or BF16 format and add up (accumulate) results in FP32 format. The developer said it doubles the instruction throughput of these operations. However, this means one specific operation gets faster, not that an entire AI program runs twice as fast.

How does it work?

The tool runs as a Linux kernel module (a small program inserted into the core of the operating system). Early in boot, it runs before the NVIDIA graphics driver initializes the GPU and changes hardware settings, after which the original, unmodified driver takes over the card as usual. It was described as not requiring direct changes to the driver or permanent changes to the graphics card firmware, and as not needing to be re-patched every time the driver is updated. It works only when activated by specifying the target card in the kernel boot options; if no card is specified, it does nothing.

Why is it getting attention?

In a matrix multiplication test the developer published (450W power limit), the developer reported that the RTX 3090 Ti's BF16 performance rose from 87.50 TFLOPS to 157.41 TFLOPS, about 80% higher. In the same test, FP16 rose from 87.46 to 118.39 TFLOPS and TF32 from 42.58 to 82.61 TFLOPS. User reports showed image and video generation throughput increasing by up to about 60%, and the prefill phase, in which an LLM reads the input prompt all at once, getting 14–36% faster. For people running local LLMs or image generation models at home, the appeal is being able to speed things up on the card they already have.

Things to know (limitations)

The gains are concentrated in workloads heavy on matrix operations. The decoding phase, in which an LLM produces its answer one token at a time, depends more on how fast data is read from memory than on compute, so it barely changed, and image generation run in INT8 format saw only a 0.3% throughput increase. It is also not a tool that increases graphics memory capacity or bandwidth. The size of the improvement varies by card type and power settings, and on some cards (e.g., a certain OEM RTX 3080 Ti) performance reportedly did not improve even after completing the unlock procedure. It is a community tool, not an official manufacturer feature; it currently targets Linux environments, and per-card validation is still ongoing.

Example

One user with an RTX 3090 Ti reported that in the image generation tool ComfyUI, with the same model, prompt and power limit, the time to generate one image in BF16 dropped from about 72 seconds to 45 seconds. In contrast, in a separate test running an LLM with the inference engine vLLM on an RTX 3090, only prompt processing speed rose by about 33%, while answer generation speed stayed nearly the same. In other words, the effect you notice depends heavily on whether your workload is 'compute-bound' or 'memory-bound.'

Related terms

GPU Precision (FP32, BF16, FP8) FLOPS VRAM Local LLM Inference engine

References

• RTX 30 graphics card 'unlock' tool appears… AI image and video throughput up to 60% higher (AI Times)
• Nvidia RTX 3090 performance nearly doubles with free GPU unlock, but there's a catch (Notebookcheck)
• NVIDIA GeForce RTX 30 GPUs Get Up To 2X Faster AI Compute Performance With New GPU Unlock Mod, Real-World Gains Of Up To 60% (Wccftech)
• Zagitta’s Nvidia Rtx 30 Gpu Unlock Mod Hits 129.6 TFLOPS (شبكة تواصل الإخبارية)
• Open-Source Project GPU Unlock Boosts RTX 30 Series AI Performance by Up to 60% (KuCoin)
• GPU-unlock - A Linux module that lifts the AI compute performance limit on the RTX 30 series (GeekNews)