GitHub radar
Run DeepSeek's giant model on a single 12GB gaming GPU
A model this size usually runs on a server with a large number of GPUs, but this compressed version runs on one desktop with a single gaming GPU with at least 12GB of memory.
This is a compressed version of a huge DeepSeek language model, squeezed down so it fits on one gaming GPU with 12GB of memory, even though the original is far larger. The rest of the model's weight stays on disk.
When it helps
Useful if you have a desktop with a 12GB+ GPU and about 250GB of free disk space and want to try a powerful model locally. Skip it if you only have a laptop, a Mac, or not enough disk space.
Pros
- Fits on one gaming GPU instead of a server full of GPUs
- The lightest version barely loses quality on the authors' own tests
- Runs fully locally on your own machine, no internet needed
Cons
- Needs a special runtime — the usual llama.cpp doesn't support this model yet
- Requires about 250GB of free disk space and a 12GB+ GPU
- A laptop or Mac won't work — you need a desktop with a gaming GPU
How to set it up — step by step
- 1Check you have a desktop with a 12GB+ GPU and about 250GB of free disk space
- 2Open your AI agent (Claude Code, Codex) in your project folder
- 3Send the agent the text from the block below
- 4Wait while the agent downloads the IQ2_XXS build and installs the required runtime tool
- 5Confirm the agent shows the model actually answering a test prompt
Text for your agent
Copy this and send it to your agent — Claude Code, Codex, any of them:
Download and run the compressed DeepSeek-V4.1-Flash-1017 version locally for me from here: https://huggingface.co/ISTA-DASLab/DeepSeek-V4.1-Flash-1017-GSQ-RCO-GGUF — take the IQ2_XXS build, install the runtime tool named in the model card (regular llama.cpp does not support this model yet), and show me it responding.
You need a separate desktop with a gaming graphics card of 12 GB or more and about 250 GB of free disk space for the weights. A regular laptop or Mac will not run it. The usual Ollama cannot run this model yet, a different tool named in the model card is needed.
Open on Hugging Face▌ More finds