You can now run DeepSeek R1-v2 on your local device!

Hello folks! Yesterday, DeepSeek did a huge update to their R1 model, bringing its performance on par with OpenAI’s o3, o4-mini-high and Google’s Gemini 2.5 Pro. They called the model ‘DeepSeek-R1-0528’ (which was when the model finished training) aka R1 version 2.

Back in January, you could actually run the full 720GB sized R1 (non-distilled) model with just an RTX 4090 (24GB VRAM) and now we’re doing the same for this even better model and better tech.

Note: if you do not have a GPU, no worries, DeepSeek also released a smaller distilled version of R1-0528 by fine-tuning Qwen3-8B. The small 8B model performs on par with Qwen3-235B so you can try running it instead That model just needs 20GB RAM to run effectively. You can get 8 tokens/s on 48GB RAM (no GPU) with the Qwen3-8B R1 distilled model.

At Unsloth, we studied R1-0528’s architecture, then selectively quantized layers (like MOE layers) to 1.58-bit, 2-bit etc. which vastly outperforms basic versions with minimal compute. Our open-source GitHub repo: https://github.com/unslothai/unsloth

1. We shrank R1, the 671B parameter model from 715GB to just 185GB (a 75% size reduction) whilst maintaining as much accuracy as possible.
2. You can use them in your favorite inference engines like llama.cpp.
3. Minimum requirements: Because of offloading, you can run the full 671B model with 20GB of RAM (but it will be very slow) – and 190GB of diskspace (to download the model weights). We would recommend having at least 64GB RAM for the big one!
4. Optimal requirements: sum of your VRAM+RAM= 120GB+ (this will be decent enough)
5. No, you do not need hundreds of RAM+VRAM but if you have it, you can get 140 tokens per second for throughput & 14 tokens/s for single user inference with 1xH100

If you find the large one is too slow on your device, then would recommend you to try the smaller Qwen3-8B one: https://huggingface.co/unsloth/DeepSeek-R1-0528-Qwen3-8B-GGUF

The big R1 GGUFs: https://huggingface.co/unsloth/DeepSeek-R1-0528-GGUF

We also made a complete step-by-step guide to run your own R1 locally: https://docs.unsloth.ai/basics/deepseek-r1-0528

Thanks so much once again for reading! I’ll be replying to every person btw so feel free to ask any questions!