@artyomita: Your GPU has 8 GB, the model is 18. Quantization is how you run it anyway: fewer bits per number, about 3x smaller, almost as smart. Still too big? Offload to RAM, or even stream experts from your SSD, like antirez does. Each step down is slower, so quantize first 🙂 #quantization #localai #llm #llamacpp #vram

adriandj3

Open In TikTok:

Region: IT

Monday 05 October 2026 22:03:18 GMT