Foreman Technology
Region: GB
Sunday 03 May 2026 14:08:13 GMT
24325
478
72
16
Music
Download
Comments
Vonzloprcek :
you know p6000s are cheap now? pair of these will run most local models super fast
2026-05-03 20:39:56
2
Faris :
Listen, your hate for Macs aside, how good would the M2 MacBook air with 24gb ram be for software engineering?
2026-05-03 20:06:18
0
rmarslwtbnp :
You suggest 64GB minimum, yet then go on to suggest two A770s which would be 32GB? Even if, good luck finding a motherboard supporting four slots, and finding four A770s in good condition, they're not retail anymore, at least not in NL. On top of that, there's also an energy bill to pay. Running 4x A770s is what, 1000W? GGUFs also run fine on Macs using llama.cpp, so what are you on about? Lastly, macs aren't good for all use cases, sure, but honestly, I've been seriously considering getting a Mac Studio with 128GB unified for my business as my self hosted headless coding LLM server. I wouldn't be doing anything else with it other than running coding models. I would set it up with WoL through my always on N100 server which sips power at 7W, and only when I need it, spin up the Mac Studio which at load is about 150W. With the VAT deduction, it's about 2800 euro which is not bad
2026-05-05 01:23:59
13
Sarah :
yeah fr you want to run a model you need a gpu and mac is slow asf, two, rtx 5060 ti 16gbs and you can run the best model currently on a Q6/Q8 which is stable, download Qwen 3.6 27B or GLM 4.7 personally I can say thats the best budget set up for me atleast.
2026-05-09 15:44:31
1
tabloxalt :
that's you're just factually wrong there I could run a 3B model on my phone which has eight gig of RAM so you could probably run at least 16b if not a bit more on that 24 gig
2026-05-03 15:23:27
3
Clockwork Void :
The problem is not RAM but VRAM though. On Win/tux pc, yes, you need to have RAM+VRAM total to fit the model, but the issue is, whatever does not fit on your GPU VRAM must be constantly swapped. Tried running some 7b param models via OLLAMA on 32GB RAM and 3070 mobile GPU with 8GB VRAM. The model was crowling rather than running, while GPU utilization was just between 25% and 35% while CPU was constantly above 70% as it was constantly swapping the pages of the model between RAM and VRAM…
2026-05-03 23:44:56
6
James Tubbritt :
Do you think a DGX Spark is a good investment ?
2026-05-03 19:40:10
2
Victor :
I tried like pretty much all models that you can reasonable run under 40gb on huffing face for months. The only thing that is remotely close to useful is Gemma 13b, got it working with codex cli/app or qwen cli. I ran the full model but it's decent until 4bit Quant. I can send all parameters if anyone is interested
2026-05-07 01:24:07
1
jw39036 :
So wrong on so many levels. You can also do smart things with kv offloading. Etc etc.
2026-05-04 06:32:16
1
alcany83 :
BS. Read up on MOE models and finetuning. 24gb is great
2026-05-04 21:42:35
1
Golden Freddy :
I do also have a gaming pc with a Ryzen 5600, Radeon 6750xt oc edition with 12GB vram, 32GB ddr4. Honestly the Mac was for Xcode but if you think the code could be generated from the pc and I just build it on the Mac then that would be ok
2026-05-03 21:05:25
0
Oblation :
what could you run on a 24 core threadripper with 256gb of ram and 48gb RTX 5000 GPU?
2026-05-06 11:10:21
0
Travis :
Wait how are you not posting anymore
2026-05-05 23:29:28
0
Vallidium :
It’s the Codex part of the question I’m interested in. I want to run a codex/claude code pipeline locally. I’ve got plenty of VRAM but it’s the interface I’m interested in. Can you point it towards a folder, give it a .codex folder for your skills and an AGENTS.md file to ready from then start generating code into an existing project, giving it full context of the whole project at once when asking it to generate new features.
2026-05-04 05:13:02
0
Wolf :
Best bet rn is qwen3.6 q6 and adjust it's max context window and kv cache quants to q5 or something to make it fit...
2026-05-03 19:15:21
1
jOhn 🇬🇧🏴 :
Have an Asus Duo 32gb/2tb/IntelUltra9.
2026-05-03 14:19:22
0
FPVDad :
Glad I’ve got 128gb!
2026-07-05 11:23:24
0
johnwatson710 :
But also graphics card?
2026-05-17 18:35:13
0
James :
idk what the obsession is with local models, for basic things like embedding and semantic search, maybe you're okay with 12gb vram + 32gb ram. I have 12gb vram and 64gb ram, and can't get any reliable output from a coding model. for a slight increase in quality, it'll be 1/10th tok/s as you'll need to load more onto system ram. it's genuinely so much cheaper to use LLM gateways on the cloud that are encrypted
2026-05-11 13:55:35
0
trex :
What about 190? What is the top model in that range?
2026-05-05 19:10:46
0
failsafecrew :
Latest Qwen is quite good. I use it on my 3090. it's puts ut quite good code.
2026-05-05 18:59:03
0
Djul :
what's a good model at 32 gb ram just to act as a general llm?
2026-05-05 18:49:17
0
TheCCSI.org :
Well, start with the coding variant of a model. The get the biggest you can run and see the throughput. Acceptable? Give it small tasks and test. Unacceptable speed? Try smaller. Bs count, but so do the domains of those Bs.
2026-05-03 17:09:50
0
To see more videos from user @foremantech, please go to the Tikwm
homepage.
