@olleai: Running the 104 GB Qwen 3.8 flash next on a MacBook Air with 24 GB of RAM using slotstream which streams the model from the SSD instead of loading it into memory. #localai #qwen #macbook #opensource

Olle
Olle
Open In TikTok:
Region: SE
Wednesday 02 September 2026 14:56:51 GMT
6721
240
23
38

Music

Download

Comments

streamy4711
Streamy4711 :
Puh, 20 GB swap is a lot
2026-09-03 08:43:05
0
brzhifi
ruslanmulyk :
Llama can do it as well😃
2026-09-03 06:50:13
1
dailria8606
gzgft :
but if you keep most experts on the disc the model is nowhere near as smart as should be ? then what is the point
2026-09-03 07:28:55
0
komy4k
komy4k :
Hur fan är en 100gb model ens i närheten av opus????
2026-09-03 01:29:02
0
balumanol
balumanol :
t/s?
2026-09-03 02:11:06
0
sionlockett
sionlockett :
Is slotstream any faster than using llama.cpp? Doesn't llama natively overflow to disc when there's not enough memory?
2026-09-02 15:04:32
0
mrhistoricsoap
MrHistoricSoap :
currently running local qwen3.8-27b how would this compare for resource usage
2026-09-03 07:24:39
0
dustinb525
dustinb525 :
12 hours later
2026-09-03 03:18:05
0
peanguinmc
Peanguin :
does it work with smaller models tho?
2026-09-03 01:45:47
0
__30571
__3057 :
eh you need more ram tbh
2026-09-03 01:07:30
0
pirkkapekkapeteli
pirkkapekkapeteli :
How much difference do you see with how fast it is?
2026-09-02 16:15:09
0
To see more videos from user @olleai, please go to the Tikwm homepage.

Other Videos


About