theartificialintelligenc
Region: US
Saturday 04 April 2026 13:08:05 GMT
149206
5118
123
694
Music
Download
Comments
tomplee✔️ :
speed about 1 token per second
2026-04-04 17:45:15
235
Dima Jr. Worcestershire :
tried. slow. deleted.
2026-04-05 16:04:50
109
sugahustler :
**Short answer:** Yes—while AirLLM pioneered layer-wise inference for extreme memory constraints (70B models on 4GB GPUs), several alternatives now of
2026-04-05 12:50:15
0
dkw999_ :
Just use chatgpt. seriously. u cant never beat their algorithm with a homemade solution
2026-04-28 12:25:46
1
EL_ PEPE :
Trying to cook a huge meal in a tiny kitchen by bringing ingredients in one at a time. It’s going to be slow like crazy…
2026-04-05 16:30:38
17
White Raven :
I actually built a better model 🤷🏼♂️I can fit 64 gb into 4 gb of physical ram
2026-04-05 12:32:40
2
Rostás Lukász Armándó :
Pretty much abandoned
2026-04-04 15:31:46
13
misticlafrite :
my GTX1650 ain't doing that
2026-04-05 10:54:10
5
🜲 마우리 🜐 :
It is for special workflows to compensate tasks but not for general chat or vibe coding. Example it helped me before to analyze/search blocks of code with Mistral on my behalf to then create a prompt to be sent to Mistral without this engine. It was just a helper but now you have better options like Gemma e2b or e4b
2026-04-28 23:54:57
1
🍐Pääruna🥔 :
or you can just use the OS built in swap file feature. this likely won't deliver much any better performance because the token must traverse through all the layers anyway before the next comes in. in the besr case you could have them pipelined in such a train where a few of the consequent layers and tokens are processed in parallel in the same memory window. This is just speculative tho I didn't read through the actual project obviously
2026-04-05 14:47:08
1
simpleuser :
1 token per second + no accuracy
2026-04-28 18:51:18
5
John Doe and 753 others :
I have a 4gb gpu but I only do gaming, what could I do with this?
2026-05-25 22:32:37
0
sleepless :
any model running on my spare 1080ti?
2026-04-05 17:47:58
0
t90955 :
1TPS on 4gb GPU for 70B model isn't bad at all
2026-04-07 06:26:55
3
AISweeties :
"Run" more like crawl 😅
2026-04-11 19:11:31
1
Kshitij :
tiktok, listen, i want valorant clips, not this bullshit
2026-04-29 04:25:03
0
M :
49 layers of hell.
2026-04-05 23:44:06
2
localhost:3000 :
you didn't tell the truth that how speed is it?
2026-04-29 05:53:49
1
javi cc :
habría que ver la velocidad y más si está almacenado en un hdd como es mi caso
2026-04-05 13:07:06
0
SJ :
There is no update for 2 years
2026-04-28 04:53:28
2
Nathanael Lie :
I think the term "walk" is more suitable than "run" here 😅
2026-04-29 13:54:34
2
Mary :
how about an 8gb gpu? double?
2026-04-28 14:16:13
1
deafmogor :
Is it actually work?
2026-04-05 11:17:24
0
ju4n_r94 :
Slow asf, but if u need to resume 50 docs in a row, u can do it
2026-04-05 15:07:30
2
natriumchl :
1 token per sometimes😭
2026-04-30 14:41:12
0
To see more videos from user @theartificialintelligenc, please go to the Tikwm
homepage.
