@momus.ai: RTX 5090 QWEN 3.6 35b 800 tok/second via ninfer. Insanely fast #qwen #localai #aiforbusiness #nvidia #5090

Quandinglus Lemar Tickleton :

unfortunately 3.6 is not that smart. Wish you could get this kind of speed for 3.8 27b

NidofDanes :

I would expect that sort of performance for $5k

thgandalph :

yeah that did not happen 😁

Zabi 🦇 :

I can’t wait for them to drop a local MoE qwen 4. I’m running that same model on an rtx 3060 and 32GB RAM. But hear me out (no it’s not vibecoded, yes I have a repo), I’m running at 20t/s at ONE MILLION context. I’m using freetoken + turboquant + my own research + other mini research paper’s online (expert cut and 2 other which I don’t remember), I also rewrote hoe to model saves kv cache to make it faster and use less computation. Recall is perfect at 1M but when I do optimizations like vectorization, it fails the needle test 😭 been at it for over a day

Damon Danieli :

Nice! That’s nearly 10x what I got for that model on my DGX spark. What are you getting for the 3.8 27B? (I’m running 3.8 flash next on two sparks and loving it — super thorough on xhigh)

Rolf :

trying it out today on my 3090 with Qwen 3.8 27b

MoonUnit :

This is madness to pay this. I'd rather spend the 200 a month to get near unlimited usage and try to make money than spend 10k on a few cards that won't be worth it in 3 years.

Maximus :

tok/sec is cool but I'd be curious to see what kind of value it can generate compared to a frontier model. what it can and can't do

Gudang Baju :

that is so fast

Mike :

its usually quick when you only process 3B weights, the 35B qwen is not a dense model

Weather Tk :

me with my 3060 trying to run qwen 3.8 27b with token per sometimes in q2

itzpezz :

Huh? What? What are we talking about?

ChickenChaser :

What is the stack to achieve this speed and what context are you running it at? I was capping out closer to 200 t/s but im sure my stuff isn’t configured or optimized correctly

paladinpoet :

It doesn't say how not useful the performance is unfortunately [Teary eyed] but sick speed

Daniel :

lvmrevolutiondotcom :

si ok fevi far vedere la generazione della risposta...😂

sharpclawser :

how bruh?

CheeseSteak :

still sucks

Casey ❌ :

😂😂😂