@marcinteodoru: Full setup is in my profile. This is Colibri, and it can run absolutely massive AI models locally by using your SSD, RAM and VRAM together. We’re talking 744 BILLION to 2.8 TRILLION parameters on consumer hardware. It’s still slow right now. But if this gets faster, local AI gets very interesting very quickly. And NVIDIA might want to pay attention. 👀

user2461141185517 :

20 minutes per complex chat request

Ten Ten :

ok but howany tokens per sec?

parse_scratch_0001 :

how about using an NVME? instead of SSD?

Eric the Shellback :

Colibri is great, but for my set up, Im gonna try qwFNfer. Looking to get Qwen3.8-Flash-Next on a Dell 7060 i7-8700, 40gb RAM, A2000 (12gb) GPU, Crucial P310 1tb 2280 nvme ssd. Crossing my fingers for a minimum of 10 tok/s. Im currently using Ornith1.5:9b with 35 tok/s. Big difference between the two...

user9762258332577858437 :

1 token per month

feelurdreams :

so we can now crack any crypto wallet?

culturecrush :

But how fast is it

Nathy Mosquera :

You do understand that it remains a hardware limitation issue? This is putting a moped engine in a tow truck. And then hopes the moped engine will be more powerfull. Even normal ram is not efficient (slow). Cpu power is also slow. Gpu hardware is very efficient but also expensive. So even Vram is not the issue, even a rtx 2060 with 1 terabyte vram will be slow for this model, even a 5090. I hope you understand.

Administrator :

1 token every 10 seconds

Patrick2018 :

Runs at 1 token per day 😏

SmiteVils :

I tried Strata. It works well

Dr.Papa :

Dude r u serius ??? XD you will get max 1t/s it cannot be optimized… read how inference works

TMM :

I built something similar in 1995 I have a386dx PC with 2MB Ram and was unable to run Aladdin game because it required 4 MB ram, so I built a driver that would provide Ram on disk, the game ran and was 1 frame per second, I was very happy 😊

anthony_s2000 :

lol ramdrive.sys enters the chat.

HyperionX :

lol 10 tokens/s

⠀ ⠀ ⠀ ⠀ ⠀ ⠀ :

AI

loudr0ar :

So is it a stand alone app? Or will it run in lmstudio ?

🌍 Deans World 🌎 :

How much space do you need?

Ev1L Un1c0rN :

1 token per month?

sifudanferrera :

AI

MonoluminAI :

That is unusable 😓 Is it works yes… but so slow that it’s Mabe good only for students…

🎤❄️Holger❄️🎤 :

Yes, but it's very time consuming

Shannon Squatch :

AI

Mitchtok :

Soon we’ll be using nvidia GPUs as brick for our next houses 100%

Frontiersman :