@forcceee_: Colibri just proved you can run a 744 billion parameter AI model on a laptop without a GPU by streaming parts from your hard drive. This smart approach slashes costs and works offline. #AI #AITools #TechInnovation #MachineLearning #Colibri

Samuel :

1token per lightyear

user3594407214355 :

from 0.05 tokens per second to max 1 token per sec... it is painfully slow.

MrJonny :

Yeah, getting tired of these claims. speed does matter, it is useless if you need to wait longer than half an hour just for an answer!

Darwin. :

slow even f u have 5090

sergeibluefur :

At 0.7 tokens per second, it feels more like posting on a forum than actual chatting.

⟬♡Ⓐ☭𝑅𝑖𝑣𝑒𝑟 𝐺𝑟𝑒𝑒𝑛☭Ⓐ♡⟭ :

my ssd is 256gb..

theebbemonster :

Tokens/second?

ab47d547 :

Called it. This is the future

🛡️ sToRm// 🎖️ :

Yeah...haha... and if you're using SSD or Nvme you will burn your hardware in weeks for a model slow like shit...

Zed :

Maybe running smaller models would be a better way to go?

Alex Tudor Bartesch :

At 3 tk/s 😂😂😂

شیخانی - Shekhani :

if I buy that nividia latest hardware I'm I be able to do it ?

Mavola79 :

Well, on the good side, while waiting, I could use the time to learn rust 😂😂

Uptix :

wildy misleading, it will be so glacially slow, its useless

Rove Monteux :

I'm already running the latest Gemma with 26B parameters like that on Ollama, without a GPU at all on a remote Debian server at 4 tokens/sec, what's the difference in practice?

fatinho1982 :

detail

simoncrawford4 :

Game changer

user3594407214355 :

from 0.05 to 0.3 token per second. max is 1 t/s

OnlyPips journal :

Tik Tok :

370 Gb....

moccadmd :

It’s speed is measured seconds per tokens rather tokens per second this is nothing but nonsense

Gianni🇫🇷 :

« That beats the model you pay for» and it nearly beats opus 4.5…. A gen old

Englishrosewow :

he did not at all

DrSales :

details

AM :

Details in comments...me