@hatsunama: Local Ai is getting easier. I told you the hardware will catch up eventually. #ai #news #llm #localai #upgrade

ALIENDISTRICT51™️ :

that's insane. I tried ollama today and oh boy let me tell you "it sucked". 😆

fudgefive :

"super intelligence"... it's a word-guessing-algorithm..

pearbear :

can I put it in a Raspberry Pi? or a Mac mini?

bendrinking :

I got qwen3.6 35b MoE on a pi as a hybrid. It is pretty cool I get 5.6 t/s. And a 9B dense on the same pi. Will be making a vid soon the 9B is all on a NPU😁😁

the polymath guild :

Yesssss I’m doing it right meow have been since 2010

RadiantLumens :

could they be daisy chained?

Athion :

A 30m parameter model isn’t capable of much, it’s barely capable of complete sentences … but it is a really cool advancement.. and shows where AI is headed.

Shortware :

Thats… not… super intelligent

bhickory :

29m parameters.. how clippy

surafad :

29m, are you serious

John Minton :

See we’re looking at like 3 billion parameter models and getting not that great of performance out of them. But a 30 million parameter model running on an ESP32S3 - which only cost five dollars - I have like 15 of these. How can I run a large larger model across multiple of them simultaneously? If you can figure that out, I’ll buy 100 of them and then we can have a 29 billion parameter model run across 100 ESP32S3s. Something tells me the overhead of the mesh network capability. Wood screw this up however, with a mesh network, you could probably find a way for it to self coordinate. If each layer of the model could fit on a single ESP then each ESP would take its parameters, run its computations, and forward the result off to the next ESP to process the next layer. I’m a software engineer, and even this is a little bit over my head. But at cost you could get them I think for like $1.50 apiece. If you have them all mounted to a nice board packaged up, it would cost you less than $200 to manufacture at scale

The_Real_Technician :

does it run nano gpt models?

i.still.want.to.believe :

github.com/slvDev/esp32-ai and it’s based on Karpathy’s llama2.c

Exler :

Fake news. The creativity of the mind will always set the trajectory of the content. 👌🏻

🦊 :

can't answer guestions or code.

crawford3737☯️ :

I'm just trying to find one. That'll help me write code and make coding a whole lot faster. That's why you can piece it together. And it won't break when I go to make tools for myself. I'm not trying to do all that fancy extra stuff, even though it's really cool

LearnAsUGrow501 :

25M is good for conversation. when it crashes on a loop. start a fresh chat.

AdamInFlorida :

Awesome

Johnohhh1 :

ok so Im back. And the best you can hope for is complete Once upon a time... it would struggle to tell you a story.

Killian :

Frontier intelligence is (ballpark) 5 or 6 orders of magnitude larger. it's definitely going in the right direction though.

💃🏼🦍💃🏼🦍💃🏼🦍 :

nice but I'm runnig 3.5 gb Gemma on my ipad, works great!

E:P:i RE🔻OLUTION \-_-/ :

Logose :

True, and somehow no one's life, except for the Billionaires that own AI, is better for it.

loudr0ar :

Million not billion😌

Llambit :

Would be v curious to see how this would do in swarm