@ai.honeycove: LLM just made running 70 billion parameter AI models possible without supercomputers by loading layers from your hard drive. Its flash attention keeps memory low, letting complex models like LLAMA 3.3 70B run smoothly on MacBooks and gaming PCs. A game-changer for students and indie devs exploring AI. #llm #llama33b #aiinnovation #opensourceai #machinelearning

Mr G :

do I still pay for token usage on a open source llm?

Renzo :

What would this do to a hard drive though

statisticalvoid :

10 seconds per token

super nabilion :

bla bla bla you need huge gpu 24g ... impossible

DogsAreTheBest :

one token per sometimes?

petar⚡️ :

one token per some years

Bon Beau Boudreaux :

everyone complains about token speed. maybe you need to ask it the most precise and comprehensive question using a very precise format with output in a particular compressed format. think about it. what if your output was version that was able to compress several responses into a few very small token group?

androiduser525 :

I already run this llama 70B locally on my asus gaming laptop with 64GB RAM and 8 GB vram. it runs slow. I dont know that this method would run faster

Ryan :

/yawn, so last month, collibri is far better, we have massive ai compute and we're building shit in python? sigh

Paul Grinstead :

One token per century.

DimiDi :

Ai companies going broke +Nvidia

KBN :