GitHub radar
28.9M-Param LLM on an $8 ESP32 Chip
A developer managed to pack a 28.9-million-parameter language model into an ESP32-S3 microcontroller. This little chip (costing ~$8) only has 512 KB RAM, but thanks to a clever memory optimization used in Google’s Gemma models, it still manages to generate ~9.5 tokens/sec of text completely on device.
A developer managed to run a 28.9-million parameter language model on an ESP32-S3 microcontroller. This little chip (costing ~$8) only has 512 KB RAM, but thanks to a clever memory optimization used in Google’s Gemma models, it still manages to generate ~9.5 tokens/sec of text completely on device. Previous LLMs have been much smaller running on these devices (~260 thousand parameters), this model is ~100x larger. This is achieved using Per-Layer Embeddings, which was designed for the Gemma 3n and Gemma 4 models. Most of the model is stored in slow flash memory and only the small fraction that is needed for a particular token is read by the device, so most of the RAM is left unused.
Why a vibe-coder should care
This demonstrates how AI can be run on very affordable hardware and can be completely cloud independent (no API key, no internet, no monthly fee). Right now the model is limited to producing short simple stories, but the memory optimization provides an opportunity for more intelligent AI in common items that are cheaper than a cup of coffee.
▌ More finds