GitHub radar
Run 80B LLMs on Your Mac or iPhone
Open-source Swift and Metal runtime that streams a 35B or 80B Qwen MoE model from disk on demand, keeping only 2–3 GB of RAM active — so an M-series Mac or an iPhone 17 runs a large language model entirely offline.
Swiftlet is a Swift and Metal inference runtime for the Qwen3-Next and Qwen3.5/3.6 Mixture-of-Experts model family, released by Alibaba's Qwen team under Apache 2.0. It keeps only the small dense core of the model in RAM and streams the sparse expert weights from local storage as each token is generated, which lets a 35B model run in 2.6 GB of RAM on an M5 Mac at 7–11 tokens per second. The same model runs on an iPhone 17 in about 2.5 GB of RAM at roughly 1 token per second. It drew attention because very few open runtimes stream full-scale MoE weights from disk rather than loading the entire model into memory.
Why a vibe-coder should care
If you build with AI locally and worry about paying for API calls or sending sensitive data to the cloud, Swiftlet lets you run a genuinely capable large model on the Mac you already own — no subscription, no internet required. It is also the starting point for building apps that keep AI inference private on Apple devices.
▌ More finds