GitHub radar
Edge0: Running a 35B Model on Your Normal Computer
Edge0’s open source inference engine loads just what it needs into memory while streaming the rest off your SSD — let me explain how this can save you money on your API costs.
Edge0 is an open source inference engine developed by the team at Edge0 for running large mixture of experts models on macOS with Apple Silicon. Rather than keeping the entire model in memory, Edge0 streams required pieces from the SSD and anticipates future needs, leading to up to 59% faster generations according to its creators. The package comes with two pre-trained models: a 35 billion parameter model (23GB disk size, ~2.9GB max memory usage) and an 8B parameter model (~4.2GB disk size). These models are based on open source base models and feature adapters to regain lost quality due to compression. Currently, Edge0 is only available for macOS with M1 through M4 processors.
Why a vibe-coder should care
So what does this mean for you? Well, the real question is this: how many times per day do you make API calls for tasks that can be completed locally? If your agents can complete these tasks using local models, you won’t have to pay any token fees, and your data won’t be sent over the internet. This is a preview release, so start by testing it out on one repeated task and seeing how it compares to your current paid model.
How to install
Copy this and send it to your agent — Claude Code, Codex, any of them:
Set up Edge0 on my Mac from https://github.com/Edge0-AI/Edge0 — follow the README, start with the 8B model, run it and show me how to ask it something, then tell me whether my Mac can handle the 35B one.
Mac on Apple Silicon (M1–M4): the large model is ~23 GB on disk and peaks at ~3 GB of RAM; the small model is ~4.2 GB on disk. Windows and Linux are not yet supported.
Open on GitHub▌ More finds