← All news·2026-09-30·3 min read

YC Startup Magnitude Enables Faster Local Models for AI Agents

YC S25 startup launches Magnitude, an open source local inference engine that optimizes for your device’s hardware & integrates with Claude Code + other agents for free!

aiopensource

I think this YC startup is onto something that I personally care about: local inference. They launched an open source local inference engine for AI models called Magnitude that optimizes computation for your device’s hardware. For example, they estimate that Apple Metal will see a 92% improvement in generation speed and 9% improvement in prompt handling compared to llama.cpp. If you’re using NVIDIA CUDA, you’ll see a 19% and 23% improvement in generation speed and prompt handling respectively, along with a 27% reduction in memory used per agent. Best of all, it’s free and licensed under the Apache 2.0 license. It also integrates effortlessly (just one click!) with Claude Code, Codex, OpenCode, etc., or any other tool through an OpenAI compatible API.

It won’t make open models better, but it will make them run faster on your hardware setup. That’s nice if you want to use an open model, but often times that’s not enough for agency work. To my knowledge, there aren’t any free open models that can match the performance of top paid models for agency work. But if you’re just playing around, doing basic repeatable tasks, or just don’t have the money for a subscription, then being able to run a faster, lower memory local inference engine could be useful. It won’t try to replace a high priced model either, which is nice. And being able to quickly integrate with Claude Code and Codex is pretty cool because you can see how well a local model works for your specific use case without interrupting your main workflow.

Source: github.com

Free course

Stop reading about AI — start building with it

The free Claude Code course: your first site, tool or game — no coding. No upsells, no cross-sells — nothing to buy here.

Start free →
EAEvgenii Arsentev

Author

Evgenii Arsentev

PhD · Founder & CEO, Dad4Kids