GitHub radar
WebLLM: Run LLMs Directly in the Browser, No Server Needed
A high-performance in-browser LLM inference engine powered by WebGPU. Add AI chat to any website — model runs in the visitor's browser with no server, no API costs, no data leaving the device.
WebLLM is an open-source, high-performance in-browser LLM inference engine from MLC AI, powered by WebGPU for hardware-accelerated computation. Language models run entirely inside the visitor's browser — no server is required, no data is sent anywhere. It is fully compatible with the OpenAI API, so existing code using openai.chat.completions.create() requires minimal changes to run locally. Supported model families include Llama 3, Phi 3, Gemma, Mistral, and Qwen. It installs as an npm package (@mlc-ai/web-llm) and runs computation in a Web Worker or Service Worker to keep the UI responsive. A live demo at chat.webllm.ai lets you try it without any installation.
Why a vibe-coder should care
Instead of paying for every API call or running your own inference server, visitors run the model entirely in their browser. Zero hosting costs for AI inference, and for privacy-sensitive applications, no user data ever leaves their device.
How to install
Copy this and send it to your agent — Claude Code, Codex, any of them:
Add an in-browser AI chat to my site with no server: https://github.com/mlc-ai/web-llm — use the @mlc-ai/web-llm npm package per the README, show me a working chat example with Llama, ask if you need any keys
Runs in the browser on a regular laptop — WebGPU support required (Chrome or Edge on modern Mac and Windows). Safari and mobile devices support WebGPU with limitations.
Open on GitHub▌ More finds