GitHub radar
Build your own GPT model from scratch and learn how LLMs really work
This is the code behind the book Build a Large Language Model (From Scratch): step by step you build a GPT-like model yourself and start understanding what happens inside the neural networks you already use every day.
A set of notebooks and Python code (interactive files mixing code and explanations) that walk you through writing a language model from scratch — from splitting text into pieces to training and fine-tuning it.
When it helps
Useful if you want to understand how AI models work, not just use them. Skip it if you need a finished product right now — all you get here is knowledge.
Pros
- Runs on a regular laptop, no powerful GPU required
- Actively maintained: 107,000+ GitHub stars, recent updates
- Bonus notebooks cover tokenizers and attention mechanisms too
Cons
- Hard to finish alone without any programming experience
- You end up with no finished product — only understanding
How to set it up — step by step
- 1Open github.com/rasbt/LLMs-from-scratch and download it with git clone --depth 1 https://github.com/rasbt/LLMs-from-scratch.git
- 2Open the chapter 2 notebook (ch02) — a file mixing code and explanations you run step by step
- 3Send the agent the text from the block below
- 4Read the agent's line-by-line explanation and make sure you understand each part of the code
- 5Move to the next chapter's notebook only after you understand the current one
- 6A GPU isn't required — the code runs on a regular laptop too, just slower
Text for your agent
Copy this and send it to your agent — Claude Code, Codex, any of them:
Open the repository https://github.com/rasbt/LLMs-from-scratch, take the chapter 2 notebook (ch02), and walk me through what the code does and why, line by line, before moving to the next chapter.
Runs on a regular laptop — the main chapters' code works in Python without a GPU; a graphics card just speeds training up if you have one.
Open on GitHub▌ More finds