Daily Dose of Data Science

Over the last twelve parts of this RAG systems course, we built RAG systems from the ground up.

The agent loop inside Claude Code is nothing special. It sends the model a task, runs whatever tools the model requests, feeds the results back, and repeats until there’s a final answer.

Anyone can write that loop in thirty lines. But a bare version of it falls apart on a real codebase, reading the wrong files, losing the goal midway, and filling its context with outpu...

Over the past twelve chapters, we understood the key machinery of reinforcement learning piece-by-piece, from bandits and value functions all the way to GRPO, judges, and agentic training loops. With Chapter 12, that machinery became complete.

So this final chapter does something different. There is nothing new to derive and nothing to train. Instead, we lo...

Andrej Karpathy summarized the entire history of LLM training in three nouns:

Pretraining ran on internet text, supervised fine-tuning ran on curated conversations, and the current era of reinforcement learning runs on environments.

OpenAI’s o1 proved that framing by training on math and coding problems with verifiable answers, an...

In Chapter 11, we solved the problem that RLVR left open. Verifiers only exist for tasks with an answer key and most agent work has no such key. So we asked where the reward comes from when the environment stays silent.

We started with an observation about GRPO itself. The optimizer consumes one number per response and normalizes within the group. It never inspect...