About

fix-the-pipeline is a practice site for AI architecture: the design of systems built around language models. It works like LeetCode or Project Euler, but each problem is a small AI system with a flaw for you to fix.

What You Practice

  • AI basics: where AI fits into existing work, which kind of AI suits a task, and when it's safe to use.
  • Cost control: spending less per request without losing quality.
  • Token and cache economics: counting and pricing tokens, and reusing them with prompt caching.
  • Memory: what a system remembers between turns, and how.
  • Context and prompt engineering: what goes into the model, and how.
  • Model selection: the right model for each step.
  • Evals: testing AI output automatically, and catching regressions.

How the Simulator Works

Most problems run against a simulated model, so you need no API key and pay nothing. A simulation can't judge prose, but it can price tokens, model caches and count right answers exactly. Every problem states its rules and prices openly: working out what they mean is the puzzle, not a trick. The same rules run in your browser for instant results and on our server for the result that counts.

Your numbers come from your own variant: the same problem with different values for each learner. That's why the server checks only your final answer.

Formats

  • Multiple choice: pick an answer. A wrong pick tells you why it's wrong.
  • Exact answer: work out a number, within a stated tolerance.
  • Config: edit a YAML or JSON file, then run it against the simulator.
  • Code: write Python behind a fixed interface (coming later).

Some problems will be live: they run on a real model through your own API key. Your key stays in your browser and goes straight to the provider, never to our server.

Points

A problem is worth up to 10 points per level of difficulty (1–10). Wrong multiple choice and exact answers lower what a right one earns. On config and code problems you can run as often as you like: passing earns half the points, and beating the pass line earns the rest. Hints cost 10% of the maximum each.

Browse the problems or pick a learning set.