Machine learning in Clay Cloud
A short book for readers who know software architecture but not machine learning: what the workload is, where the time goes, and how Clay's design handles it.
Status: a local prototype. GPT-2 small runs as Clay code on a native tensor host on one developer machine, CPU only. Nothing is hosted, and nothing here has been measured on a GPU, in a browser, or on a server. Every number names the repository document it comes from, and is labelled measured, computed, inferred or illustrative.
- 1. What is a model, really?Weights plus a recipe; vectors, matrices, tensors; what "124M parameters, 548 MB" means. Animation: matrix multiply.
- 2. How a language model writes textTokens, embeddings, attention, the one-token loop, the KV cache. Animations: attention table, token loop with a growing cache.
- 3. Where the time goesCompute versus memory bandwidth, why one token is memory-bound, batching, our measured numbers. Pictures: roofline, time split.
- 4. How Python does it, and how Clay does itGlue over native kernels; the Tensor ability; handlers as backends; real code. Animation: swap the handler.
- 5. Models as content-addressed thingsWeights by hash at a pinned revision, the registry, why floats differ across machines, licences.
- 6. Porting modelsThe reference-diff harness and the real GPT-2 result, growing the operation set, PyTorch versus tinygrad, what agents can do.
- 7. Decisions, trade-offs and open questionsEvery decision with its status and alternatives, linked to its source document.
- 8. Status and glossaryWhat exists, what is measured, every term used.
The design behind this book is in the repository: docs/ml-workloads.md (design and rungs), docs/ml-m0-scope.md (scope, operations, harness), docs/ml-m2-gpt2.md (the GPT-2 port and its measurements). The pages are plain HTML, one file per chapter, with the shared styles in ml.css and each animation in its own viz-*.js using ml-anim.js.