AI Theory (LLM Foundations from Scratch)

Build a rigorous foundation in modern AI by implementing a small language model from scratch. The program covers BPE tokenization; the Transformer architecture, including self-attention, RoPE, and RMSNorm; end-to-end pretraining with cross-entropy, AdamW, mixed precision, and gradient clipping; efficient training with FlashAttention, activation checkpointing, and parallelism; scaling laws and pretraining-data curation; inference with KV-cache and quantization; and post-training with SFT, LoRA, DPO, and RLHF.

Weekly PyTorch and Hugging Face labs run on a free GPU. Assessment includes a midterm, a comprehensive written final, graded coding labs, and a capstone in which you train, evaluate, and fine-tune your own small language model.

The four-week program is taught in English and includes two two-hour live sessions each week plus an approximately two-hour asynchronous lab per session. Grading is Pass/Fail, with 70% required to pass.