Rivflyyy / HappyTorch

HappyTorch **A PyTorch coding practice platform — covering LLM, Diffusion, PEFT, RLHF, and more** *Like LeetCode, but for tensors. Self-hosted. Supports both Jupyter and Web interfaces. Instant auto-grading feedback. No GPU required.* 中文版 README > **News** > - 2026-03-16: Thanks to damaoooo for reporting the notebook matching bug (attention vs multihead_attention). Replaced fragile suffix-based matching with an explicit name mapping for long-term reliability. > - 2026-03-13: Thanks to wavetao2010 for adding Docker image support with dual-mode (Web/Jupyter) and pre-built images. > - 2026-03-12: Web UI now groups problems by category (Fundamentals, Attention, RLHF, etc.) with collapsible sections in the sidebar for easier topic-based practice. > - 2026-03-10: Thanks to SongHuang1 for contributing the MLP XOR training problem (pure NumPy, manual forward + backward). Fixed Web UI issues: class-based tasks (LoRA, SwiGLU, etc.) now work correctly, added / / / to execution namespace, fixed OpenMP crash on Windows, added MHA solution lookup, added 60s request timeout. > - 2026-03-09: Thanks to chaoyitud for adding ML and RLHF practice problems. Thanks to fiberproduct for fixing . Welcome everybody to contribute more problems! > - 2026-03-06: The plugin happytorch-plugin has been released. --- Why HappyTorch? If you're learning deep learning or preparing for ML interviews, you might have encountered these challenges: • You've read many papers, but don't know where to start when it comes to implementing things from scratch • You're asked to implement or in an interview, and your mind goes blank • You want to deeply understand Transformer, LoRA, Diffusion, RLHF, but lack systematic practice **HappyTorch** provides a friendly hands-on practice environment with **36 curated problems**, from basic activation functions to complete Transformer components and RLHF algorithms. | Feature | Description | |---------|-------------| | **36 curated problems** | From basics to advanced, covering mainstream deep learning topics | | **Auto-grading** | Instant feedback showing what you got right and where to improve | | **Two interfaces** | LeetCode-like Web UI (Monaco Editor) or Jupyter notebooks | | **Helpful hints** | Get nudges when stuck, not full spoilers | | **Reference solutions** | Compare and learn after your own attempt | | **Progress tracking** | Record your learning journey | --- Quick Start Docker Progress data ( ) is persisted via Docker volume. --- Web Mode A LeetCode-like practice interface with: • **Monaco Editor** — VS Code's editor with Python syntax highlighting • **Random / Sequential Mode** — Get random unsolved problems or work through them in order • **Instant Testing** — Run tests with one click ( ) • **Solution Tab** — View reference solutions with markdown explanation and copyable code • **Progress Dashboard** — Track solved / attempted / todo status • **Dark Theme** — Modern, eye-friendly interface --- Problem Set (36 Problems) Fundamentals | # | Problem | Function / Class | Difficulty | Key Concepts | |:-:|---------|-----------------|:----------:|--------------| | 1 | ReLU | | | Activation functions, element-wise ops | | 2 | Softmax | | | Numerical stability, exp/log tricks | | 3 | Linear Layer | | | y = xW^T + b, Kaiming init, nn.Parameter | | 4 | LayerNorm | | | Normalization, affine transform | | 7 | BatchNorm | | | Batch vs layer statistics, train/eval | | 8 | RMSNorm | | | LLaMA-style norm, simpler than LayerNorm | Attention Mechanisms | # | Problem | Function / Class | Difficulty | Key Concepts | |:-:|---------|-----------------|:----------:|--------------| | 5 | Scaled Dot-Product Attention | | | softmax(QK^T/sqrt(d_k))V | | 6 | Multi-Head Attention | | | Parallel heads, split/concat, projections | | 9 | Causal Self-Attention | | | Autoregressive masking, GPT-style | | 10 | Grouped Query Attention | | | GQA (LLaMA 2), KV sharing | | 11 | Sliding Window Attention | | | Mistral-style local attention | | 12 | Linear Attention | | | Kernel trick, O(n*d^2) | Full Architecture | # | Problem | Function / Class | Difficulty | Key Concepts | |:-:|---------|-----------------|:----------:|--------------| | 13 | GPT-2 Block | | | Pre-norm, causal MHA + MLP, residual | Modern Activation Functions *(V2)* | # | Problem | Function / Class | Difficulty | Key Concepts | |:-:|---------|-----------------|:----------:|--------------| | 14 | GELU | | | Gaussian CDF, erf, BERT/GPT/DiT | | 15 | SiLU (Swish) | | | x * sigmoid(x), LLaMA component | | 16 | SwiGLU | | | Gated activation, LLaMA MLP | Parameter-Efficient Fine-Tuning *(V2)* | # | Problem | Function / Class | Difficulty | Key Concepts | |:-:|---------|-----------------|:----------:|--------------| | 17 | LoRA | | | Low-rank BA, zero-init B, alpha/r scaling | | 18 | DoRA | | | Weight decomposition, magnitude + direction | Conditional Modulation — Diffusion *(V2)* | # | Problem | Function / Class | Difficulty | Key Concepts | |:-:|---------|-----------------|:----------:|--------------| | 19 | AdaLN | | | Adaptive LayerNorm, DiT-style | | 20 | AdaLN-Zero | | | Zero-init gate, stable training | | 21 | FiLM | | | Feature-wise modulation | LLM Inference *(V2)* | # | Problem | Function / Class | Difficulty | Key Concepts | |:-:|---------|-----------------|:----------:|--------------| | 22 | RoPE | | | Rotary embedding, 2D rotation | | 23 | KV Cache | | | Incremental caching for generation | Diffusion Training *(V2)* | # | Problem | Function / Class | Difficulty | Key Concepts | |:-:|---------|-----------------|:----------:|--------------| | 24 | Sigmoid Schedule | | | S-curve noise schedule | ML Fundamentals & Decoding *(V3 — Community)* | # | Problem | Function / Class | Difficulty | Key Concepts | |:-:|---------|-----------------|:----------:|--------------| | 25 | K-Means Clustering | | | Iterative centroid update, assignment | | 26 | K-Nearest Neighbors | | | Distance-based classification | | 27 | MLP Backward…