davidkimai / Context-Engineering

Context Engineering > **"Context engineering is the delicate art and science of filling the context window with just the right information for the next step." — **Andrej Karpathy**** > > **Software Is Changing (Again) Talk @YC AI Startup School** ## DeepGraph Chat with NotebookLM + Podcast Deep Dive Comprehensive Course Under Construction > ### **Context Engineering Survey-Review of 1400 Research Papers** > > **Awesome Context Engineering Repo** Operationalizing the Latest Research on Context With First Principles & Visuals — July 2025 from ICML, IBM, NeurIPS, OHBM, and more > **"Providing “cognitive tools” to GPT-4.1 increases its pass@1 performance on AIME2024 from 26.7% to 43.3%, bringing it very close to the performance of o1-preview."** — **IBM Zurich** **Support for Claude Code | OpenCode | Amp | Kiro | Codex | Gemini CLI** Context Engineering Survey-Review of 1400 Research Papers | Context Rot | IBM Zurich | Quantum Semantics | Emergent Symbolics ICML Princeton | MEM1 Singapore-MIT | LLM Attractors Shanghai AI | MemOS Shanghai | Latent Reasoning | Dynamic Recursive Depths A frontier, first-principles handbook for moving beyond prompt engineering to the wider discipline of context design, orchestration, and optimization. Definition of Context Engineering > **Context is not just the single prompt users send to an LLM. Context is the complete information payload provided to a LLM at inference time, encompassing all structured informational components that the model needs to plausibly accomplish a given task.** > > — **Definition of Context Engineering from A Systematic Analysis of Over 1400 Research Papers** Why This Repository Exists > **"Meaning is not an intrinsic, static property of a semantic expression, but rather an emergent phenomenon" — Agostino et al. — July 2025, Indiana University** Prompt engineering received all the attention, but we can now get excited for what comes next. Once you've mastered prompts, the real power comes from engineering the **entire context window** that surrounds those prompts. Guiding thought, if you will. This repository provides a progressive, first-principles approach to context engineering, built around a biological metaphor: > "Abstraction is the cost of generalization"— **Grant Sanderson (3Blue1Brown)** *A Survey of Context Engineering - July 2025* **On Emergence, Attractors, and Dynamical Systems Theory | Columbia DST** https://github.com/user-attachments/assets/9f046259-e5ec-4160-8ed0-41a608d8adf3 Quick Start • **Read ** (5 min) Understand why prompts alone often underperform • **Run ** (Jupyter Notebook style) Experiment with a minimal working example • **Explore ** Copy/paste a template into your own project • **Study ** See a complete implementation with context management Learning Path What You'll Learn | Concept | What It Is | Why It Matters | |---------|------------|----------------| | **Token Budget** | Optimizing every token in your context | More tokens = more $$ and slower responses | | **Few-Shot Learning** | Teaching by showing examples | Often works better than explanation alone | | **Memory Systems** | Persisting information across turns | Enables stateful, coherent interactions | | **Retrieval Augmentation** | Finding & injecting relevant documents | Grounds responses in facts, reduces hallucination | | **Control Flow** | Breaking complex tasks into steps | Solve harder problems with simpler prompts | | **Context Pruning** | Removing irrelevant information | Keep only what's necessary for performance | | **Metrics & Evaluation** | Measuring context effectiveness | Iterative optimization of token use vs. quality | | **Cognitive Tools & Prompt Programming** | Learm to build custom tools and templates | Prompt programming enables new layers for context engineering | | **Neural Field Theory** | Context as a Neural Field | Modeling context as a dynamic neural field allows for iterative context updating | | **Symbolic Mechanisms** | Symbolic architectures enable higher order reasoning | Smarter systems = less work | | **Quantum Semantics** | Meaning as observer-dependent | Design context systems leveraging superpositional techniques | Karpathy + 3Blue1Brown Inspired Style > For learners of all experience levels • **First principles** – start with the fundamental context • **Iterative add-on** – add only what the model demonstrably lacks • **Measure everything** – token cost, latency, quality score • **Delete ruthlessly** – pruning beats padding • **Code > slides** – every concept has a runnable cell • **Visualize everything** — every concept is visualized with ASCII and symbolic diagrams Research Evidence Memory + Reasoning **MEM1: Learning to Synergize Memory and Reasoning for Efficient Long-Horizon Agents - Singapore-MIT June 2025** > “Our results demonstrate the promise of reasoning-driven memory consolidation as a scalable alternative to existing solutions for training long-horizon interactive agents, where both efficiency and performance are optimized." — Singapore-MIT • **MEM1 trains AI agents to keep only what matters—merging memory and reasoning at every step—so they never get overwhelmed, no matter how long the task.** • **Instead of piling up endless context, MEM1 compresses each interaction into a compact “internal state,” just like a smart note that gets updated, not recopied.** • **By blending memory and thinking into a single flow, MEM1 learns to remember only the essentials—making agents faster, sharper, and able to handle much longer conversations.** • **Everything the agent does is tagged and structured, so each action, question, or fact is clear and easy to audit—no more mystery meat memory.** • **With every cycle, old clutter is pruned and only the latest, most relevant insights are carried forward, mirroring how expert problem-solvers distill their notes.** • **MEM1 proves that recursive, protocol-driven memory—where you always refine and integrate—outperforms traditional “just…