Context Engineering

Understanding context is essential for building AI applications that actually work. This chapter covers everything you need to know about giving AI the right information at the right time.

What is Context?

Context is all the information you give to AI alongside your question. Think of it like this:

What's the status?

You are a project manager assistant. The user is working on Project Alpha, which is due Friday. The last update was: 'Backend complete, frontend 80% done.'

User: What's the status?

Without context, the AI has no idea what "status" you're asking about. With context, it can give a useful answer.

The Context Window

Remember from earlier chapters: AI has a limited "context window" - the maximum amount of text it can see at once. This includes:

AI is Stateless

This is why chatbots send your entire conversation history with each message. It's not that the AI remembers - it's that the app re-sends everything.

Pretend this is a new conversation with no history.

What did I just ask you about?

The AI will say it doesn't know because it truly doesn't have access to any previous context.

RAG: Retrieval-Augmented Generation

RAG is a technique for giving AI access to knowledge it wasn't trained on. Instead of trying to fit everything into the AI's training, you:

1. Store your documents in a searchable database
2. Search for relevant documents when a user asks a question
3. Retrieve the most relevant pieces
4. Augment your prompt with those pieces
5. Generate an answer using that context

Why RAG?

Embeddings: How Search Works

How does RAG know which documents are "relevant"? It uses embeddings - a way to turn text into numbers that capture meaning.

What Are Embeddings?

An embedding is a list of numbers (a "vector") that represents the meaning of text. Similar meanings = similar numbers.

Semantic Search

With embeddings, you can search by meaning, not just keywords:

Query: 'return policy'
Finds: Documents containing 'return' and 'policy'
Misses: 'How to get a refund'

Query: 'return policy'
Finds: All related documents including:
- 'Refund guidelines'
- 'How to send items back'
- 'Money-back guarantee'

This is why RAG is so powerful - it finds relevant information even when the exact words don't match.

Function Calling / Tool Use

Function calling lets AI use external tools - like searching the web, checking a database, or calling an API.

How It Works

1. You tell the AI what tools are available
2. AI decides if it needs a tool to answer
3. AI outputs a structured request for the tool
4. Your code runs the tool and returns results
5. AI uses the results to form its answer

You have access to these tools:

1. get_weather(city: string) - Get current weather for a city
2. search_web(query: string) - Search the internet
3. calculate(expression: string) - Do math calculations

User: What's the weather like in Tokyo right now?

Think step by step: Do you need a tool? Which one? What parameters?

Summarization: Managing Long Conversations

As conversations get longer, you'll hit the context window limit. Since AI is stateless (it doesn't remember anything), long conversations can overflow. The solution? Summarization.

The Problem

Message 1 (500 tokens)
Message 2 (800 tokens)
Message 3 (600 tokens)
... 50 more messages ...
────────────────────
= 40,000+ tokens
= OVER THE LIMIT!

[Summary]: 200 tokens
Recent messages: 2,000 tokens
Current query: 100 tokens
────────────────────
= 2,300 tokens
= Fits perfectly!

Summarization Strategies

Different approaches work for different use cases. Click each strategy to see how it processes the same conversation:

What to Capture in Summaries

A good conversation summary preserves what matters:

Try It: Create a Summary

Summarize this conversation for context management. The summary will replace the full conversation in the AI's memory.

CONVERSATION:
User: Hi, I'm learning Python for data analysis
Assistant: Welcome! Python is great for data analysis. What's your current experience level?
User: I know basic Excel. Complete beginner with programming.
Assistant: Perfect starting point! Let's begin with variables - they're like Excel cells that store data.
User: Can you explain variables?
Assistant: Variables are containers for storing data. In Python: name = "Alice" or age = 25
User: What about lists? I need to handle multiple values.
Assistant: Lists are like Excel columns! Create one like: prices = [10, 20, 30]. Access items with prices[0].
User: Can I do calculations on lists?
Assistant: Yes! Use sum(prices), len(prices), or max(prices). For complex analysis, we'll use pandas later.
User: What's pandas?
Assistant: Pandas is a library for data analysis - think "Excel on steroids". It has DataFrames (like spreadsheets).

CREATE A SUMMARY that captures:
1. User's goal and background (1 sentence)
2. Topics covered so far (1 sentence)  
3. User's learning style/preferences (1 sentence)
4. What to cover next (1 sentence)

When to Summarize

You are managing a conversation's context window. Given these conditions, decide when to trigger summarization:

CONTEXT WINDOW: 8,000 tokens max
CURRENT USAGE:
- System prompt: 500 tokens
- Conversation history: 6,200 tokens  
- Buffer for response: 1,500 tokens

RULES:
- Summarize when history exceeds 70% of available space
- Keep last 5 messages intact
- Preserve all user preferences and decisions

Should you summarize now? If yes, what messages should be summarized vs kept intact?

MCP: Model Context Protocol

MCP (Model Context Protocol) is a standard way to connect AI to external data and tools. Instead of building custom integrations for each AI provider, MCP provides a universal interface.

Why MCP?

MCP Provides

• Resources: Data the AI can read (files, database records, API responses)
• Tools: Actions the AI can take (search, create, update, delete)
• Prompts: Pre-built prompt templates

Building Context: The Complete Picture

--- SYSTEM PROMPT ---
You are a helpful customer support agent for TechStore. Be friendly and concise.

--- RETRIEVED DOCUMENTS (RAG) ---
From knowledge base:
- Return policy: 30 days, original packaging required
- Shipping: Free over $50
- Warranty: 1 year on electronics

--- CONVERSATION HISTORY ---
[Summary] User asked about order #12345. Product: Wireless Mouse. Status: Shipped yesterday.

User: When will it arrive?
Assistant: Based on standard shipping, it should arrive in 3-5 business days.

--- USER QUERY ---
Can I return it if I don't like it?

Best Practices

Summary

Context engineering is about giving AI the right information:

• AI is stateless - include everything it needs every time
• RAG retrieves relevant documents to augment prompts
• Embeddings enable semantic search (meaning, not just keywords)
• Function calling lets AI use external tools
• Summarization manages long conversations
• MCP standardizes how AI connects to data and tools