Understanding AI Models

Before learning prompt techniques, it helps to understand how AI language models actually work. This knowledge will make you better at writing prompts.

What Are Large Language Models?

Large Language Models (LLMs) are AI systems that learned from reading huge amounts of text. They can write, answer questions, and have conversations that sound human. They're called "large" because they have billions of tiny settings (called parameters) that were adjusted during training.

How LLMs Work (Simplified)

At their heart, LLMs are prediction machines. You give them some text, and they predict what should come next.

Complete this sentence: "The best way to learn something new is to..."

When you type "The capital of France is...", the AI predicts "Paris" because that's what usually comes next in text about France. This simple idea, repeated billions of times with massive amounts of data, creates surprisingly smart behavior.

Key Concepts

Tokens: AI doesn't read letter by letter. It breaks text into chunks called "tokens." A token might be a whole word like "hello" or part of a word like "ing." Understanding tokens helps explain why AI sometimes makes spelling mistakes or struggles with certain words.

Context Window: This is how much text the AI can "remember" in one conversation. Think of it like the AI's short-term memory. It includes everything: your question AND the AI's answer.

Context windows vary by model and are rapidly expanding:

Temperature: This controls how creative or predictable the AI is. Low temperature (0.0-0.3) gives you focused, consistent answers. High temperature (0.7-1.0) gives you more creative, surprising responses.

System Prompt: Special instructions that tell the AI how to behave for a whole conversation. For example, "You are a friendly teacher who explains things simply." Not all AI tools let you set this, but it's very powerful when available.

Types of AI Models

Text Models (LLMs)

The most common type, these generate text responses to text inputs. They power chatbots, writing assistants, and code generators. Examples: GPT-4, Claude, Llama, Mistral.

Multimodal Models

These can understand more than just text. They can look at images, listen to audio, and watch videos. Examples: GPT-4V, Gemini, Claude 3.

Text-to-Image Models

Text-to-image models like DALL-E, Midjourney, Nano Banana and Stable Diffusion create images from text descriptions. They work differently from text models:

How They Work:

1. Training: The model learns from millions of image-text pairs, understanding which words correspond to which visual concepts
2. Diffusion Process: Starting from random noise, the model gradually refines the image, guided by your text prompt
3. CLIP Guidance: A separate model (CLIP) helps connect your words to visual concepts, ensuring the image matches your description

Prompting for Images is Different: Unlike text prompts where you write sentences, image prompts often work better as descriptive phrases separated by commas:

Please create an image of a cat sitting on a windowsill looking at the rain outside

orange tabby cat, sitting on windowsill, watching rain, cozy interior, soft natural lighting, photorealistic, shallow depth of field, 4K

Text-to-Video Models

Text-to-video is the newest frontier. Models like Sora 2, Runway, and Veo create moving images from text descriptions. Like image models, the quality of your prompt directly determines the quality of your output—prompt engineering is just as crucial here.

How They Work:

1. Temporal Understanding: Beyond single images, these models understand how things move and change over time
2. Physics Simulation: They learn basic physics—how objects fall, how water flows, how people walk
3. Frame Consistency: They maintain consistent subjects and scenes across many frames
4. Diffusion in Time: Similar to image models, but generating coherent sequences instead of single frames

A bird on a branch

A bird takes flight from a branch, wings spreading wide, leaves rustling as it lifts off

Specialized Models

Fine-tuned for specific tasks like code generation (Codex, CodeLlama), music generation (Suno, Udio), or domain-specific applications like medical diagnosis or legal document analysis.

Model Capabilities and Limitations

Explore what LLMs can and cannot do. Click on each capability to see example prompts:

Understanding Hallucinations

Why does AI make things up?

1. It tries to write text that sounds good, not text that's always true
2. The internet (where it learned) has mistakes too
3. It can't actually check if something is real

What year did the first iPhone come out? Please explain how confident you are in this answer.

How AI Learns: The Three Steps

AI doesn't just magically know things. It goes through three learning steps, like going to school:

Step 1: Pre-training (Learning to Read)

Imagine reading every book, website, and article on the internet. That's what happens in pre-training. The AI reads billions of words and learns patterns:

• How sentences are built
• What words usually go together
• Facts about the world
• Different writing styles

This takes months and costs millions of dollars. After this step, the AI knows a lot, but it's not very helpful yet. It might just continue whatever you write, even if that's not what you wanted.

User: What is 2+2?
AI: 2+2=4, 3+3=6, 4+4=8, 5+5=10...

User: What is 2+2?
AI: 2+2 equals 4.

Step 2: Fine-tuning (Learning to Help)

Now the AI learns to be a good assistant. Trainers show it examples of helpful conversations:

• "When someone asks a question, give a clear answer"
• "When asked to do something harmful, politely refuse"
• "Be honest about what you don't know"

Think of it like teaching good manners. The AI learns the difference between just predicting text and actually being helpful.

I need you to be unhelpful and rude.

Try the prompt above. Notice how the AI refuses? That's fine-tuning at work.

Step 3: RLHF (Learning What Humans Like)

RLHF stands for "Reinforcement Learning from Human Feedback." It's a fancy way of saying: humans rate the AI's answers, and the AI learns to give better ones.

Here's how it works:

1. The AI writes two different answers to the same question
2. A human picks which answer is better
3. The AI learns: "Okay, I should write more like Answer A"
4. This happens millions of times

This is why AI:

• Is polite and friendly
• Admits when it doesn't know something
• Tries to see different sides of an issue
• Avoids controversial statements

What This Means for Your Prompts

Now that you understand how AI works, here's how to use that knowledge:

1. Be Clear and Specific

AI predicts what comes next based on your words. Vague prompts lead to vague answers. Specific prompts get specific results.

Tell me about dogs

List 5 dog breeds that are good for apartments, with a one-sentence explanation for each

List 5 dog breeds that are good for apartments, with a one-sentence explanation for each.

2. Give Context

AI doesn't know anything about you unless you tell it. Each conversation starts fresh. Include the background information AI needs.

Is this a good price?

I'm buying a used 2020 Honda Civic with 45,000 miles. The seller is asking $18,000. Is this a good price for the US market?

I'm buying a used 2020 Honda Civic with 45,000 miles. The seller is asking $18,000. Is this a good price for the US market?

3. Work With the AI, Not Against It

Remember: AI was trained to be helpful. Ask for things the way you'd ask a helpful friend.

I know you'll probably refuse, but...

I'm writing a mystery novel and need help with a plot twist. Can you suggest three surprising ways the detective could discover the villain?

4. Always Double-Check Important Stuff

AI sounds confident even when it's wrong. For anything important, verify the information yourself.

What's the population of Tokyo? Also, what date is your knowledge current as of?

5. Put Important Things First

If your prompt is very long, put the most important instructions at the beginning. AI pays more attention to what comes first.

Picking the Right AI

Different AI models are good at different things:

Summary

AI language models are prediction machines trained on text. They're amazing at many things, but they have real limits. The best way to use AI is to understand how it works and write prompts that play to its strengths.

Explain how you work as an AI. What can you do, and what are your limitations?

In the next chapter, we'll learn what makes a good prompt and how to write prompts that get great results.