How AI Actually Works
Time: ~10 minutes What you'll learn: The mental model that changes how you use AI
This Page Covers
- The Mental Model - AI predicts words based on patterns, not "thinking"
- What Are Models - Understanding GPT-4, Claude, Gemini and what new releases mean
- Hallucinations - Why AI is confident even when wrong
- Context Windows - AI's limited "working memory"
- Practical Implications - How this knowledge improves your results
The Mental Model
When you type something into ChatGPT or Claude, here's what's actually happening:
AI predicts the next word. That's it. It looks at everything you've written and predicts what word should come next, then repeats this process thousands of times to form a response.
This isn't "thinking" in any human sense. It's extremely sophisticated pattern matching. The AI was trained on massive amounts of text from the internet - books, articles, code, conversations - and learned patterns about how words follow other words.
Think of it like the world's most advanced autocomplete. Your phone suggests "you" after you type "thank" because it's seen that pattern millions of times. AI does the same thing, just at a scale and complexity that produces surprisingly coherent responses.
Why This Matters
Understanding this changes how you use AI:
- AI doesn't "know" things - It recognizes patterns that look like knowledge
- It can't fact-check itself - It has no way to verify if what it's saying is true
- Confidence doesn't mean accuracy - It predicts confident-sounding text because that's what good responses look like
Pattern Matching in Action
AI is excellent at tasks where patterns are abundant in its training data:
| Task | Why AI is Good at It |
|---|---|
| Writing emails | Seen millions of emails |
| Explaining concepts | Seen countless explanations |
| Writing code | Trained on vast code repositories |
| Summarizing text | Seen many summary examples |
AI struggles with:
- Novel problems it hasn't seen patterns for
- Very recent events (training data has a cutoff)
- Niche topics with little training data
- Tasks requiring real-world verification
Never the Same Twice
Run the exact same prompt twice and you'll get different responses. This isn't a bug - it's by design.
When predicting the next word, AI doesn't always pick the single most likely option. Instead, it samples from the top candidates with some randomness. This is controlled by a setting called temperature:
- Low temperature → More predictable, focused responses
- High temperature → More creative, varied responses
This means:
- Don't expect identical outputs - Even copy-pasting the same prompt produces variations
- Regenerate if unsatisfied - The next attempt might be better
- Slight differences are normal - Tone, word choice, and structure will vary
TIP
If you need consistent outputs (like for testing or automation), some AI APIs let you set temperature to 0 for near-deterministic responses. But for everyday use, embrace the variation - it often surfaces better ideas.
What Are Models?
You'll hear about "GPT-4," "Claude Sonnet," "Gemini Pro" - these are all models. A model is the trained neural network that does the actual predicting. Think of it as the "brain" that powers the AI tool.
Why Different Models Exist
Companies train different models with different trade-offs:
| Trade-off | Example |
|---|---|
| Speed vs. capability | Smaller models respond faster but handle less complexity |
| Cost vs. quality | Larger models cost more to run but produce better results |
| General vs. specialized | Some models excel at code, others at conversation |
This is why you see model names like "Claude Sonnet" (balanced) vs "Claude Haiku" (fast and cheap) vs "Claude Opus" (most capable).
What "New Model" Means
When a company announces a new model, they've typically:
- Trained on more or better data
- Improved the underlying architecture
- Expanded capabilities (longer context, better reasoning, new features)
Headlines will claim the new model is "X% better" - but better at what?
Benchmarks Don't Tell the Whole Story
AI companies publish benchmark scores showing how models perform on standardized tests. The problem:
- Benchmarks test specific tasks - Your work probably isn't a standardized test
- Models can be optimized for benchmarks - High scores don't guarantee real-world performance
- Your use case is unique - A model that's "best" at coding may not be best for your marketing copy
Evaluate for Yourself
When a new model launches, test it on your actual tasks - not hypothetical ones. Run the same prompts you use daily and compare results. Your experience matters more than any benchmark.
Hallucinations: Confident but Wrong
AI "hallucinations" are when the model generates information that sounds authoritative but is completely made up. This happens because:
- AI optimizes for plausible-sounding text - If asked for a citation, it generates something that looks like a citation
- It can't distinguish fact from fiction - Both real and made-up facts are just patterns to match
- It doesn't know what it doesn't know - There's no internal "I'm not sure" signal
Common Hallucination Examples
- Fake citations - Academic papers that don't exist, with convincing-sounding authors and journals
- Invented statistics - "Studies show that 73% of..." with no real source
- Confident technical errors - Code that looks right but has subtle bugs
- Fictional events - Detailed descriptions of things that never happened
Important
AI will state made-up facts with the same confidence as real ones. Always verify important information from primary sources.
Context Windows: AI's Working Memory
AI has a limited "context window" - the amount of text it can consider at once. Think of it as working memory.
Claude: ~100,000+ tokens (roughly 75,000 words) ChatGPT: Varies by model (8k to 128k tokens)
What This Means in Practice
- Long conversations degrade - As the context fills up, AI may "forget" earlier parts
- Upload large documents - AI can lose track of details in very long files
- Starting fresh helps - When responses get weird, a new conversation often works better
Signs You're Hitting Context Limits
- AI contradicts something it said earlier
- It forgets details you mentioned
- Responses become less coherent
- It starts repeating itself
Solution: Start a new conversation and provide key context upfront.
What This Means for You
Now that you understand how AI works, here's how to use it better:
Be Specific
Vague inputs produce vague outputs. The AI fills gaps with assumptions.
| Instead of... | Try... |
|---|---|
| "Write an email" | "Write a 3-paragraph email to a client explaining the project delay" |
| "Help me with code" | "Debug this Python function that should return the sum of a list" |
| "Give me ideas" | "Give me 5 marketing taglines for a sustainable coffee brand targeting millennials" |
Provide Context
AI can't read your mind. Tell it:
- Who you are (your role, expertise level)
- What you're trying to accomplish
- Any constraints or requirements
- Examples of what you want
Verify Important Facts
For anything consequential:
- Check statistics against primary sources
- Verify citations actually exist
- Test code before using it
- Cross-reference with authoritative sources
Start Fresh When Needed
If AI responses are getting strange:
- Start a new conversation
- Provide essential context upfront
- Don't try to "fix" a degraded conversation
Key Takeaways
- AI predicts words, it doesn't think - It's sophisticated pattern matching, not reasoning
- Models are the trained "brains" - Different models have different trade-offs; evaluate new ones yourself
- Confidence ≠ accuracy - AI sounds confident even when completely wrong
- Hallucinations are normal - Always verify important facts from primary sources
- Context is limited - Start fresh when conversations degrade
- Specificity wins - Clear inputs produce better outputs