📚 Learning Objectives
- Understand how computers process human language (NLP)
- Learn tokenization, embeddings, and semantic meaning
- Understand Large Language Models (LLMs) like ChatGPT, GPT-4
- Learn training process: pretraining, fine-tuning, RLHF
- Recognize capabilities and limitations of LLMs
- Understand prompt engineering and AI safety concerns
🎥 Learning Videos
1. Natural Language Processing Basics
2. Word Embeddings & Semantic Meaning
3. How ChatGPT and Large Language Models Work
4. Training Large Language Models: Pretraining to Deployment
5. Prompt Engineering & Effective AI Use
📊 NLP Techniques & LLM Evolution
| Technique | What It Does | Breakthrough | Modern Example |
|---|---|---|---|
| Tokenization | Break text into words/subwords | BPE (Byte-Pair Encoding) handles unknowns | ChatGPT tokenizes prompts |
| Word Embeddings | Convert words to numbers capturing meaning | Word2Vec: "king"-"man"="queen" | Semantic search |
| Sequence Models | Process text in order (RNN → Transformer) | Attention = parallelization breakthrough | BERT, GPT |
| Pretraining | Train on massive unlabeled text data | Emerges knowledge without labels | GPT-4 trained on web-scale data |
| Fine-tuning | Adapt pretrained model to specific task | Reduces data/compute needed | ChatGPT fine-tuned for conversation |
| RLHF | Use human feedback to align behavior | Makes AI "safe" and "helpful" | ChatGPT uses RLHF |
📋 6 NLP & LLM Concepts
Concept 1: Tokenization & Vocabulary
Language is continuous strings. Tokenization breaks into manageable units (words, subwords, characters). Vocabulary size: typical LLM has 50K tokens. Unknown words handled by subword tokenization (splitting to smaller pieces). Token efficiency matters: fewer tokens = cheaper to run.
Concept 2: Word Embeddings & Semantic Space
Words represented as vectors in high-dimensional space (300D typical). Related words cluster together. Arithmetic works: vector("king") - vector("man") + vector("woman") ≈ vector("queen"). This captures semantic relationships learned from data.
Concept 3: Language Modeling & Next-Token Prediction
LLM training task: "Given previous tokens, predict next token." ChatGPT trained on billions of examples of this. After training, it can generate coherent text by repeatedly predicting next token. Stochastic process: same prompt can generate different outputs (temperature parameter controls randomness).
Concept 4: Pretraining vs Fine-tuning
PRETRAINING: Train on massive text corpus (billions of tokens). Learns general language, facts, reasoning. Requires enormous compute (GPT-4: trillions of tokens, millions of GPU hours). FINE-TUNING: Take pretrained model, train on specific data (conversation, coding, medical). Cheaper, faster, task-specific. Most practical approach today.
Concept 5: RLHF - Reinforcement Learning from Human Feedback
Raw LLM outputs can be inaccurate, toxic, biased. RLHF: humans rate outputs, train reward model, use RL to optimize for high-scored outputs. Makes AI "helpful, harmless, honest." Trade-off: sometimes less creative/informative to be "safe." Example: ChatGPT trained this way.
Concept 6: Prompt Engineering & Context Window
Prompt quality dramatically affects outputs. Techniques: few-shot examples, clear instructions, role-playing ("you are expert..."). Context window: max tokens the model can attend to (ChatGPT-4: 8K or 32K, Claudee: 100K+). Longer context = better for complex tasks but slower/more expensive. Hallucinations: LLM confidently generates false information (serious risk).
🎯 Module 4 Quiz
1. What does RLHF do in large language models?
2. What is hallucination in LLMs?
3. What is the advantage of fine-tuning over training from scratch?
🚀 Real-World Project: LLM Exploration & Ethics
Part 1: Prompt Engineering Experiment
Test ChatGPT/Claude with 5 different prompts on same question. How much does prompt quality matter? Document: best prompt, why it works, worst prompt and why.
Part 2: Hallucination Detection
Ask LLM about recent events, specific facts requiring up-to-date knowledge. Verify: how many hallucinations? How confident does it sound even when wrong?
Part 3: Bias & Fairness Analysis
Test LLM responses on sensitive topics. Does it show bias? How? Propose solutions.
Part 4: Use Case Design
Propose LLM application for your field. What works well? What are risks?
Part 5: Context Window Limits
Experiment: how much context before quality degrades? Where's the breaking point?
Part 6: Cost-Benefit Analysis
Using LLMs vs alternatives: compute cost, accuracy, human time saved. When justified?
Part 7: Responsible AI Implementation
If deploying LLM: what safeguards? Human oversight? Audit trails? Limits?
Part 8: Future Predictions
Where will LLMs be in 2-5 years? Capabilities? Risks? How prepare?