Introduction to ML strategy
- Why ML Strategy?
- Orthogonalization
Setting up your goal
- Single number evaluation metric
- Satisficing and optimizing metrics
- Train/dev/test distributions
- Size of dev and test sets
- When to change dev/test sets and metrics
Comparing to human-level performance
- Why human-level performance?
- Avoidable bias
- Understanding human-level performance
- Surpassing human-level performance
- Improving your model performance
Error Analysis
- Carrying out error analysis
- Cleaning up Incorrectly labeled data
- Build your first system quickly, then iterate
Mismatched training and dev/test data
- Training and testing on different distributions
- Bias and Variance with mismatched data distributions
- Addressing data mismatch
Learning from multiple tasks
- Transfer learning
- Multi-task learning
End-to-end deep learning
- What is end-to-end deep learning
- Whether to use end-to-end learning