Transfer Learning
Domain adaptation, fine-tuning, and representation transfer.
01Transfer Learning (1): Fundamentals and Core Concepts
A beginner-friendly guide to transfer learning fundamentals: why it works, formal definitions, taxonomy, negative …
02Transfer Learning (2): Pre-training and Fine-tuning
Why pre-training learns a powerful prior from unlabeled data and how fine-tuning adapts it to your task. Covers …
03Transfer Learning (3): Domain Adaptation
A practical guide to domain adaptation: covariate shift, label shift, DANN with gradient reversal, MMD alignment, CORAL, …
04Transfer Learning (4): Few-Shot Learning
Learn new concepts from a handful of examples. Covers the N-way K-shot protocol, metric learning (Siamese, Prototypical, …
05Transfer Learning (5): Knowledge Distillation
Compress large teacher models into small student models without losing much accuracy. Covers dark knowledge, temperature …
06Transfer Learning (6): Multi-Task Learning
Train one model on multiple tasks simultaneously. Covers hard vs. soft parameter sharing, gradient conflicts (PCGrad, …
07Transfer Learning (7): Zero-Shot Learning
A first-principles tour of zero-shot learning: attribute prototypes (DAP), compatibility functions, DeViSE, generative …
08Transfer Learning (8): Multimodal Transfer
Derive contrastive learning (InfoNCE), CLIP's vision-language pretraining, BLIP's Q-Former bridge to LLMs, cross-modal …
09Transfer Learning (9): Parameter-Efficient Fine-Tuning
Derive LoRA's low-rank adaptation, the Adapter bottleneck, Prefix-Tuning, Prompt-Tuning, BitFit and QLoRA. Includes a …
10Transfer Learning (10): Continual Learning
Derive catastrophic forgetting from gradient interference and the Fisher information matrix. Covers EWC, MAS, LwF, …
11Transfer Learning (11): Cross-Lingual Transfer
Derive cross-lingual transfer from bilingual word-embedding alignment to multilingual pretraining (mBERT, XLM-R). Covers …
12Transfer Learning (12): Industrial Applications and Best Practices
Series finale. A field guide to shipping transfer learning to production: when to use it, the end-to-end pipeline, …