26 Lab 26: Lifelong Learning: How the Brain Avoids Forgetting
26.1 Learning Objectives
- Understand core concepts from Chapter 26
- Implement key algorithms and techniques
- Apply methods to practical problems
- Analyze results and draw insights
- Connect to broader NeuroAI themes
26.2 Prerequisites
- Reading: Chapter 26: Lifelong Learning
- Libraries: NumPy, Matplotlib, relevant frameworks
- Concepts: Continual learning
26.3 Setup
import numpy as np
import matplotlib.pyplot as plt
np.random.seed(42)
plt.rcParams['figure.figsize'] = (12, 8)26.4 Part 1: Fundamentals
Core concepts and theory from Lifelong Learning.
26.4.1 1.1 Tag-and-Capture Inspired Consolidation
This algorithm demonstrates how synaptic tagging and capture from neuroscience can inspire continual learning in AI. The key idea is that only synapses (weights) that were active during learning get “tagged” and later consolidated based on available resources.
# Pseudocode for tag-and-capture inspired consolidation
class TagAndCaptureConsolidation:
def __init__(self, model, protein_budget=0.1):
self.tags = {} # Synaptic tags (gradient activity)
self.protein_budget = protein_budget # Limited resources
def learn_task(self, task_data):
# Standard learning creates "tags"
for x, y in task_data:
loss = self.model.loss(x, y)
gradients = loss.backward()
# Tag synapses based on gradient magnitude
for param, grad in zip(self.model.parameters(), gradients):
param.tag = grad.abs() # Synaptic tag
# Strong stimulation (high loss) triggers "protein synthesis"
if loss > threshold:
self.consolidate_tagged_synapses()
def consolidate_tagged_synapses(self):
# Limited "proteins" mean we can only consolidate top-k synapses
all_tags = [(p, p.tag) for p in self.model.parameters()]
all_tags.sort(key=lambda x: x[1], reverse=True)
# Consolidate top synapses (capture proteins)
num_to_consolidate = int(len(all_tags) * self.protein_budget)
for param, tag in all_tags[:num_to_consolidate]:
param.consolidation_strength += tag # Like Fisher InfoExercise: Implement a complete version with a simple neural network model and test it on a sequence of classification tasks.
26.5 Part 2: Hands-On Implementation
Practical exercises exploring Continual learning.
26.5.1 2.1 LoRA (Low-Rank Adaptation) for Continual Learning
LoRA is a parameter-efficient method that adds low-rank matrices to frozen weights, allowing adaptation without modifying the base model. This is particularly effective for foundation models and prevents catastrophic forgetting since the original weights remain unchanged.
Key concept: Original transformation is W * x, LoRA adds (B @ A) * x where W is frozen and A, B are much smaller learnable matrices.
# LoRA adds low-rank matrices to frozen weights
# Original: W * x
# LoRA: W * x + (B @ A) * x
# where W is frozen, A and B are learned (much smaller)
class LoRALayer:
def __init__(self, original_layer, rank=4):
self.W = original_layer.weight # Frozen
self.W.requires_grad = False
d_in, d_out = self.W.shape
self.A = nn.Parameter(torch.randn(d_in, rank) * 0.01)
self.B = nn.Parameter(torch.zeros(rank, d_out))
def forward(self, x):
# Frozen pathway + learned low-rank adaptation
return F.linear(x, self.W) + F.linear(x, self.B @ self.A)Exercise: Apply LoRA to a pre-trained model and fine-tune it on a new task. Compare the parameter count between full fine-tuning and LoRA.
26.5.2 2.2 Episodic Memory Transformer
Modern continual learning systems use transformers with explicit memory mechanisms. This architecture stores past experiences as key-value pairs and uses attention to retrieve relevant memories.
class EpisodicMemoryTransformer:
"""Transformer with explicit episodic memory."""
def __init__(self, d_model=512, n_memories=10000):
self.encoder = TransformerEncoder(d_model)
# Episodic memory: stored key-value pairs
self.memory_keys = nn.Parameter(torch.randn(n_memories, d_model))
self.memory_values = nn.Parameter(torch.randn(n_memories, d_model))
def forward(self, x):
# Encode input
h = self.encoder(x)
# Attention over episodic memory
attention_scores = torch.matmul(h, self.memory_keys.T)
attention_weights = F.softmax(attention_scores / np.sqrt(d_model), dim=-1)
# Retrieve and integrate memories
retrieved = torch.matmul(attention_weights, self.memory_values)
output = h + retrieved # Integrate current encoding with past memories
return output
def update_memory(self, new_key, new_value, memory_id):
"""Update specific memory slot."""
self.memory_keys[memory_id] = new_key
self.memory_values[memory_id] = new_valueExercise: Implement the TransformerEncoder class and test the episodic memory system on a sequence of NLP tasks.
26.6 Part 3: Analysis
Analyze and visualize results.
26.6.1 3.1 Comparing Continual Learning Strategies
Exercise: Implement and compare the following metrics across different continual learning approaches: - Average Accuracy: Mean performance across all tasks after training completes - Forgetting Measure: How much performance drops on earlier tasks - Forward Transfer: How much learning earlier tasks helps with later ones - Backward Transfer: How much learning later tasks improves earlier ones
26.7 Exercises
26.7.1 Exercise 1
Implement the basic algorithm.
26.7.2 Exercise 2
Explore parameter variations.
26.7.3 Exercise 3
Apply to real or simulated data.
26.7.4 Exercise 4
Compare different approaches.
26.7.5 Exercise 5
Discuss NeuroAI connections.
26.8 Challenge Problems
26.8.1 Challenge 1
Advanced implementation.
26.8.2 Challenge 2
Reproduce research findings.
26.8.3 Challenge 3
Novel application.
26.9 Discussion Questions
26.9.1 Question 1
How does this relate to neuroscience?
26.9.2 Question 2
What are the AI implications?
26.9.3 Question 3
Future research directions?
26.10 Summary
Explored Continual learning through theory and practice.
Key Takeaways: - Core concepts mastered - Practical implementation skills developed - NeuroAI connections understood