19 Lab 19: The Future of NeuroAI: Three Frontiers
19.1 Learning Objectives
- Understand core concepts from Chapter 19
- Implement key algorithms and techniques
- Apply methods to practical problems
- Analyze results and draw insights
- Connect to broader NeuroAI themes
19.2 Prerequisites
- Reading: Chapter 19: Future Directions
- Libraries: NumPy, Matplotlib, PyTorch, torchvision
- Concepts: Neuromorphic computing, continual learning, catastrophic forgetting
19.3 Setup
Run this cell to import the necessary libraries and configure the environment.
import numpy as np
import matplotlib.pyplot as plt
import torch
import torch.nn as nn
import torch.optim as optim
import torch.nn.functional as F
from torch.utils.data import DataLoader, Subset
from torchvision import datasets, transforms
np.random.seed(42)
torch.manual_seed(42)
plt.rcParams['figure.figsize'] = (12, 8)
plt.rcParams['figure.dpi'] = 10019.4 Part 1: Neuromorphic Computing - Spiking Neurons
This section demonstrates a simple spiking neuron model, the fundamental building block of neuromorphic computing systems.
19.4.1 Exercise 1: Spiking Neuron Simulation
Run this cell to implement and simulate a basic leaky integrate-and-fire (LIF) spiking neuron.
class SpikingNeuron:
"""
Leaky Integrate-and-Fire (LIF) neuron model.
This is the fundamental computational unit in neuromorphic systems,
mimicking how biological neurons process information through spikes.
Parameters:
- threshold: Membrane potential threshold for spike generation
- tau_m: Membrane time constant (controls decay rate)
"""
def __init__(self, threshold=1.0, tau_m=10.0):
self.threshold = threshold
self.tau_m = tau_m
self.membrane_potential = 0.0
def update(self, input_current, dt=1.0):
"""
Update membrane potential and generate spike if threshold reached.
Returns 1 if spike generated, 0 otherwise.
"""
# Leaky integration: membrane potential decays toward 0
d_v = (-self.membrane_potential + input_current) / self.tau_m
self.membrane_potential += d_v * dt
# Fire if threshold reached
if self.membrane_potential >= self.threshold:
self.membrane_potential = 0.0 # Reset after spike
return 1 # Spike
return 0 # No spike
# Simulate the spiking neuron with varying input currents
print("Simulating Spiking Neuron Response to Input Currents")
print("=" * 60)
neuron = SpikingNeuron(threshold=1.0, tau_m=10.0)
time_steps = 100
input_currents = [0.5, 1.2, 2.0] # Different input levels
fig, axes = plt.subplots(len(input_currents), 1, figsize=(12, 8), sharex=True)
for idx, current in enumerate(input_currents):
neuron = SpikingNeuron(threshold=1.0, tau_m=10.0)
membrane_trace = []
spike_trace = []
for t in range(time_steps):
# Step input current
input_i = current if 20 <= t <= 80 else 0.0
spike = neuron.update(input_i)
membrane_trace.append(neuron.membrane_potential)
spike_trace.append(spike)
ax = axes[idx]
ax.plot(membrane_trace, 'b-', linewidth=1.5, label='Membrane Potential')
ax.axhline(y=1.0, color='r', linestyle='--', label='Threshold')
ax.fill_between(range(time_steps), 0, spike_trace, alpha=0.3, color='red', label='Spikes')
ax.set_ylabel(f'Input: {current}')
ax.legend(loc='upper right')
ax.grid(True, alpha=0.3)
axes[-1].set_xlabel('Time Step')
plt.suptitle('Spiking Neuron Response to Different Input Currents', fontsize=14, fontweight='bold')
plt.tight_layout()
plt.show()19.4.2 Exercise 2: Analyze Energy Efficiency
Discuss how spiking neurons achieve energy efficiency compared to traditional artificial neurons.
19.5 Part 2: Continual Learning - Catastrophic Forgetting & Experience Replay
This section demonstrates the catastrophic forgetting problem and how experience replay can mitigate it.
19.5.1 Exercise 3: Setting Up the Continual Learning Experiment
Run this cell to define the neural network and data loading utilities for our continual learning experiment.
# Simple neural network for MNIST classification
class SimpleNet(nn.Module):
"""
A simple feedforward network for MNIST digit classification.
Architecture: 784 -> 256 -> 256 -> 10
"""
def __init__(self, input_size=784, hidden_size=256, num_classes=10):
super(SimpleNet, self).__init__()
self.fc1 = nn.Linear(input_size, hidden_size)
self.relu = nn.ReLU()
self.fc2 = nn.Linear(hidden_size, hidden_size)
self.fc3 = nn.Linear(hidden_size, num_classes)
def forward(self, x):
x = x.view(x.size(0), -1)
x = self.relu(self.fc1(x))
x = self.relu(self.fc2(x))
x = self.fc3(x)
return x
# Experience Replay Buffer
class ReplayBuffer:
"""
Stores samples from previous tasks for replay during new task training.
This mimics hippocampal replay in biological systems, where recent
memories are reactivated during rest/sleep to consolidate learning.
"""
def __init__(self, capacity=1000):
self.capacity = capacity
self.buffer = []
self.position = 0
def push(self, data, target):
"""Add a sample to the buffer with circular overwriting."""
if len(self.buffer) < self.capacity:
self.buffer.append((data, target))
else:
self.buffer[self.position] = (data, target)
self.position = (self.position + 1) % self.capacity
def sample(self, batch_size):
"""Sample a random batch from the buffer."""
indices = np.random.choice(len(self.buffer), batch_size, replace=False)
batch = [self.buffer[i] for i in indices]
data = torch.stack([item[0] for item in batch])
targets = torch.tensor([item[1] for item in batch])
return data, targets
def __len__(self):
return len(self.buffer)
# Load and prepare MNIST
print("Loading MNIST dataset...")
transform = transforms.Compose([transforms.ToTensor(),
transforms.Normalize((0.1307,), (0.3081,))])
train_dataset = datasets.MNIST('./data', train=True, download=True, transform=transform)
test_dataset = datasets.MNIST('./data', train=False, transform=transform)
# Create task-specific datasets
def create_task_dataset(dataset, task_labels):
"""Filter dataset to include only specified labels."""
indices = [i for i, (_, label) in enumerate(dataset) if label in task_labels]
return Subset(dataset, indices)
task1_labels = [0, 1, 2, 3, 4]
task2_labels = [5, 6, 7, 8, 9]
task1_train = create_task_dataset(train_dataset, task1_labels)
task2_train = create_task_dataset(train_dataset, task2_labels)
task1_test = create_task_dataset(test_dataset, task1_labels)
task2_test = create_task_dataset(test_dataset, task2_labels)
print(f"Task 1 (digits 0-4): {len(task1_train)} training, {len(task1_test)} test samples")
print(f"Task 2 (digits 5-9): {len(task2_train)} training, {len(task2_test)} test samples")19.5.2 Exercise 4: Training Functions
Run this cell to define the training and evaluation functions.
# Training function with optional experience replay
def train_epoch(model, dataloader, optimizer, criterion, replay_buffer=None, replay_ratio=0.5):
"""
Train for one epoch with optional experience replay.
Parameters:
- replay_buffer: If provided, mixes replay samples with current task data
- replay_ratio: Fraction of batch to fill with replay samples
"""
model.train()
total_loss = 0
for data, target in dataloader:
optimizer.zero_grad()
output = model(data)
loss = criterion(output, target)
# Add replay samples if buffer exists
if replay_buffer is not None and len(replay_buffer) > 0:
replay_batch_size = int(len(data) * replay_ratio)
replay_data, replay_target = replay_buffer.sample(
min(replay_batch_size, len(replay_buffer))
)
replay_output = model(replay_data)
replay_loss = criterion(replay_output, replay_target)
loss = loss + replay_loss
loss.backward()
optimizer.step()
total_loss += loss.item()
# Store some samples in replay buffer
if replay_buffer is not None:
for i in range(min(5, len(data))): # Store 5 samples per batch
replay_buffer.push(data[i].cpu(), target[i].item())
return total_loss / len(dataloader)
# Evaluation function
def evaluate(model, dataloader):
"""Evaluate model accuracy on a dataset."""
model.eval()
correct = 0
total = 0
with torch.no_grad():
for data, target in dataloader:
output = model(data)
_, predicted = torch.max(output.data, 1)
total += target.size(0)
correct += (predicted == target).sum().item()
return 100 * correct / total
# Create data loaders
task1_loader = DataLoader(task1_train, batch_size=64, shuffle=True)
task2_loader = DataLoader(task2_train, batch_size=64, shuffle=True)
task1_test_loader = DataLoader(task1_test, batch_size=64, shuffle=False)
task2_test_loader = DataLoader(task2_test, batch_size=64, shuffle=False)
print("Data loaders created successfully.")19.5.3 Exercise 5: Experiment 1 - Without Experience Replay
Run this cell to observe catastrophic forgetting in action.
print("=" * 70)
print("EXPERIMENT 1: WITHOUT Experience Replay (Catastrophic Forgetting)")
print("=" * 70)
model_no_replay = SimpleNet()
optimizer = optim.Adam(model_no_replay.parameters(), lr=0.001)
criterion = nn.CrossEntropyLoss()
# Track performance over time
history_no_replay = {'task1': [], 'task2': []}
# Train on Task 1 (digits 0-4)
print("\nTraining on Task 1 (digits 0-4)...")
for epoch in range(5):
train_epoch(model_no_replay, task1_loader, optimizer, criterion)
task1_acc = evaluate(model_no_replay, task1_test_loader)
task2_acc = evaluate(model_no_replay, task2_test_loader)
history_no_replay['task1'].append(task1_acc)
history_no_replay['task2'].append(task2_acc)
print(f"Epoch {epoch+1}: Task 1 Acc = {task1_acc:.2f}%, Task 2 Acc = {task2_acc:.2f}%")
# Train on Task 2 (digits 5-9) - WITHOUT replay
print("\nTraining on Task 2 (digits 5-9) WITHOUT replay...")
for epoch in range(5):
train_epoch(model_no_replay, task2_loader, optimizer, criterion)
task1_acc = evaluate(model_no_replay, task1_test_loader)
task2_acc = evaluate(model_no_replay, task2_test_loader)
history_no_replay['task1'].append(task1_acc)
history_no_replay['task2'].append(task2_acc)
print(f"Epoch {epoch+1}: Task 1 Acc = {task1_acc:.2f}%, Task 2 Acc = {task2_acc:.2f}%")19.5.4 Exercise 6: Experiment 2 - With Experience Replay
Run this cell to see how experience replay mitigates forgetting.
print("\n" + "=" * 70)
print("EXPERIMENT 2: WITH Experience Replay (Forgetting Mitigated)")
print("=" * 70)
model_replay = SimpleNet()
optimizer = optim.Adam(model_replay.parameters(), lr=0.001)
replay_buffer = ReplayBuffer(capacity=2000)
history_replay = {'task1': [], 'task2': []}
# Train on Task 1 (digits 0-4)
print("\nTraining on Task 1 (digits 0-4)...")
for epoch in range(5):
train_epoch(model_replay, task1_loader, optimizer, criterion, replay_buffer)
task1_acc = evaluate(model_replay, task1_test_loader)
task2_acc = evaluate(model_replay, task2_test_loader)
history_replay['task1'].append(task1_acc)
history_replay['task2'].append(task2_acc)
print(f"Epoch {epoch+1}: Task 1 Acc = {task1_acc:.2f}%, Task 2 Acc = {task2_acc:.2f}%")
print(f"\nReplay buffer size after Task 1: {len(replay_buffer)} samples")
# Train on Task 2 (digits 5-9) - WITH replay
print("\nTraining on Task 2 (digits 5-9) WITH replay...")
for epoch in range(5):
train_epoch(model_replay, task2_loader, optimizer, criterion, replay_buffer)
task1_acc = evaluate(model_replay, task1_test_loader)
task2_acc = evaluate(model_replay, task2_test_loader)
history_replay['task1'].append(task1_acc)
history_replay['task2'].append(task2_acc)
print(f"Epoch {epoch+1}: Task 1 Acc = {task1_acc:.2f}%, Task 2 Acc = {task2_acc:.2f}%")19.5.5 Exercise 7: Visualize Results
Run this cell to visualize and compare the results.
# Visualize results
fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(14, 5))
# Plot without replay
epochs = range(1, 11)
ax1.plot(epochs, history_no_replay['task1'], 'b-', label='Task 1 (digits 0-4)', linewidth=2)
ax1.plot(epochs, history_no_replay['task2'], 'r-', label='Task 2 (digits 5-9)', linewidth=2)
ax1.axvline(x=5, color='gray', linestyle='--', label='Switch to Task 2')
ax1.set_xlabel('Epoch', fontsize=12)
ax1.set_ylabel('Test Accuracy (%)', fontsize=12)
ax1.set_title('WITHOUT Experience Replay\n(Catastrophic Forgetting)', fontsize=14, fontweight='bold')
ax1.legend(fontsize=10)
ax1.grid(True, alpha=0.3)
ax1.set_ylim([0, 100])
# Plot with replay
ax2.plot(epochs, history_replay['task1'], 'b-', label='Task 1 (digits 0-4)', linewidth=2)
ax2.plot(epochs, history_replay['task2'], 'r-', label='Task 2 (digits 5-9)', linewidth=2)
ax2.axvline(x=5, color='gray', linestyle='--', label='Switch to Task 2')
ax2.set_xlabel('Epoch', fontsize=12)
ax2.set_ylabel('Test Accuracy (%)', fontsize=12)
ax2.set_title('WITH Experience Replay\n(Forgetting Mitigated)', fontsize=14, fontweight='bold')
ax2.legend(fontsize=10)
ax2.grid(True, alpha=0.3)
ax2.set_ylim([0, 100])
plt.tight_layout()
plt.show()
# Calculate forgetting metrics
task1_peak_no_replay = max(history_no_replay['task1'][:5])
task1_final_no_replay = history_no_replay['task1'][-1]
forgetting_no_replay = task1_peak_no_replay - task1_final_no_replay
task1_peak_replay = max(history_replay['task1'][:5])
task1_final_replay = history_replay['task1'][-1]
forgetting_replay = task1_peak_replay - task1_final_replay
print("\n" + "=" * 60)
print("CATASTROPHIC FORGETTING ANALYSIS")
print("=" * 60)
print(f"\nWithout Replay:")
print(f" Peak Task 1 Accuracy: {task1_peak_no_replay:.2f}%")
print(f" Final Task 1 Accuracy: {task1_final_no_replay:.2f}%")
print(f" Forgetting: {forgetting_no_replay:.2f}%")
print(f"\nWith Experience Replay:")
print(f" Peak Task 1 Accuracy: {task1_peak_replay:.2f}%")
print(f" Final Task 1 Accuracy: {task1_final_replay:.2f}%")
print(f" Forgetting: {forgetting_replay:.2f}%")
print(f"\nImprovement: {forgetting_no_replay - forgetting_replay:.2f}% reduction in forgetting")
print("=" * 60)
19.6 Part 3: Continual Learning - Elastic Weight Consolidation (EWC)
This section demonstrates EWC, an alternative approach that protects important weights instead of storing past data.
19.6.1 Exercise 8: EWC Implementation
Run this cell to implement the Elastic Weight Consolidation algorithm.
class EWC:
"""
Elastic Weight Consolidation implementation.
EWC identifies important weights for a task using the Fisher Information
Matrix and penalizes changes to these weights during future learning.
This is inspired by synaptic consolidation in the brain, where important
synapses become more stable over time.
"""
def __init__(self, model, dataloader, device='cpu'):
self.model = model
self.device = device
# Store current parameters (important weights to protect)
self.params = {n: p.clone().detach() for n, p in model.named_parameters() if p.requires_grad}
# Compute Fisher Information (measure of weight importance)
self.fisher = self._compute_fisher(dataloader)
def _compute_fisher(self, dataloader):
"""
Compute Fisher Information Matrix.
The Fisher Information measures how sensitive the model output is
to changes in each weight. Higher values = more important weights.
"""
fisher = {n: torch.zeros_like(p) for n, p in self.model.named_parameters() if p.requires_grad}
self.model.eval()
for data, target in dataloader:
data, target = data.to(self.device), target.to(self.device)
self.model.zero_grad()
output = self.model(data)
# Use negative log-likelihood as loss
loss = F.cross_entropy(output, target)
loss.backward()
# Accumulate squared gradients (Fisher Information)
for n, p in self.model.named_parameters():
if p.requires_grad and p.grad is not None:
fisher[n] += p.grad.pow(2).clone().detach()
# Average over dataset
for n in fisher:
fisher[n] /= len(dataloader)
return fisher
def penalty(self, model):
"""
Compute EWC penalty term.
Penalizes deviation from stored parameters, weighted by importance.
"""
loss = 0
for n, p in model.named_parameters():
if p.requires_grad:
loss += (self.fisher[n] * (p - self.params[n]).pow(2)).sum()
return loss
# Training function with EWC
def train_epoch_ewc(model, dataloader, optimizer, criterion, ewc=None, ewc_lambda=1000):
"""
Train for one epoch with optional EWC regularization.
Parameters:
- ewc: EWC object containing Fisher Information from previous task
- ewc_lambda: Strength of the consolidation penalty
"""
model.train()
total_loss = 0
total_task_loss = 0
total_ewc_loss = 0
for data, target in dataloader:
optimizer.zero_grad()
output = model(data)
# Task loss
task_loss = criterion(output, target)
# EWC penalty
ewc_loss = 0
if ewc is not None:
ewc_loss = ewc.penalty(model)
# Total loss
loss = task_loss + ewc_lambda * ewc_loss
loss.backward()
optimizer.step()
total_loss += loss.item()
total_task_loss += task_loss.item()
total_ewc_loss += ewc_loss.item() if isinstance(ewc_loss, torch.Tensor) else ewc_loss
return (total_loss / len(dataloader),
total_task_loss / len(dataloader),
total_ewc_loss / len(dataloader))
print("EWC implementation defined successfully.")19.6.2 Exercise 9: EWC Experiment - Without Consolidation
Run this cell to train without EWC and observe forgetting.
print("=" * 70)
print("EXPERIMENT 1: Training WITHOUT EWC")
print("=" * 70)
model_no_ewc = SimpleNet()
optimizer = optim.Adam(model_no_ewc.parameters(), lr=0.001)
criterion = nn.CrossEntropyLoss()
history_no_ewc = {'task1': [], 'task2': []}
# Train on Task 1
print("\nTask 1 training (digits 0-4)...")
for epoch in range(5):
train_epoch_ewc(model_no_ewc, task1_loader, optimizer, criterion)
task1_acc = evaluate(model_no_ewc, task1_test_loader)
task2_acc = evaluate(model_no_ewc, task2_test_loader)
history_no_ewc['task1'].append(task1_acc)
history_no_ewc['task2'].append(task2_acc)
print(f"Epoch {epoch+1}: Task 1 = {task1_acc:.2f}%, Task 2 = {task2_acc:.2f}%")
# Compute Fisher Information after Task 1 (but don't use it)
ewc = EWC(model_no_ewc, task1_loader)
print(f"\nFisher Information computed (not used in this experiment)")
print("\nTask 2 training (digits 5-9) WITHOUT EWC...")
for epoch in range(5):
train_epoch_ewc(model_no_ewc, task2_loader, optimizer, criterion)
task1_acc = evaluate(model_no_ewc, task1_test_loader)
task2_acc = evaluate(model_no_ewc, task2_test_loader)
history_no_ewc['task1'].append(task1_acc)
history_no_ewc['task2'].append(task2_acc)
print(f"Epoch {epoch+1}: Task 1 = {task1_acc:.2f}%, Task 2 = {task2_acc:.2f}%")19.6.3 Exercise 10: EWC Experiment - With Consolidation
Run this cell to train with EWC and observe how it protects previous knowledge.
print("\n" + "=" * 70)
print("EXPERIMENT 2: Training WITH EWC")
print("=" * 70)
model_ewc = SimpleNet()
optimizer = optim.Adam(model_ewc.parameters(), lr=0.001)
history_ewc = {'task1': [], 'task2': []}
# Train on Task 1
print("\nTask 1 training (digits 0-4)...")
for epoch in range(5):
train_epoch_ewc(model_ewc, task1_loader, optimizer, criterion)
task1_acc = evaluate(model_ewc, task1_test_loader)
task2_acc = evaluate(model_ewc, task2_test_loader)
history_ewc['task1'].append(task1_acc)
history_ewc['task2'].append(task2_acc)
print(f"Epoch {epoch+1}: Task 1 = {task1_acc:.2f}%, Task 2 = {task2_acc:.2f}%")
# Compute Fisher Information and store weights
ewc_object = EWC(model_ewc, task1_loader)
print(f"\nFisher Information computed. Starting Task 2 with EWC penalty...")
ewc_lambda = 5000 # Consolidation strength
print(f"\nTask 2 training (digits 5-9) WITH EWC (lambda={ewc_lambda})...")
for epoch in range(5):
task_loss, ewc_loss, penalty = train_epoch_ewc(
model_ewc, task2_loader, optimizer, criterion,
ewc=ewc_object, ewc_lambda=ewc_lambda
)
task1_acc = evaluate(model_ewc, task1_test_loader)
task2_acc = evaluate(model_ewc, task2_test_loader)
history_ewc['task1'].append(task1_acc)
history_ewc['task2'].append(task2_acc)
print(f"Epoch {epoch+1}: Task 1 = {task1_acc:.2f}%, Task 2 = {task2_acc:.2f}%")19.6.4 Exercise 11: Visualize EWC Results
Run this cell to visualize the EWC results and weight importance analysis.
# Visualize weight importance
fig, axes = plt.subplots(2, 2, figsize=(14, 10))
# Plot 1: Learning curves without EWC
ax = axes[0, 0]
epochs = range(1, 11)
ax.plot(epochs, history_no_ewc['task1'], 'b-', linewidth=2, label='Task 1')
ax.plot(epochs, history_no_ewc['task2'], 'r-', linewidth=2, label='Task 2')
ax.axvline(x=5, color='gray', linestyle='--', alpha=0.7)
ax.set_xlabel('Epoch')
ax.set_ylabel('Test Accuracy (%)')
ax.set_title('WITHOUT EWC', fontweight='bold')
ax.legend()
ax.grid(True, alpha=0.3)
ax.set_ylim([0, 100])
# Plot 2: Learning curves with EWC
ax = axes[0, 1]
ax.plot(epochs, history_ewc['task1'], 'b-', linewidth=2, label='Task 1')
ax.plot(epochs, history_ewc['task2'], 'r-', linewidth=2, label='Task 2')
ax.axvline(x=5, color='gray', linestyle='--', alpha=0.7)
ax.set_xlabel('Epoch')
ax.set_ylabel('Test Accuracy (%)')
ax.set_title('WITH EWC', fontweight='bold')
ax.legend()
ax.grid(True, alpha=0.3)
ax.set_ylim([0, 100])
# Plot 3: Fisher Information distribution
ax = axes[1, 0]
fisher_values = []
for n, f in ewc_object.fisher.items():
fisher_values.extend(f.cpu().numpy().flatten())
fisher_values = np.array(fisher_values)
ax.hist(np.log10(fisher_values + 1e-10), bins=50, color='purple', alpha=0.7, edgecolor='black')
ax.set_xlabel('Log10(Fisher Information)')
ax.set_ylabel('Number of Weights')
ax.set_title('Weight Importance Distribution', fontweight='bold')
ax.grid(True, alpha=0.3)
# Plot 4: Protected vs Plastic weights
ax = axes[1, 1]
# Calculate weight changes
weight_changes = []
weight_importances = []
for n, p in model_ewc.named_parameters():
if p.requires_grad:
change = (p - ewc_object.params[n]).abs().cpu().detach().numpy().flatten()
importance = ewc_object.fisher[n].cpu().numpy().flatten()
weight_changes.extend(change)
weight_importances.extend(importance)
weight_changes = np.array(weight_changes)
weight_importances = np.array(weight_importances)
# Sample for visualization
sample_size = min(5000, len(weight_changes))
indices = np.random.choice(len(weight_changes), sample_size, replace=False)
ax.scatter(weight_importances[indices], weight_changes[indices],
alpha=0.3, s=1, c='blue')
ax.set_xlabel('Fisher Information (Weight Importance)')
ax.set_ylabel('Weight Change After Task 2')
ax.set_title('Protected vs Plastic Weights', fontweight='bold')
ax.set_xscale('log')
ax.set_yscale('log')
ax.grid(True, alpha=0.3)
plt.tight_layout()
plt.show()
# Summary statistics
print("\n" + "=" * 60)
print("EWC ANALYSIS")
print("=" * 60)
print(f"\nWithout EWC:")
print(f" Task 1 forgetting: {max(history_no_ewc['task1'][:5]) - history_no_ewc['task1'][-1]:.2f}%")
print(f"\nWith EWC (lambda={ewc_lambda}):")
print(f" Task 1 forgetting: {max(history_ewc['task1'][:5]) - history_ewc['task1'][-1]:.2f}%")
print(f"\nWeight Statistics:")
print(f" Highly protected (top 10%): {np.percentile(weight_importances, 90):.2e}")
print(f" Plastic (bottom 10%): {np.percentile(weight_importances, 10):.2e}")
print(f" Mean weight change: {np.mean(weight_changes):.2e}")
print("=" * 60)19.7 Additional Exercises
19.7.1 Exercise 12: Parameter Exploration
Experiment with different replay buffer sizes (500, 1000, 2000, 5000) and analyze the trade-off between memory usage and forgetting prevention.
19.7.2 Exercise 13: EWC Lambda Sensitivity
Explore different values of the EWC lambda parameter (100, 1000, 5000, 10000) and observe how it affects the balance between stability and plasticity.
19.7.3 Exercise 14: Multi-Task Learning
Extend the experiments to 3 or more sequential tasks and compare the performance of experience replay vs EWC.
19.7.4 Exercise 15: Combined Approach
Implement a hybrid method that combines experience replay with EWC. Does this provide better performance than either method alone?
19.8 Challenge Problems
19.8.1 Challenge 1: Progressive Neural Networks
Implement Progressive Neural Networks, which avoid forgetting by adding new network columns for each task while freezing previous columns.
19.8.2 Challenge 2: Generative Replay
Instead of storing examples, train a generative model (VAE or GAN) on Task 1 data and use it to generate pseudo-samples for replay during Task 2 training.
19.8.3 Challenge 3: Online EWC
Implement Online EWC, which maintains a running estimate of the Fisher Information rather than recomputing it from scratch for each task.
19.9 Discussion Questions
19.9.1 Question 1
How does the hippocampal-cortical memory system in the brain relate to experience replay in AI? What are the similarities and differences?
19.9.2 Question 2
Why might EWC be preferable to experience replay in certain scenarios? Consider privacy, memory constraints, and computational efficiency.
19.9.3 Question 3
How do neuromorphic computing and continual learning intersect? What advantages might spiking neural networks have for continual learning?
19.9.4 Question 4
What are the ethical implications of AI systems that can learn continuously without forgetting? Consider both benefits (personalization, adaptation) and risks (bias accumulation, unwanted persistence).
19.10 Summary
This lab explored three key frontiers in NeuroAI:
Part 1: Neuromorphic Computing - Implemented a leaky integrate-and-fire spiking neuron - Understood how event-driven computation enables energy efficiency - Connected to biological neuron models
Part 2: Experience Replay - Demonstrated catastrophic forgetting in sequential learning - Implemented experience replay to mitigate forgetting - Connected to hippocampal replay in memory consolidation
Part 3: Elastic Weight Consolidation - Implemented EWC using Fisher Information - Analyzed weight importance and selective protection - Connected to synaptic consolidation in the brain
Key Takeaways: - Neuromorphic systems offer orders of magnitude energy savings through event-driven computation - Continual learning requires mechanisms to protect previously learned knowledge - Both replay-based and regularization-based approaches have trade-offs - Biological systems provide inspiration for AI architectures and learning algorithms