25 Lab 25: Embodied AI: Intelligence Needs a Body

Open In Colab

25.1 Learning Objectives

Understand core concepts from Chapter 25
Implement key algorithms and techniques
Apply methods to practical problems
Analyze results and draw insights
Connect to broader NeuroAI themes

25.2 Prerequisites

Reading: Chapter 25: Embodied AI and Robotics
Libraries: NumPy, Matplotlib, relevant frameworks
Concepts: Sensorimotor learning

25.3 Setup

import numpy as np
import matplotlib.pyplot as plt

np.random.seed(42)
plt.rcParams['figure.figsize'] = (12, 8)

25.4 Part 1: Fundamentals

Core concepts and theory from Embodied AI and Robotics.

25.4.1 1.1 Curiosity-Driven Exploration

One of the key principles in developmental robotics is curiosity-driven exploration. The idea is that agents (robots or humans) should explore actions that maximize learning progress rather than just reward. This mirrors how infants learn - they focus on activities where they’re making rapid progress.

The curiosity signal can be measured as the rate of change in prediction error:

\[ \text{Curiosity}(s) = |L_t(s) - L_{t-\Delta t}(s)| \]

where $L_t(s)$ is the prediction error in situation $s$ at time $t$. High curiosity corresponds to situations where prediction error is decreasing rapidly, meaning learning is happening.

Run this cell to see a simple implementation of curiosity-driven action selection:

class CuriousBabyRobot:
    """A robot that selects actions based on predicted learning progress."""

    def __init__(self, action_space):
        self.action_space = action_space
        self.learning_history = {a: [] for a in action_space}

    def estimate_learning_progress(self, action):
        """
        Estimate how much learning progress an action would produce.
        Based on the rate of change in prediction error.

        Returns:
            float: Estimated learning progress (higher = more interesting)
        """
        history = self.learning_history[action]
        if len(history) < 2:
            return float('inf')  # Unknown actions are maximally interesting

        # Learning progress = change in prediction error
        recent_error = history[-1]
        previous_error = history[-2]
        learning_progress = abs(recent_error - previous_error)

        return learning_progress

    def choose_next_action(self):
        """
        Select the action with highest predicted learning progress.
        This is the core of curiosity-driven exploration.
        """
        predicted_learning = {}

        for action in self.action_space:
            predicted_learning[action] = self.estimate_learning_progress(action)

        # Choose action with highest predicted learning
        best_action = max(predicted_learning, key=predicted_learning.get)
        return best_action

    def update_learning_history(self, action, prediction_error):
        """Record the prediction error after taking an action."""
        self.learning_history[action].append(prediction_error)


# Example usage
action_space = ['reach', 'grasp', 'push', 'look_around']
robot = CuriousBabyRobot(action_space)

# Simulate some learning history
robot.update_learning_history('reach', 0.8)
robot.update_learning_history('reach', 0.5)  # Learning progress!
robot.update_learning_history('grasp', 0.9)
robot.update_learning_history('push', 0.9)
robot.update_learning_history('push', 0.85)

# The robot will choose 'reach' because it shows most learning progress
next_action = robot.choose_next_action()
print(f"Robot chooses to: {next_action}")

Exercise: Modify the CuriousBabyRobot to include an exploration bonus for actions that haven’t been tried recently.

25.5 Part 2: Hands-On Implementation

Practical exercises exploring Sensorimotor learning.

25.6 Part 3: Analysis

Analyze and visualize results.

25.7 Exercises

25.7.1 Exercise 1

Implement the basic algorithm.

25.7.2 Exercise 2

Explore parameter variations.

25.7.3 Exercise 3

Apply to real or simulated data.

25.7.4 Exercise 4

Compare different approaches.

25.7.5 Exercise 5

Discuss NeuroAI connections.

25.8 Challenge Problems

25.8.1 Challenge 1

Advanced implementation.

25.8.2 Challenge 2

Reproduce research findings.

25.8.3 Challenge 3

Novel application.

Open In Colab

25.9 Discussion Questions

25.9.1 Question 1

How does this relate to neuroscience?

25.9.2 Question 2

What are the AI implications?

25.9.3 Question 3

Future research directions?

25.10 Summary

Explored Sensorimotor learning through theory and practice.

Key Takeaways: - Core concepts mastered - Practical implementation skills developed - NeuroAI connections understood

--- title: "Lab 25: Embodied AI: Intelligence Needs a Body" --- ::: {.content-visible when-format="html"} [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ricyoung/NeuroAI-Handbook_v3/blob/main/workbook/notebooks/lab25_embodied_ai_robotics.ipynb) ::: ## Learning Objectives 1. **Understand** core concepts from Chapter 25 2. **Implement** key algorithms and techniques 3. **Apply** methods to practical problems 4. **Analyze** results and draw insights 5. **Connect** to broader NeuroAI themes ## Prerequisites - **Reading**: Chapter 25: Embodied AI and Robotics - **Libraries**: NumPy, Matplotlib, relevant frameworks - **Concepts**: Sensorimotor learning ## Setup ```python import numpy as np import matplotlib.pyplot as plt np.random.seed(42) plt.rcParams['figure.figsize'] = (12, 8) ``` ## Part 1: Fundamentals Core concepts and theory from Embodied AI and Robotics. ### 1.1 Curiosity-Driven Exploration One of the key principles in developmental robotics is **curiosity-driven exploration**. The idea is that agents (robots or humans) should explore actions that maximize learning progress rather than just reward. This mirrors how infants learn - they focus on activities where they're making rapid progress. The curiosity signal can be measured as the rate of change in prediction error: $$ \text{Curiosity}(s) = |L_t(s) - L_{t-\Delta t}(s)| $$ where $L_t(s)$ is the prediction error in situation $s$ at time $t$. High curiosity corresponds to situations where prediction error is decreasing rapidly, meaning learning is happening. Run this cell to see a simple implementation of curiosity-driven action selection: ```python class CuriousBabyRobot: """A robot that selects actions based on predicted learning progress.""" def __init__(self, action_space): self.action_space = action_space self.learning_history = {a: [] for a in action_space} def estimate_learning_progress(self, action): """ Estimate how much learning progress an action would produce. Based on the rate of change in prediction error. Returns: float: Estimated learning progress (higher = more interesting) """ history = self.learning_history[action] if len(history) < 2: return float('inf') # Unknown actions are maximally interesting # Learning progress = change in prediction error recent_error = history[-1] previous_error = history[-2] learning_progress = abs(recent_error - previous_error) return learning_progress def choose_next_action(self): """ Select the action with highest predicted learning progress. This is the core of curiosity-driven exploration. """ predicted_learning = {} for action in self.action_space: predicted_learning[action] = self.estimate_learning_progress(action) # Choose action with highest predicted learning best_action = max(predicted_learning, key=predicted_learning.get) return best_action def update_learning_history(self, action, prediction_error): """Record the prediction error after taking an action.""" self.learning_history[action].append(prediction_error) # Example usage action_space = ['reach', 'grasp', 'push', 'look_around'] robot = CuriousBabyRobot(action_space) # Simulate some learning history robot.update_learning_history('reach', 0.8) robot.update_learning_history('reach', 0.5) # Learning progress! robot.update_learning_history('grasp', 0.9) robot.update_learning_history('push', 0.9) robot.update_learning_history('push', 0.85) # The robot will choose 'reach' because it shows most learning progress next_action = robot.choose_next_action() print(f"Robot chooses to: {next_action}") ``` **Exercise**: Modify the `CuriousBabyRobot` to include an exploration bonus for actions that haven't been tried recently. ## Part 2: Hands-On Implementation Practical exercises exploring Sensorimotor learning. ## Part 3: Analysis Analyze and visualize results. ## Exercises ### Exercise 1 Implement the basic algorithm. ### Exercise 2 Explore parameter variations. ### Exercise 3 Apply to real or simulated data. ### Exercise 4 Compare different approaches. ### Exercise 5 Discuss NeuroAI connections. --- ## Challenge Problems ### Challenge 1 Advanced implementation. ### Challenge 2 Reproduce research findings. ### Challenge 3 Novel application. --- ::: {.content-visible when-format="html"} [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ricyoung/NeuroAI-Handbook_v3/blob/main/workbook/notebooks/lab25_embodied_ai_robotics.ipynb) ::: ## Discussion Questions ### Question 1 How does this relate to neuroscience? ### Question 2 What are the AI implications? ### Question 3 Future research directions? --- ## Summary Explored Sensorimotor learning through theory and practice. **Key Takeaways**: - Core concepts mastered - Practical implementation skills developed - NeuroAI connections understood