Skip to content

mlinnen/mr-pumpkin

BranchesTags

Repository files navigation

Mr. Pumpkin - Fullscreen Pumpkin Face Renderer

A standalone Python program that renders an animated pumpkin face on fullscreen and responds to network commands to change facial expressions.

Features

  • Fullscreen rendering with 2D vector graphics
  • 7 facial expressions: neutral, happy, sad, angry, surprised, scared, sleeping
  • Smooth animations between expression transitions
  • Network socket server on port 5000 for receiving commands
  • Keyboard shortcuts for quick testing (1-7 keys for expressions, arrow keys for alignment, B/L/R for animations, U/J for eyebrows, [ ] for individual eyebrows, C/X for eye rolling)
  • Customizable appearance (colors, sizes, animation speeds)
  • Command recording & playback: Record command sequences and replay them with frame-accurate timing
    • Record commands into timelines with automatic timestamping
    • Play, pause, resume, stop, and seek through recorded sequences
    • List, delete, and rename recorded timelines
  • Gaze control: Direct eye positioning with individual eye control
  • Eye animations: Blink, wink, and roll eyes
  • Eyebrow control: Raise, lower, and control individual eyebrows

Installation

Option 1: Download Release Package (Recommended)

  1. Download the latest release from GitHub Releases

  2. Extract the ZIP file:

    unzip mr-pumpkin-v0.1.0.zip
cd mr-pumpkin-v0.1.0

  3. Run the install script:

    Linux/macOS/Raspberry Pi:

    chmod +x install.sh
./install.sh

    Windows (PowerShell):

    .\install.ps1

The install script will:

  • Install SDL2 system dependencies (Linux/Raspberry Pi only)
  • Install Python dependencies via pip
  • On Raspberry Pi, prefer apt-managed Python packages for dependencies that ship with Raspberry Pi OS and fall back to pip only for packages not available via apt
  • Provide usage instructions

Option 2: Install from Source

  1. Clone the repository and install dependencies:
git clone https://github.com/mlinnen/mr-pumpkin.git
cd mr-pumpkin
pip install -r requirements.txt

Note: On Raspberry Pi or Linux, you may need to install SDL2 system libraries first:

sudo apt-get update
sudo apt-get install -y libsdl2-dev libsdl2-image-dev libsdl2-mixer-dev libsdl2-ttf-dev

Raspberry Pi note: Raspberry Pi OS may reject global pip install calls for some system-managed packages. install.sh installs the Raspberry Pi-packaged versions of pygame, websockets, and mutagen with apt, then uses pip only for the remaining PyPI-only dependencies. update.sh stays non-root and cron-safe by default, so on Raspberry Pi it only refreshes the pip-managed subset unless you explicitly opt into apt refresh.

Development Setup

To set up a development environment with testing capabilities:

pip install -r requirements-dev.txt

This installs all production dependencies plus pytest for running the test suite.

Auto-Update

Mr. Pumpkin includes automated update scripts that check for new releases on GitHub and deploy them automatically.

Usage

Linux/macOS/Raspberry Pi:

./update.sh

Windows (PowerShell):

.\update.ps1

The update script will:

  1. Check the current installed version against the latest GitHub release
  2. Download the new release if available
  3. Stop the running pumpkin_face.py process (if running)
  4. Deploy the updated files
  5. Restart pumpkin_face.py with the same arguments

On Raspberry Pi, update.sh keeps unattended updates non-root by default: it refreshes only the pip-managed dependencies and leaves the apt-managed packages to install.sh or a manual sudo apt-get install ... step. If you explicitly want apt refresh during updates, set MR_PUMPKIN_ALLOW_PI_APT_UPDATE=1 and run with root/passwordless sudo available.

Scheduling Automatic Updates

Linux/macOS/Raspberry Pi (crontab):

Run daily at 3 AM:

crontab -e

Add this line:

0 3 * * * /absolute/path/to/mr-pumpkin/update.sh

Windows (Task Scheduler):

Create a scheduled task using PowerShell:

$action = New-ScheduledTaskAction -Execute "powershell.exe" -Argument "-ExecutionPolicy Bypass -File C:\path\to\mr-pumpkin\update.ps1"
$trigger = New-ScheduledTaskTrigger -Daily -At 3am
Register-ScheduledTask -Action $action -Trigger $trigger -TaskName "MrPumpkinAutoUpdate" -Description "Daily check for Mr. Pumpkin updates"

Or use Task Scheduler GUI:

  1. Open Task Scheduler → Create Basic Task
  2. Name: "Mr Pumpkin Auto Update"
  3. Trigger: Daily at 3:00 AM
  4. Action: Start a program
  5. Program: powershell.exe
  6. Arguments: -ExecutionPolicy Bypass -File C:\path\to\mr-pumpkin\update.ps1

Configuration

Custom Installation Directory:

Set the INSTALL_DIR environment variable to specify where Mr. Pumpkin is installed:

export INSTALL_DIR=/custom/path/to/mr-pumpkin
./update.sh
$env:INSTALL_DIR = "C:\custom\path\to\mr-pumpkin"
.\update.ps1

Log File

All update operations are logged to mr-pumpkin-update.log in the installation directory with timestamps.

For detailed setup and troubleshooting, see docs/auto-update.md.

Usage

Run the pumpkin face application:

Fullscreen (default on monitor 0):

python pumpkin_face.py

Fullscreen on specific monitor:

python pumpkin_face.py 0      # First monitor
python pumpkin_face.py 1      # Second monitor
python pumpkin_face.py 2      # Third monitor

Windowed mode:

python pumpkin_face.py --window           # Windowed on monitor 0
python pumpkin_face.py 1 --window         # Windowed on monitor 1

Custom host and port:

python pumpkin_face.py --host 0.0.0.0     # Listen on all network interfaces
python pumpkin_face.py --port 8080        # Listen on port 8080
python pumpkin_face.py --host 0.0.0.0 --port 8080  # Custom host and port

Usage:

python pumpkin_face.py [OPTIONS] [monitor_number]

Options:
  --window              Run in windowed mode (default: fullscreen)
  --fullscreen          Run in fullscreen mode
  --host HOST           IP address or hostname to bind to (default: localhost)
  --port PORT           Port number to listen on (default: 5000)
  -h, --help            Show this help message

The program will list available monitors and run on your selected output. Press ESC to exit.

Send commands via network socket:

Using the example client (interactive menu):

python client_example.py

This provides an interactive menu with commands for:

  • Changing expressions (neutral, happy, sad, angry, surprised, scared, sleeping)
  • Animation controls (blink, roll eyes, control eyebrows)
  • Gaze control (point eyes at angles)
  • Recording and playback:
    • record start - Begin recording
    • record stop <filename> - Save a recording
    • record cancel - Discard recording
    • record status - Check recording state
    • list - View all recordings
    • download_timeline <filename> - Download a recording file as JSON
    • play <filename> - Play a recording
    • upload_timeline <filename> <json_file> - Upload a recording file from disk
    • pause / resume / stop - Control playback
    • seek <position_ms> - Jump to position
    • timeline_status - Check playback state

Via Python:

import socket

client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
client.connect(('localhost', 5000))
client.send(b'happy')
response = client.recv(1024)  # Some commands return responses
client.close()

Via command line (netcat/nc):

echo "happy" | nc localhost 5000
echo "record start" | nc localhost 5000
echo "record stop my_recording" | nc localhost 5000
echo "play my_recording" | nc localhost 5000
echo "timeline_status" | nc localhost 5000

Via PowerShell:

$socket = New-Object System.Net.Sockets.TcpClient
$socket.Connect('localhost', 5000)
$stream = $socket.GetStream()
[byte[]]$bytes = [System.Text.Encoding]::ASCII.GetBytes("happy")
$stream.Write($bytes, 0, $bytes.Length)
$stream.Close()

WebSocket Interface (Browser Clients)

The pumpkin face server also provides a WebSocket interface on port 5001 for browser-based control panels and real-time communication.

Via JavaScript:

const ws = new WebSocket('ws://localhost:5001');

ws.onopen = () => {
  ws.send('happy');
  ws.send('blink');
  ws.send('timeline_status');
};

ws.onmessage = (event) => {
  console.log('Response:', event.data);
};

ws.onerror = (error) => {
  console.error('WebSocket error:', error);
};

WebSocket protocol:

  • Each command is sent as a plain text message (same format as TCP)
  • Responses are sent back immediately as text messages
  • Empty responses (for animations) are not sent
  • Connection can be reused for multiple commands
  • Server handles concurrent WebSocket clients simultaneously

Simple HTML test client:

<!DOCTYPE html>
<html>
<head>
  <title>Mr. Pumpkin WebSocket Client</title>
</head>
<body>
  <h1>Mr. Pumpkin Control</h1>
  <input type="text" id="command" placeholder="Enter command">
  <button onclick="send()">Send</button>
  <div id="response"></div>
  
  <script>
    const ws = new WebSocket('ws://localhost:5001');
    const responseDiv = document.getElementById('response');
    
    function send() {
      const cmd = document.getElementById('command').value;
      if (cmd) {
        ws.send(cmd);
      }
    }
    
    ws.onmessage = (event) => {
      responseDiv.innerHTML = '<p>' + event.data + '</p>';
    };
  </script>
</body>
</html>

Port allocation:

  • TCP (port 5000): Text-based commands, command-line clients, legacy support
  • WebSocket (port 5001): Browser clients, real-time dashboards, concurrent connections

Both protocols support the same commands and are always available when the server runs.

Full-featured test client: For comprehensive testing and debugging, open websocket-test-client.html in your browser. This client includes:

  • Connection status monitoring with automatic fallback
  • Quick test buttons for common commands
  • Timeline upload testing (multi-line JSON)
  • Full event logging with timestamps
  • Error handling and diagnostics

Supported Commands

Expressions

  • neutral - Default expression
  • happy - Smiling face
  • sad - Frowning face
  • angry - Angry eyes and mouth
  • surprised - Wide eyes, open mouth
  • scared - Frightened expression
  • sleeping - Eyes closed (sleeping)

Recording & Playback

  • record_start or record start - Begin recording commands
  • record_stop <filename> or record stop <filename> - Stop recording and save with filename
  • record_cancel or record cancel - Discard current recording without saving
  • recording_status or record status - Show recording state (is_recording, command_count, duration_ms)
  • list_recordings or list - Show all available recordings
  • delete_recording <filename> - Remove a saved recording
  • rename_recording <old_name> <new_name> - Rename a saved recording
  • upload_timeline <filename> <json_file> - Upload a recording file from disk (filename: name to store as, json_file: local path to JSON file)
  • download_timeline <filename> - Download a recording file as JSON string (filename: name of recording to download)
  • play <filename> - Start playing a recorded timeline
  • pause - Pause current playback
  • resume - Resume from paused state
  • stop - Stop playback and return to start
  • seek <position_ms> - Jump to specific position in recording
  • timeline_status - Show current playback state (state, filename, position, duration, is_playing)

Animation Controls

  • blink - Blink both eyes
  • wink_left - Wink left eye only
  • wink_right - Wink right eye only
  • roll_clockwise - Roll eyes clockwise
  • roll_counterclockwise - Roll eyes counter-clockwise
  • raise_eyebrows - Raise both eyebrows
  • lower_eyebrows - Lower both eyebrows
  • raise_left_eyebrow - Raise left eyebrow only
  • lower_left_eyebrow - Lower left eyebrow only
  • raise_right_eyebrow - Raise right eyebrow only
  • lower_right_eyebrow - Lower right eyebrow only

Gaze Control

  • gaze <x> <y> - Point both eyes at angle (x, y in degrees, -90 to +90)
  • gaze <x1> <y1> <x2> <y2> - Point left and right eyes independently

Nose Animation

  • wiggle_nose - Animate nose wiggle
  • reset_nose - Stop nose animation and return to neutral

Recording Storage

Recording File Format

Recordings are stored as JSON files in the user's home directory. Each recording contains:

File location: ~/.mr-pumpkin/recordings/{filename}.json

File structure:

{
  "version": "1.0",
  "duration_ms": 5230,
  "commands": [
    {
      "time_ms": 0,
      "command": "happy"
    },
    {
      "time_ms": 500,
      "command": "blink"
    },
    {
      "time_ms": 2000,
      "command": "roll_clockwise"
    }
  ]
}

Field descriptions:

  • version - Format version for future compatibility (currently "1.0")
  • duration_ms - Total duration of the recording in milliseconds
  • commands - Array of commands, each with:
    • time_ms - Timestamp in milliseconds from the start of the recording
    • command - The pumpkin command to execute at this time (e.g., "happy", "blink", "gaze")
    • args (optional) - Additional command arguments for commands that need them (e.g., gaze angles)

Recording Directory

All recordings are stored in ~/.mr-pumpkin/recordings/. The directory is created automatically when you save your first recording.

  • On Windows: C:\Users\{username}\.mr-pumpkin\recordings\
  • On macOS/Linux: /Users/{username}/.mr-pumpkin/recordings/ or ~/.mr-pumpkin/recordings/

You can browse recordings in this directory to:

  • Back up important sequences
  • Edit timeline files directly (JSON format is human-readable)
  • Share recordings with others
  • Delete old recordings manually if desired

Keyboard Controls

While the application is running:

  • 1 → Neutral
  • 2 → Happy
  • 3 → Sad
  • 4 → Angry
  • 5 → Surprised
  • 6 → Scared
  • 7 → Sleeping
  • ESC → Exit fullscreen

Projection Alignment (Jog Controls):

  • Arrow Keys → Nudge projection in any direction (5px steps)
  • 0 → Reset projection offset to center

Animation Controls:

  • B → Blink
  • L → Wink left
  • R → Wink right
  • C → Roll eyes clockwise
  • X → Roll eyes counter-clockwise
  • U → Raise both eyebrows
  • J → Lower both eyebrows
  • [ → Raise left eyebrow (Shift+[ to lower)
  • ] → Raise right eyebrow (Shift+] to lower)

Customization

Edit pumpkin_face.py to customize:

  • Colors: Modify PUMPKIN_COLOR, EYE_COLOR, MOUTH_COLOR, etc.
  • Animation speed: Change self.transition_speed
  • Face size: Adjust pumpkin_radius
  • Socket port: Change the port in _run_socket_server()
  • Recordings storage path: Modify the recordings_dir in the FileManager class

Headless Mode

The application can run in headless mode (without a display) for automation and CI/CD environments. In this mode:

  • The TCP socket server runs fully functional
  • All commands work normally (expressions, animations, recording/playback)
  • The pygame display is optional and gracefully skipped if unavailable
  • Useful for running the pumpkin server on a headless Linux machine that communicates with a remote display

Architecture

  • pumpkin_face.py: Main application with rendering and network server
  • client_example.py: Example client for sending commands
  • tests/: Test suite directory with all test modules
  • requirements.txt: Production dependencies (pygame only)
  • requirements-dev.txt: Development dependencies (includes pytest for testing)

Skill CLI Tools

The skill/ package provides three command-line tools:

  • mr-pumpkin-record — generate and upload a timeline from a natural-language prompt
  • mr-pumpkin-lipsync — generate a lip-synced timeline from an audio file
  • mr-pumpkin-list-models — inspect live Gemini/OpenAI model IDs available to your configured account

The generation tools require a GEMINI_API_KEY environment variable by default (and optionally OPENAI_API_KEY when using the OpenAI provider). The model-listing tool requires the provider-specific API key for the provider you query.

mr-pumpkin-record

Translates a natural language prompt into a timeline animation and uploads it to the Mr. Pumpkin server.

usage: mr-pumpkin-record prompt -f FILENAME [--host HOST] [--tcp-port N]
                                [--ws-port N] [--protocol {tcp,ws}]
                                [--dry-run] [--provider PROVIDER]

Arguments

Argument Default Description
prompt (required) Natural language description of the desired animation
-f / --filename (required) Name to store the timeline as on the server (no .json extension)
--host localhost Mr. Pumpkin server hostname
--tcp-port 5000 TCP port
--ws-port 5001 WebSocket port
--protocol tcp Upload protocol: tcp or ws (WebSocket)
--provider gemini LLM provider (currently only gemini is supported)
--dry-run — Generate and print the timeline without uploading

Environment variables

  • GEMINI_API_KEY — required when using the default Gemini provider

Exit codes: 0 success · 1 generation/upload error · 2 argument error

Examples

# Generate and upload a surprised/blinking animation
python -m skill.cli "make the pumpkin look surprised then blink" -f my_show

# Generate and upload via WebSocket to a remote host
python -m skill.cli "wave hello" -f wave --host 192.168.1.10 --protocol ws

# Preview the generated timeline without uploading
python -m skill.cli "pumpkin laughs heartily" -f laugh --dry-run

mr-pumpkin-lipsync

Generates a lip-synced Mr. Pumpkin animation from an audio file using a two-pass pipeline: audio analysis (Gemini multimodal) followed by LLM choreography generation.

usage: mr-pumpkin-lipsync audio_file [-f FILENAME] [--prompt PROMPT]
                                     [--host HOST] [--tcp-port N] [--ws-port N]
                                     [--protocol {tcp,ws}]
                                     [--audio-provider PROVIDER]
                                     [--provider PROVIDER] [--model MODEL]
                                     [--audio-model AUDIO_MODEL]
                                     [--api-key KEY] [--dry-run]

Arguments

Argument Default Description
audio_file (required) Path to audio file (.mp3, .wav, .ogg)
-f / --filename audio file stem Name to store the recording as on the server
--prompt — Artistic guidance for the animation (e.g., "pumpkin sings this joyfully")
--host localhost Mr. Pumpkin server hostname
--tcp-port 5000 TCP port
--ws-port 5001 WebSocket port
--protocol tcp Upload protocol: tcp or ws (WebSocket)
--audio-provider gemini Provider for audio analysis
--provider gemini LLM provider for timeline generation (gemini or openai)
--model — Override the default LLM model (e.g., gpt-4o, gemini-1.5-pro)
--audio-model — Override the default audio analysis model
--api-key — API key override (supersedes GEMINI_API_KEY / OPENAI_API_KEY env vars)
--dry-run — Analyze and generate, print JSON, do NOT upload

Environment variables

  • GEMINI_API_KEY — required for Gemini provider (default)
  • OPENAI_API_KEY — required when using --provider openai

Exit codes: 0 success · 1 generation/analysis/upload error · 2 argument error

Examples

# Basic lip-sync from an MP3 file
python -m skill.lipsync_cli song.mp3 -f my_song

# Add artistic direction and test without uploading
python -m skill.lipsync_cli speech.wav -f story \
  --prompt "pumpkin tells this story with wide-eyed wonder" --dry-run

# Use OpenAI for timeline generation with a specific model
python -m skill.lipsync_cli song.mp3 -f dance \
  --provider openai --model gpt-4o

mr-pumpkin-list-models

Lists the live model IDs exposed by Gemini or OpenAI so you can choose valid values for --model and related overrides before generating animations.

usage: mr-pumpkin-list-models [--provider PROVIDER] [--filter TEXT]
                              [--all] [--api-key KEY]

Arguments

Argument Default Description
--provider / -p gemini Provider to query: gemini or openai
--filter / -f — Case-insensitive substring filter applied to model IDs
--all / -a — Query all supported providers and print grouped results
--api-key — Override the provider API key instead of reading environment variables

Environment variables

  • GEMINI_API_KEY or GOOGLE_API_KEY — required for Gemini queries
  • OPENAI_API_KEY — required for OpenAI queries

Exit codes: 0 success · 1 provider/API/import error · 2 argument error

Examples

# List all Gemini models visible to the configured account
python -m skill.list_models

# Show only OpenAI models containing "gpt-4o"
python -m skill.list_models --provider openai --filter gpt-4o

# Query both providers in one pass
python -m skill.list_models --all

Testing

Run the test suite to validate projection mapping and other features:

# Install development dependencies (if not already installed)
pip install -r requirements-dev.txt

# Run all tests (430+ tests)
pytest

# Run specific test file
pytest tests/test_projection_mapping.py -v

# Run specific test class
pytest tests/test_projection_mapping.py::TestProjectionMappingColors -v

# Run with detailed output
pytest -vv

The test suite validates:

  • Black background (RGB 0,0,0) for projection mapping
  • White features (RGB 255,255,255) for eyes, nose, mouth
  • Minimum 15:1 contrast ratio for reliable projection
  • All expression states render correctly
  • Resolution independence
  • Transition behavior

Integration Tests

The project includes comprehensive integration tests for dual-protocol operation (TCP + WebSocket):

# Run integration tests only
pytest tests/test_integration_dual_protocol.py -v

# Run TCP integration tests
pytest tests/test_tcp_integration.py -v

Dual-Protocol Integration Tests (test_integration_dual_protocol.py) validate:

  1. Identical Responses (5 tests)

    • Both TCP and WebSocket return identical responses for the same commands
    • Status queries (timeline_status, recording_status) return consistent JSON
    • List operations return matching results

  2. Protocol Switching (3 tests)

    • Commands work when alternating between TCP and WebSocket
    • No state corruption when switching protocols mid-session
    • 10+ command sequences alternating protocols

  3. Concurrent Commands (2 tests)

    • Multiple concurrent commands from both protocols execute correctly
    • Concurrent status queries from both protocols

  4. Error Handling Consistency (3 tests)

    • Invalid commands return errors on both protocols
    • Malformed commands handled gracefully
    • Nonexistent file errors consistent across protocols

  5. Timeline Upload/Download (3 tests)

    • Upload timeline via TCP, verify via WebSocket
    • Upload timeline via WebSocket, verify via TCP
    • Download same timeline via both protocols yields identical results

  6. State Synchronization (3 tests)

    • Recording state visible from both protocols
    • Playback state visible from both protocols
    • Expression changes propagate across protocols

  7. Connection Resilience (2 tests)

    • TCP disconnect doesn't affect WebSocket
    • WebSocket disconnect doesn't affect TCP

  8. Large Payloads (2 tests)

    • Large timelines (>100KB JSON, 200+ commands) work on both protocols

  9. Stress Testing (1 test)

    • 50 rapid commands alternating TCP/WebSocket

  10. Playback Integration (2 tests)

    • Start playback on one protocol, control via the other
    • Cross-protocol pause/resume/stop

  11. Clean Shutdown (1 test)

    • Graceful disconnect on both protocols without orphaned resources

Total: 27 integration tests covering all critical dual-protocol scenarios.

Development

Mr. Pumpkin was built using Squad — an agentic coding solution created by Brady Gaster that orchestrates a team of specialized AI agents to collaboratively design, build, test, and document software.

The Squad Team

The following AI agents collaborated to develop Mr. Pumpkin, drawn from the Arcane universe:

Agent Role Responsibility
🏗️ Jinx Lead Architecture, scope decisions, code review
⚛️ Ekko Graphics Dev 2D rendering, animations, facial expressions
🔧 Vi Backend Dev Network socket server, command handling, timeline playback
🧪 Mylo Tester Test suite, quality assurance, edge cases
📋 Scribe Memory & Logging Decisions, session logs, cross-agent context
🔄 Ralph Work Monitor Work queue, backlog tracking, issue triage

Want to build your own project with an AI team like this? Check out Brady Gaster's Squad.

License

MIT

About

mlinnen.github.io/mr-pumpkin/

Resources

Readme

License

MIT license

Contributing

Contributing
Activity

Stars

1 star

Watchers

0 watching

Forks

0 forks
Report repository

Releases 23

v0.5.17 Latest
Mar 13, 2026
+ 22 releases

Packages

 
 
 

Contributors

Languages