Physical AI Hack 2026 - Team Preparation

Event: January 31 - February 1, 2026
Location: Founders Inc, San Francisco
Team: 4 people
Robot: SO-101 arm
Approach: ACT (Action Chunking with Transformers) + Imitation Learning

---

📁 Documentation Structure

🚀 Quick Start (Start Here!)

1. START-HERE.md ⭐ - Complete day-by-day guide (READ THIS FIRST!)
2. COLAB-PRO-SETUP.md - Google Colab Pro+ setup (industry standard)
3. colab-notebook-template.ipynb - Production-ready training notebook
4. QUICKSTART.md - Local setup in 15 minutes
5. setup_env.sh - Automated environment setup

📖 Main Guides

• track1-sim.md - Complete simulation setup & workflow (updated for cloud GPU)
• cloud-gpu-setup.md - Comprehensive cloud GPU guide (Colab, Lambda, RunPod)
• ideas-track1.md - Track 1 (Shape Insertion) project ideas & implementation plans
• ideas-track2.md - Track 2 (Charger Plugging) project ideas & implementation plans
• background.md - Team technical background & approach

📋 Event Information

• hackathon.md - Complete hackathon details (consolidated)
• hackathon-invite-page.md - Luma event page details

---

🎯 Our Strategy

Track Selection

PRIMARY: Track 1 - Shape Insertion ⭐

• Perfect for ACT + SO-101
• Fast iteration cycles
• Clear success metrics
• Achievable in 48 hours

BACKUP: Track 2 - Charger Plugging

• Higher difficulty
• Real-world applicability
• Good if we want more challenge

Technical Approach

Framework: LeRobot (Hugging Face)
Policy: ACT (Action Chunking with Transformers)
Hardware: SO-101 robotic arm
Training: Google Colab Pro+ (A100 GPU, industry standard)

Why ACT?

• Fast training (2-4 hours)
• Data efficient (50 demos work)
• Proven for manipulation
• Lightweight (~80M parameters)

---

🚀 Getting Started (Your Mac with No GPU)

Phase 1: Local Setup (Mac)

# 1. Run automated setup
cd /Users/bencxr/dev/physicalaihack
./setup_env.sh

# 2. Activate environment
source lerobot-env/bin/activate

# 3. Create simulation environment
# Follow track1-sim.md Phase 2 to create shape_insertion_env.py

# 4. Test simulation
python test_sim.py

# 5. Collect demonstrations
python teleop_sim.py  # Collect 20-50 successful insertions

# 6. Package for cloud
cd sim_data
tar -czf shape_insertion_data.tar.gz *

Phase 2: Cloud Training (Google Colab Pro+)

# 1. Subscribe: https://colab.research.google.com/signup ($50/month)
# 2. Upload colab-notebook-template.ipynb to Drive
# 3. Open in Colab, enable A100 GPU
# 4. Upload data to Drive
# 5. Run all cells (2-4 hours on A100)
# 6. Download trained model from Drive

Phase 3: Local Evaluation (Mac)

# 1. Place model in models/ folder
mkdir -p models
mv ~/Downloads/act_shape_insertion_final.pth models/

# 2. Evaluate
python eval_act_sim.py

# 3. Iterate if needed

---

📅 Timeline

Before Hackathon (This Week)

Day 1 (Today):

• ✅ Run ./setup_env.sh
• ✅ Create simulation environment
• ✅ Test with python test_sim.py

Day 2:

• 🎯 Subscribe to Colab Pro+ ($50/month)
• 🎯 Collect 20-50 demonstrations
• 🎯 Upload colab-notebook-template.ipynb + data to Drive
• 🎯 Start training on A100 (2-4 hours)

Day 3:

• 🎯 Download model
• 🎯 Evaluate in simulation
• 🎯 Iterate if success rate < 60%

Day 4:

• 🎯 Optimize and improve
• 🎯 Collect more demos if needed
• 🎯 Re-train if necessary

Day 5:

• 🎯 Final testing
• 🎯 Document approach
• 🎯 Prepare for hardware transition

At Hackathon (Jan 31 - Feb 1)

Saturday:

• 9am-12pm: Hardware checkout & setup
• 12pm-6pm: Collect 20 real demos
• 6pm-12am: Fine-tune on real data

Sunday:

• 12am-6am: Continue training/testing
• 6am-12pm: Optimize, measure metrics
• 12pm-5pm: Demo & present!

---

🎯 Success Metrics

Minimum Viable (Must Achieve)

• ✅ 50% success rate
• ✅ <15 seconds cycle time
• ✅ Working demo

Target Performance

• 🎯 70% success rate
• 🎯 <10 seconds cycle time
• 🎯 2-3 different shapes

Stretch Goals

• 🚀 85%+ success rate
• 🚀 <8 seconds cycle time
• 🚀 Multiple shapes + orientations
• 🚀 Error recovery behavior

---

💰 Cost Breakdown

Cloud GPU Training

• Google Colab Pro+: $50/month (A100, recommended) ⭐ YOUR CHOICE
• Google Colab Pro: $10/month (T4/V100, may disconnect)
• Google Colab Free: $0 (T4, will disconnect)

Why Pro+:

• Industry standard (90% of users)
• Best integrations
• Zero unknowns
• No disconnects
• A100 GPU (fastest)

Total Prep Cost: $50-60 (Colab Pro+ subscription)

---

📚 Key Resources

Documentation

• LeRobot Docs: https://huggingface.co/docs/lerobot
• ACT Paper: https://arxiv.org/abs/2304.13705
• LeRobot GitHub: https://github.com/huggingface/lerobot

Event

• Website: https://physicalaihack.com/
• Registration: https://luma.com/8ca2z1rr?tk=iPsGPA

Our Files

• Local: /Users/bencxr/dev/physicalaihack/
• LeRobot: /Users/bencxr/dev/physicalaihack/lerobot/
• SO-ARM100: /Users/bencxr/dev/physicalaihack/SO-ARM100/

---

🔧 Tech Stack

Local (Mac)

• OS: macOS Darwin 23.6.0
• Python: 3.13.7
• LeRobot: Latest from GitHub
• PyTorch: CPU version
• Simulation: Gymnasium + OpenCV

Cloud (Training)

• Platform: Google Colab (free T4 GPU)
• PyTorch: CUDA version
• Training Time: 2-4 hours
• Cost: FREE

Hardware (At Hackathon)

• Robot: LeRobot SO-101
• Cameras: RGB (wrist + overhead)
• Compute: Laptop + cloud GPU
• Power: Venue provided

---

📋 Checklist

Pre-Hackathon

• Environment setup (./setup_env.sh)
• Simulation working (python test_sim.py)
• 50+ demos collected
• Trained model (>60% success in sim)
• Evaluation metrics documented
• Code backed up

At Hackathon

• Hardware checked out
• SO-101 motors configured
• Cameras calibrated
• 20+ real demos collected
• Policy fine-tuned
• >50% success on real hardware
• Metrics measured
• Demo prepared

---

🆘 Need Help?

Documentation Order

1. Start: QUICKSTART.md (15 min)
2. Cloud training: CLOUD-QUICKSTART.md (15 min setup + 2-4hr training)
3. Deep dive: track1-sim.md (complete guide)
4. Cloud options: cloud-gpu-setup.md (Colab, Lambda, RunPod)
5. Ideas: ideas-track1.md (implementation plans)

Quick Commands

# Activate environment
source lerobot-env/bin/activate

# Test simulation
python test_sim.py

# Collect demos
python teleop_sim.py

# Package for cloud
cd sim_data && tar -czf shape_insertion_data.tar.gz *

# After cloud training, evaluate
python eval_act_sim.py

# Check status
lerobot-info

---

🎉 Ready to Start?

1. Read: QUICKSTART.md
2. Run: ./setup_env.sh
3. Train: Follow CLOUD-QUICKSTART.md
4. Win: Build something amazing!

Your Mac + Free Cloud GPU = Ready for the hackathon! 🚀🤖

---

Last updated: January 2026