An end-to-end pipeline that turns raw video into quantified, classifiable behavior: video processing, pose estimation (MediaPipe and YOLO), joint angles, derived indices, and sequence-based classification. Work through it in numbered order.
Prerequisites: Module 00B (Python Fundamentals)
Estimated time: ~12–18 hours
After this module you can extract body keypoints from video, compute kinematic features, and train a behavior classifier.
| Stage | Notebook | What it covers |
|---|---|---|
| 00 | Getting Started | Loading and inspecting video frames in Python |
| 01 | Video Processing - Frame Reduction | Lowering frame rate while preserving clip duration |
| 01 | Video Processing - Video Clipping | Generating paired clips from longer recordings |
| 01 | Video Processing - Video Chunker | Splitting a video into fixed-frame-count chunks |
| 02 | Pose Estimation - MediaPipe | Extracting body keypoints from video with MediaPipe |
| 02 | Pose Estimation - YOLO | Pose estimation for biomechanics applications with YOLO |
| 02 | Pose Estimation - YOLOv8 GPU (Colab) | GPU-based keypoint extraction for single files and batches |
| 02 | Training a Custom YOLO Model | Training a model to track custom keypoints or objects (Label Studio to training pipeline) |
| 03 | Extracting Joint Angles | Computing joint angles from MediaPipe and YOLO landmarks |
| 04 | Deriving Indices from Angles | Converting angle time series into biomechanical indices |
| 05 | Sequence-Based Classification | Preparing data for training behavior classification models from pose sequences |
| 05 | Sequence-Based Classification | Training behavior classification models |
Content in this module¶
- NB01: Loading Video Data and Doing Useful Work
- NB02: Frame Reduction Script — Understanding the Packages
- NB03: Video Chunker by Frame Count (OpenCV)
- NB04: Video Pair Clip Generator
- Pose Landmarks with MediaPipe — From Local Videos & Folders Using Python
- Human Pose Estimation for Sports Biomechanics using YOLO
- Batch GPU Pose Estimation with YOLO-Pose (on Google Colab)
- GPU Pose Estimation with YOLO-Pose (Colab)
- Training a YOLO Pose Model for Custom Keypoint Tracking
- Calculating Joint Angles & Derived Metrics from MediaPipe Output
- Calculating Joint Angles and Derived Metrics from YOLO Output
- Biomechanical Indices from Pose Angles
- Sequence-Based Classification using Machine Learning
- Sequence-Based Classification using Machine Learning