AI Ping Pong
A deep reinforcement learning project that teaches an AI agent to balance and guide a ball into a basket using advanced physics simulation.
Key Features
Deep Q-Learning
Utilizes advanced DQN with prioritized experience replay and double Q-learning for optimal decision making.
Physics Engine
Powered by Matter.js for realistic physics simulation with precise collision detection and momentum calculations.
Interactive Training
Watch and control the training process in real-time with visual feedback and performance metrics.
Technical Details
AI Architecture
- • Neural network with residual connections
- • Prioritized experience replay buffer
- • Adaptive learning rates
- • Progressive reward shaping
- • Early stopping optimization
State & Action Space
- • 16-dimensional state representation
- • Continuous control through discretization
- • Momentum and stability tracking
- • Relative position and velocity features
- • Dynamic difficulty adjustment
How It Works
The AI agent uses Deep Q-Learning to learn optimal control strategies for balancing a ball and guiding it into a basket. The system processes 16 different state variables including position, velocity, and derived features to make decisions about platform movement and rotation.
During training, the agent receives rewards based on:
- Distance reduction to the target basket
- Stability of the ball on the platform
- Smoothness of control actions
- Successful basket placements
The neural network architecture includes residual connections and dropout layers for better gradient flow and regularization. The experience replay system prioritizes important experiences while maintaining a diverse set of training samples.