DQN-Chase: Deep Q-Network Reinforcement Learning

A reinforcement learning project implementing a Deep Q-Network (DQN) agent that learns to evade an enemy in a grid-based pursuit-evasion game.

Project Overview

This project demonstrates core DQN concepts including:

• Deep Q-Learning: Neural network-based Q-value approximation
• Experience Replay: Breaking correlations in sequential experiences
• Target Networks: Stabilizing training with separate target and online networks
• Epsilon-Greedy Exploration: Balancing exploration and exploitation

Architecture

┌─────────────┐
│ ChaseEnv    │  (Grid-based game environment)
└─────────────┘
       ↓
┌─────────────┐
│ DQN Model   │  (Neural network for Q-value estimation)
└─────────────┘
       ↓
┌─────────────┐
│ ReplayBuffer│  (Experience storage and sampling)
└─────────────┘
       ↓
┌─────────────┐
│ Training    │  (DQN training loop)
└─────────────┘

File Structure

• env.py: Chase game environment with Pygame visualization

  • 5×5 grid-based pursuit-evasion game
  • Player vs. deterministic enemy AI
  • Reward: +1 for survival, -10 for being caught

• model.py: Deep Q-Network neural network

  • 2 hidden layers with 64 neurons each
  • ReLU activation functions
  • Input: 4D state [player_x, player_y, enemy_x, enemy_y]
  • Output: 4 Q-values (one per action)

• buffer.py: Experience Replay Buffer

  • Stores up to 2000 transitions
  • Random sampling for mini-batch training
  • Reduces correlation between consecutive samples

• train.py: Main training loop

  • 20 episodes of gameplay
  • Epsilon-greedy action selection with decay
  • Target network updates every 5 episodes
  • Model saved after training

Installation

# Clone or download the repository
cd DQN-Chase

# Install dependencies
pip install -r requirements.txt

Usage

# Train the DQN agent
python train.py

The training process will:

1. Initialize the environment and networks
2. Run 20 episodes of the game
3. Display the game window with real-time rendering
4. Print training progress (episode, score, epsilon)
5. Save the trained model to dqn_model.pth

Training Parameters

Parameter	Value	Description
grid_size	5	Size of the game grid
num_episodes	20	Number of training episodes
max_steps	100	Maximum steps per episode
batch_size	8	Mini-batch size for training
gamma	0.99	Discount factor for future rewards
epsilon_min	0.05	Minimum exploration rate
epsilon_decay	0.995	Exploration decay rate per episode
learning_rate	0.001	Adam optimizer learning rate

Game Rules

• Player (Blue Square): Controlled by the DQN agent; must evade the enemy
• Enemy (Red Square): Moves deterministically toward the player
• Actions: 0=Right, 1=Left, 2=Up, 3=Down
• Reward: +1.0 for each step survived, -10.0 if caught
• Episode Ends: When player is caught or max_steps reached

Output

After training, the model is saved as dqn_model.pth and console output includes:

Episode 1: Random chance: 0.995
Episode 1 finished | Score: 42.00 | Random chance: 0.995
Episode 5: Target network updated
...
Training finished. Model saved as 'dqn_model.pth'

Hyperparameter Tuning Tips

• Increase gamma (→0.99) for longer-term planning
• Lower learning_rate for more stable training
• Increase batch_size for more accurate gradient estimates
• Increase num_episodes for better convergence
• Adjust epsilon_decay to control exploration schedule

Requirements

• Python 3.8+
• PyTorch 2.0+
• NumPy 1.20+
• Pygame 2.0+

See requirements.txt for specific versions.

Future Enhancements

• Larger grid environments (10×10)
• Multiple enemies
• Obstacles on the grid
• Training visualization with TensorBoard
• Policy evaluation metrics
• Double DQN for improved stability
• Dueling DQN architecture

License

This project is open source and available for educational purposes.

---

Author: Faedrop
Repository: DQN-Chase