Build Your Own Load Balancer

A load balancer built from scratch in Python using asyncio and aiohttp. Inspired by John Crickett's Coding Challenges.

Features

• Round-Robin routing across multiple backend servers
• Health Checks — periodic pings to backends, automatically removes unhealthy servers and re-adds them when they recover
• Async I/O — handles concurrent connections efficiently using asyncio
• Fault Tolerance — returns 502 on backend failure, 503 when no healthy backends available

Architecture

Client Request
      │
      ▼
┌─────────────┐
│Load Balancer│ (port 8080)
└─────┬───────┘
      │ Round Robin
      ├──────────────┬──────────────┐
      ▼              ▼              ▼
┌──────────┐  ┌──────────┐  ┌──────────┐
│Backend :1│  │Backend :2│  │Backend :3│
│  (8081)  │  │  (8082)  │  │  (8083)  │
└──────────┘  └──────────┘  └──────────┘

Getting Started

Prerequisites

• Python 3.8+
• pip

Installation

git clone https://github.com/<your-username>/LoadBalancer.git
cd LoadBalancer
pip install -r requirements.txt

Running

1. Start backend servers (each in a separate terminal):

python backend.py 8081
python backend.py 8082
python backend.py 8083

2. Start the load balancer:

# Basic (round-robin only)
python load_balancer.py

# With health checks
python health_check.py

3. Send requests:

curl http://localhost:8080/
curl http://localhost:8080/
curl http://localhost:8080/

Each request will be routed to a different backend in round-robin order.

Testing Health Checks

1. Start all backends and health_check.py
2. Kill one backend (e.g., Ctrl+C on port 8082)
3. Wait ~10 seconds — the load balancer will detect it's down and stop routing to it
4. Restart the backend — it will be automatically added back to the pool

Project Structure

├── backend.py                  # Simple HTTP backend server
├── load_balancer.py            # Basic load balancer with round-robin
├── health_check.py              # Load balancer with health checks
├── requirements.txt            # Python dependencies
└── README.md

How It Works

1. The load balancer listens on port 8080 for incoming HTTP requests
2. For each request, it selects the next backend using round-robin
3. It forwards the request (method, headers, body) to the chosen backend
4. The backend's response is returned to the client
5. (With health checks) A background task pings each backend every 10 seconds and updates the healthy pool

Built With

• Python asyncio — asynchronous I/O
• aiohttp — async HTTP client/server framework