How to Set Up a HAProxy Load Balancer on VPS: Step-by-Step Guide for Beginners

Back in 2021, I was working as a junior NOC engineer at a small hosting company. I still remember that night I was asked to set up a HAProxy load balancer for a client. Honestly, I thought it was just a few lines of config and we’re done. Turns out… not that simple. That night, I learned a bunch of things they definitely don’t teach in class.

If you think about it, the concept is pretty straightforward. Picture the checkout line at your local grocery store: one register, everyone queues forever. Add two more registers, and the line falls apart. HAProxy is basically the security guy directing customers to whichever register is free, using rules we get to define. But in the real world, that “security guy” has to be smart, has to sense which register is overloaded, and has to be tough. It’s not just pointing people through.

Why Does Your Server Need a Load Balancer?

The classic problem is always the same: one server, traffic that can’t be handled. You start seeing symptoms like response time creeping up, clients complaining the site is slow, or timeouts during certain hours. Check the CPU usage and it’s stuck above 90 percent. Load average higher than the number of cores. Those are all screams from a server saying, “I’m overwhelmed, please help!”

Then a lot of people immediately answer: “just upgrade the server.” More RAM, more CPU, move to a bigger plan. That’s fine, but think about it — you’ll be paying a premium for resources that only get used during peak hours. Outside those hours, the server just sits idle. So why not split the load across two or three cheaper servers instead? That’s the whole essence of load balancing: not one giant machine, but several regular machines working together.

And that’s where HAProxy comes in. It’s open source, free, battle-tested, and used across thousands of production servers worldwide. Installation is easy, the config is fairly simple, and most importantly it can manage multiple backends at once with health checks you can actually rely on. For a VPS running Nginx or Apache, HAProxy sits at the front and becomes the entry point for every request.

What You’ll Need

Before we start, let’s lay out the plan. For this guide, the scenario looks like this:

• Two web servers (Nginx) running the same application — let’s call them web-1 and web-2
• One dedicated VPS for HAProxy as the load balancer
• Every server locked down with a firewall, only the needed ports open

So the flow is: user hits domain.com → goes to HAProxy → forwarded to web-1 or web-2. If one of the web servers dies or slows down, HAProxy automatically sends traffic to the healthy one. If your web server doesn’t have Nginx yet, check our Nginx installation guide first so you don’t get stuck halfway through.

Step 1: Install HAProxy

On Ubuntu 22.04, HAProxy is already in the official repositories. Just run these as root or with sudo:

apt update
apt install haproxy -y
haproxy -v

Expected output looks roughly like this:

HAProxy version 2.6.x
Copyright 2000-2022 Willy Tarreau et al.

If the version shows up, the install worked. Small note: newer distros may ship a different version, but for basic needs it really doesn’t matter.

Step 2: Understand the Config Structure

The main config file is at /etc/haproxy/haproxy.cfg. Before we touch anything, let’s understand the four main sections you’ll use the most:

• global — settings for the HAProxy process itself, like user, max connections, and SSL options
• defaults — default values used by every frontend and backend below
• frontend — the entry point for requests, where you define the IP and port to listen on
• backend — the destination of requests; this is where you list the web servers to balance between

Think of a restaurant: the frontend is the host who receives guests, the backend is the kitchen. Guests come in through the front, then get directed to the… uh, kitchen that’s free.

Step 3: Write a Basic Config

Okay, time to write the config. I recommend backing up the original file first so you’re safe:

cp /etc/haproxy/haproxy.cfg /etc/haproxy/haproxy.cfg.bak

Then open the file and replace its contents with the config below. Don’t worry, I’ll explain each part slowly.

global
    log /dev/log local0
    maxconn 50000
    user haproxy
    group haproxy

defaults
    mode http
    log global
    option httplog
    option dontlognull
    retries 3
    timeout connect 5s
    timeout client 30s
    timeout server 30s

frontend http-in
    bind *:80
    default_backend web-servers

backend web-servers
    balance roundrobin
    option httpchk GET /health
    server web-1 10.0.0.11:80 check
    server web-2 10.0.0.12:80 check

Here’s the quick breakdown:

• mode http puts HAProxy at layer 7, so it can read the HTTP request contents
• balance roundrobin means requests are distributed evenly, turn by turn, to each server
• option httpchk GET /health is the health check. HAProxy checks the /health URL every few seconds; if the server doesn’t respond, it’s removed from the backend pool
• the check at the end of each server line enables the health check for that server

Replace the IPs with your actual web servers. And don’t forget to create a /health endpoint in your web app that returns status 200. It’s dead simple — an empty file or an “ok” response is enough.

Step 4: Validate Before You Reload

This is a habit I drill into everyone I mentor: always validate the config before restarting the service. HAProxy has a built-in checker:

haproxy -c -f /etc/haproxy/haproxy.cfg

If the config is correct, you’ll see something like:

Configuration file is valid

If something’s wrong, it will point out exactly which line is the problem. Never skip this step. I skipped it once. Result? The service wouldn’t start and a client was waiting. Expensive lesson.

Once it’s valid, restart:

systemctl restart haproxy
systemctl status haproxy

Normally the service goes straight to active (running).

Step 5: Testing and Checking Logs

Now for the fun part. From another machine, hit the load balancer’s domain or IP a few times:

curl -I http://lb.example.com
curl http://lb.example.com

To make sure traffic is being split evenly, check the HAProxy log:

tail -f /var/log/haproxy.log

You’ll see lines like this:

web-servers/web-1 0/0/0/5/6 200 1234 - - ---- 2/2/1/1/0 0/0 "GET / HTTP/1.1"
web-servers/web-2 0/0/0/4/5 200 1234 - - ---- 2/2/1/1/0 0/0 "GET / HTTP/1.1"

Look at the very beginning: there’s the backend name and the server that served the request. If web-1 and web-2 show up alternately, your round robin is working. If only one keeps showing, something’s off with the health check. To confirm the health check is really active, stop one of the web servers briefly and watch the logs — HAProxy will record that the server went down and stop sending traffic to it. Crazy, right? It’s genuinely satisfying to watch.

Speaking of logs, this is also a good time to start building a monitoring habit. Once your traffic grows, you’ll want a clearer view of what’s happening on the servers. This guide to monitoring with Netdata is a solid first step.

Step 6: Open the Firewall

Don’t forget that HAProxy itself needs to be reachable from outside. Make sure ports 80 (and later 443) are open on the VPS firewall:

ufw allow 80/tcp
ufw allow 443/tcp

If you use a cloud provider like DigitalOcean or Vultr, also check the security group or firewall panel. Full details on firewall setup are in our VPS firewall setup guide.

Quick Troubleshooting Table

Here’s the table I really wish I had when I first started learning HAProxy:

Symptom	Likely Cause	Solution
502 Bad Gateway	All backends down or unreachable	Check backend status in the log, fix the health check
503 Service Unavailable	No server passes the health check	Check the /health endpoint on each server
Traffic goes to only one server	Wrong health check or another server is down	Check health check logs, verify the /health endpoint
Connections are very slow	Timeouts too small or server overwhelmed	Raise timeouts, add more backend servers
Config valid but service fails to start	Port 80 already in use by another process	Fix the bind, stop the other service or change the port

Things That Used to Send Me in Circles

Okay, let me vent a little so you don’t repeat my detours. First, sticky sessions. If your app stores sessions locally on the server (not in a database or Redis), a user’s request might land on a different server and lose the session. The fix is cookies: cookie SRV insert indirect nocache in the frontend, plus cookie web-1 / cookie web-2 on each server line. For a deeper dive into sessions in PHP apps, this Redis setup article is a good follow-up read.

Second, never underestimate the health check. A health check that only probes the port (without option httpchk) is basically blind — the server might answer on port 80 while the application is actually broken. A proper health check needs to verify something that represents “this app is alive”. That’s why I suggested a dedicated /health endpoint earlier.

Third, and this is important: a single HAProxy is a single point of failure. If it dies, all requests stop, no matter how great your backends are. For production scale, at least think about failover, or use two HAProxy instances with keepalived (VRRP). But that’s another story for another day.

Quick Summary

A load balancer isn’t rocket science, but a bad setup hits your whole production. Start simple: understand the config structure, build a proper health check, always validate before restarting, and don’t forget the firewall. From there you can level up — SSL termination, zero-downtime reloads, even high availability. If you’re ready for the next step, this production-ready HAProxy setup with SSL termination article is a great follow-up.

I hope that story of mine is useful for anyone starting out with load balancers. If you’ve ever had a “why is this so complicated” moment setting up HAProxy for the first time, or have other load balancer horror stories, drop them in the comments. Hopefully this article helps someone stuck late at night, just like I was back then.