Fix Redis Bgsave Broken Alert in Netdata — Step by Step

Picture this: it’s 3 AM, you’re half asleep, and your phone buzzes with a Telegram notification from Netdata. You squint at the screen and see a big red CRITICAL alert staring back at you. Your stomach drops. The message reads redis_local.bgsave_health — Redis background save is broken. That moment hits every NOC engineer at least once during their career, and if you’re reading this right now, you’re probably in that exact situation.

Here’s the thing — Redis is the backbone of countless applications. It handles session storage, caching, message queues, and real-time data. When Redis bgsave fails, it means the periodic snapshot of your data to disk isn’t working. And that’s a big deal because if your server crashes unexpectedly, any data that wasn’t saved to disk is gone. Poof. Like a house without a fire alarm — everything seems fine until it’s not.

I’ve been handling server infrastructure for years, and I can tell you that this specific alert — redis_local.bgsave_health CRITICAL — is one of the most common Redis issues on servers running CyberPanel or DirectAdmin. The good news? It’s usually fixable in under 5 minutes with a simple service restart sequence. The bad news? If you ignore it, you’re playing Russian roulette with your data.

The full notification you’ll typically see in Netdata looks something like this:

server.iixcp.domainku.net is critical - redis bgsave broken
redis_local.bgsave_health
Redis background save = ok
Status of the last RDB save operation (0: ok, 1: error)

Here’s what trips people up: the log says Redis background save = ok but the alarm is still CRITICAL. This happens because Netdata caches the bgsave status and needs a service refresh to clear it. So even though the most recent save succeeded, the alarm stays active because Netdata hasn’t re-evaluated the condition yet. This is one of those quirks that drives junior engineers crazy — they see “ok” in the output and wonder why the alarm won’t clear.

Why Does Redis Bgsave Fail in the First Place?

Before we jump into the fix, let’s understand what’s actually happening under the hood. Bgsave (background save) is Redis’s mechanism for creating periodic snapshots of your data on disk. If your redis.conf has save 900 1, Redis will automatically save data to disk whenever at least 1 key has changed in the last 900 seconds. This process runs in the background — hence the name.

When bgsave fails, several things could be going wrong:

• Disk space exhaustion — Redis can’t write the .rdb file because the disk is full
• Memory pressure (fork failure) — bgsave requires a fork() call, which essentially duplicates Redis’s memory usage. If Redis is using a lot of RAM, the fork can fail
• Permission issues — Redis doesn’t have write permissions to the target directory
• Corrupt RDB file — An existing dump.rdb file is damaged from a previous crash
• autom8redis service conflicts — On CyberPanel servers, supporting services like autom8redis_broker and autom8redis_taskq can interfere with Redis operations

In my experience, the most common culprit isn’t Redis itself but the supporting services. Specifically, autom8redis_broker and autom8redis_taskq are message brokers used by CyberPanel that can experience deadlocks or connection pool exhaustion, which then cascades into Redis bgsave failures.

Symptoms and Error Messages

Here are the typical symptoms you’ll encounter when Redis bgsave is having problems:

The Fix: Restart Redis and Related Services

Alright, let’s get to it. Based on extensive field experience, the most effective way to resolve the redis_local.bgsave_health CRITICAL alert is to restart three services in sequence. Not just Redis alone, but also the autom8redis services that support it.

Step 1: Check Current Redis Status

Before restarting anything, check the current state of Redis:

systemctl status redis

Look at the output carefully. If the status shows active (running) but there are errors in the log section, Redis is running but has internal issues. If it shows inactive (dead) or failed, that’s a more serious problem that we’ll address.

Step 2: Check the Redis Socket

On CyberPanel or DirectAdmin servers with custom Redis sockets, verify the socket is active:

ls -la /home/userpanel/tmp/redis.sock

This socket file is crucial because many applications connect to Redis through a Unix socket rather than a TCP port. If the socket is missing or corrupt, restarting Redis alone won’t be enough — you also need to ensure applications can reconnect to the new socket.

redis-cli -s /home/userpanel/tmp/redis.sock ping

Expected output: PONG. If you get a PONG response, Redis is still responding. If you get an error or timeout, Redis definitely needs a restart.

Step 3: Restart All Three Services

This is the main event. Run these three restart commands in sequence:

systemctl restart redis
systemctl restart autom8redis_broker
systemctl restart autom8redis_taskq

Why all three? Because autom8redis_broker and autom8redis_taskq maintain persistent connections to the main Redis instance. If you only restart Redis without restarting these two services, you can end up in a state where Redis is fresh but the autom8redis services are still holding onto stale connections. The result? bgsave continues to fail due to connection conflicts.

After running these commands, wait about 10-30 seconds for all services to fully initialize.

Step 4: Verify Everything Is Running

After the restart, verify that all three services are running properly:

systemctl status redis
systemctl status autom8redis_broker
systemctl status autom8redis_taskq

All three should show active (running). If any of them show failed, check the logs:

journalctl -u redis --since "5 minutes ago" --no-pager
journalctl -u autom8redis_broker --since "5 minutes ago" --no-pager
journalctl -u autom8redis_taskq --since "5 minutes ago" --no-pager

Step 5: Test bgsave Manually

To confirm bgsave is working, trigger a manual save:

redis-cli -s /home/userpanel/tmp/redis.sock bgsave

Expected output: Background saving started. Wait a few seconds, then check if the dump.rdb file was updated:

ls -la /var/lib/redis/dump.rdb

The timestamp on dump.rdb should show a very recent time (within the last few seconds). If the timestamp is old, there’s a deeper issue that needs investigation.

Step 6: Wait for Netdata to Clear the Alarm

After bgsave succeeds, Netdata should automatically clear the alarm within a few minutes. Check the Netdata dashboard to confirm. If the alarm persists after 5-10 minutes, you can force a clear by restarting the Netdata agent:

systemctl restart netdata

Above is an illustration of the Netdata dashboard when the bgsave alarm is active. After you complete the restart and verification steps, this status should change to OK.

Troubleshooting: When Restart Alone Doesn’t Fix It

Sometimes a simple restart isn’t enough. Here are some advanced issues you might encounter and how to handle them:

Case: vm.overcommit_memory

This is a pretty common issue on servers with limited RAM. When Redis tries to perform bgsave, it needs to execute a fork() system call to duplicate its process. If Linux refuses to allocate the additional memory, bgsave fails.

Check if this is the problem:

cat /proc/sys/vm/overcommit_memory

If the output is 0, Linux is using the default heuristic which can reject forks. For Redis, it’s recommended to set this to 1:

echo 1 > /proc/sys/vm/overcommit_memory

To make this permanent:

echo "vm.overcommit_memory = 1" >> /etc/sysctl.conf
sysctl -p

Pro Tips from the NOC Floor

Quick Reference Command Sheet

Preventing This Issue from Happening Again

Once you’ve fixed the immediate problem, here are some preventive measures to keep it from recurring:

1. Monitor disk space proactively — Set up alerting in Netdata for disk usage. When disk hits 80%, it’s time to clean up.
2. Set maxmemory in Redis — Prevent Redis from consuming all server RAM. Edit /etc/redis.conf and set maxmemory 2gb (adjust based on your needs).
3. Enable AOF (Append Only File) — In addition to RDB snapshots, activate AOF as a backup mechanism. Edit /etc/redis.conf and set appendonly yes.
4. Schedule periodic Redis restarts — Some engineers set up a cron job to restart Redis weekly during off-peak hours. This can prevent memory leaks from accumulating over time.
5. Keep Redis updated — Some bugs in older versions have been fixed in newer releases. Check the changelog before upgrading.

If you want to learn more about server monitoring, check out our comprehensive guide on setting up server monitoring with Netdata. For similar issues with other services, our article on restarting services on Linux with systemctl is a useful reference.

So there you have it — the complete guide to fixing the redis_local.bgsave_health CRITICAL alert in Netdata. The bottom line: don’t panic, check your disk and memory first, then restart all three services (redis, autom8redis_broker, autom8redis_taskq) in sequence. In most cases, that’s all it takes to clear the alarm and get Redis back to normal.

If you’re dealing with more complex issues — like Redis repeatedly crashing or data loss — it might be time to set up better monitoring or consult with a professional managed server team. Don’t let a small problem become a big one just because you kept putting it off.