Redis appendonly.aof file size out of control

redis

I host a Rails 5 website that uses Sidekiq to manage its background jobs. The problem I have is that regularly the appendonly.aof file grows to big that it fills that whole disk where it is saved.

It there a way to limit the size of appendonly.aof ?

My redis.conf file is pretty standard

########################## GENERAL

daemonize yes pidfile /var/run/redis/redis-server.pid port 6379
tcp-backlog 511 bind 0.0.0.0

timeout 0 tcp-keepalive 0

loglevel notice

logfile /var/log/redis/redis-server.log

databases 16

########################## SNAPSHOTTING

save 900 1 save 300 10 save 60 10000 stop-writes-on-bgsave-error yes
rdbcompression yes rdbchecksum yes dbfilename dump.rdb dir
/var/lib/redis

########################### REPLICATION

slave-serve-stale-data yes slave-read-only yes repl-diskless-sync no
repl-diskless-sync-delay 5 repl-disable-tcp-nodelay no slave-priority
100

############################ SECURITY

############################# LIMITS

maxmemory-policy noeviction

######################## APPEND ONLY MODE

appendonly yes appendfilename "appendonly.aof" appendfsync everysec
no-appendfsync-on-rewrite no auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb aof-load-truncated yes

########################## LUA SCRIPTING

lua-time-limit 5000

############################ SLOW LOG

slowlog-log-slower-than 10000

slowlog-max-len 128

########################## LATENCY MONITOR

latency-monitor-threshold 0

####################### Event notification

notify-keyspace-events ""

######################### ADVANCED CONFIG

hash-max-ziplist-entries 512 hash-max-ziplist-value 64

list-max-ziplist-entries 512 list-max-ziplist-value 64

set-max-intset-entries 512

zset-max-ziplist-entries 128 zset-max-ziplist-value 64

hll-sparse-max-bytes 3000

activerehashing yes

client-output-buffer-limit normal 0 0 0 client-output-buffer-limit
slave 256mb 64mb 60 client-output-buffer-limit pubsub 32mb 8mb 60 hz
10

aof-rewrite-incremental-fsync yes

Best Answer

Refer to https://redislabs.com/ebook/part-2-core-concepts/chapter-4-keeping-data-safe-and-ensuring-performance/4-1-persistence-options/4-1-3-rewritingcompacting-append-only-files/

With auto-aof-rewrite-percentage and auto-aof-rewrite-min-size options, you should be able to have a handle on the size.

You can also execute command BGREWRITEAOF in your redis-cli to trigger it manually.

Related Topic