Redis基礎知識（學習筆記16--持久化 (2)）

三. AOF持久化

AOF，Append Only File，是指Redis將每一次的寫操作（命令）都以日誌的形式記錄到一個AOF文件中的持久化技術。當需要恢復記憶體數據時，將這些寫操作重新執行一次，便會恢復到之前的記憶體數據狀態。

3.1 AOF基礎配置

（1）AOF的開啟

直接修改配置文件的方式

# Please check https://redis.io/docs/latest/operate/oss_and_stack/management/persistence/ for more information.

appendonly no

預設情況下AOF持久化是沒有開啟的，通過修改配置文件中的appendonly的屬性為yes。這種方式需要重啟生效。

 命令的查看修改方式

狀態查看的，可通過下麵的命令：

> config get appendonly

修改的話

>config set appendonly yes

註意，這個命令只是修改了記憶體中的配置。這種命令修改的方式，不需要重啟生效。

如果要刷新到配置文件中

>config rewrite

 （2）文件名的配置

# - appendonly.aof.1.base.rdb as a base file.
# - appendonly.aof.1.incr.aof, appendonly.aof.2.incr.aof as incremental files.
# - appendonly.aof.manifest as a manifest file.

appendfilename "appendonly.aof"  ###註意：這隻是定義了這類文件的首碼

Redis 7 在這裡發生了重大變化。原來只有一個appendonly.aof文件，現在具有了三類多個文件：

*** 基礎文件：可以是RDF格式也可以是AOF格式。其存放的內容是由RDB轉為AOF當時記憶體的快照數據。該文件可以有多個。

*** 增量文件：以操作日誌形式記錄轉為AOF後的寫入操作。該文件可以有多個。

*** 清單文件：用於維護AOF文件的創建順序，保障激活時的應用順序，該文件只有一個。

（3）混合式持久化開啟

# Redis can create append-only base files in either RDB or AOF formats. Using
# the RDB format is always faster and more efficient, and disabling it is only
# supported for backward compatibility purposes.
aof-use-rdb-preamble yes

對於基本文件(例如：appendonly.aof.1.base.rdb as a base file) 可以是RDF格式也可以是AOF格式，通過aof-use-rdb-preamble 屬性可以選擇。其預設值為yes，即預設AOF持久化的基本文件為rdb格式文件，也就是預設採用混合式持久化。

（4）AOF完整配置參數

############################## APPEND ONLY MODE ###############################

# By default Redis asynchronously【非同步】 dumps the dataset on disk. This mode is
# good enough in many applications, but an issue with the Redis process or
# a power outage may result into a few minutes of writes lost (depending on
# the configured save points).###概況說RDB存在的風險
#
# The Append Only File is an alternative【可供替代的】 persistence mode that provides
# much better durability【持久性】. For instance using the default data fsync policy
# (see later in the config file) Redis can lose just one second of writes in a
# dramatic【巨大的，引人註目的】 event like a server power outage, or a single write if something
# wrong with the Redis process itself happens, but the operating system is
# still running correctly.###概況說AOF開啟帶來的好處
#
# AOF and RDB persistence can be enabled at the same time without problems.
# If the AOF is enabled on startup Redis will load the AOF, that is the file
# with the better durability guarantees.
#
# Note that changing this value in a config file of an existing database and
# restarting the server can lead to data loss. A conversion needs to be done
# by setting it via CONFIG command on a live server first.
#
# Please check https://redis.io/docs/latest/operate/oss_and_stack/management/persistence/ for more information.

appendonly no

# The base name of the append only file.
#
# Redis 7 and newer use a set of append-only files to persist the dataset
# and changes applied to it. There are two basic types of files in use:
#
# - Base files, which are a snapshot representing the complete state of the
#   dataset at the time the file was created. Base files can be either in
#   the form of RDB (binary serialized) or AOF (textual commands).
# - Incremental files, which contain additional commands that were applied
#   to the dataset following the previous file.
#
# In addition, manifest【貨單；清單】 files are used to track the files and the order in
# which they were created and should be applied.
#
# Append-only file names are created by Redis following a specific pattern.
# The file name's prefix is based on the 'appendfilename' configuration
# parameter, followed by additional information about the sequence【序號】 and type【類型】.
#
# For example, if appendfilename is set to appendonly.aof, the following file
# names could be derived【產生；獲得】:
#
# - appendonly.aof.1.base.rdb as a base file.【大多數情況下，這個類型的文件就一個；如果記憶體裡面的數據量很大時，也可能會不止一個】
# - appendonly.aof.1.incr.aof, appendonly.aof.2.incr.aof as incremental files.【這個類型的文件會同時有多個】
# - appendonly.aof.manifest as a manifest file.

appendfilename "appendonly.aof"

# For convenience, Redis stores all persistent append-only files in a dedicated【專門的；特定的】
# directory. The name of the directory is determined by the appenddirname
# configuration parameter.

appenddirname "appendonlydir"

# The fsync() call tells the Operating System to actually write data on disk
# instead of waiting for more data in the output buffer. Some OS will really flush
# data on disk, some other OS will just try to do it ASAP.
#
# Redis supports three different modes:
#
# no: don't fsync, just let the OS flush the data when it wants. Faster.
# always: fsync after every write to the append only log. Slow, Safest.
# everysec: fsync only one time every second. Compromise.
#
# The default is "everysec", as that's usually the right compromise between
# speed and data safety. It's up to you to understand if you can relax this to
# "no" that will let the operating system flush the output buffer when
# it wants, for better performances (but if you can live with the idea of
# some data loss consider the default persistence mode that's snapshotting),
# or on the contrary, use "always" that's very slow but a bit safer than
# everysec.
#
# More details please check the following article:
# http://antirez.com/post/redis-persistence-demystified.html
#
# If unsure, use "everysec".

# appendfsync always
appendfsync everysec
# appendfsync no

# When the AOF fsync policy is set to always or everysec, and a background
# saving process (a background save or AOF log background rewriting) is
# performing a lot of I/O against the disk, in some Linux configurations
# Redis may block too long on the fsync() call. Note that there is no fix for
# this currently, as even performing fsync in a different thread will block
# our synchronous write(2) call.
#
# In order to mitigate this problem it's possible to use the following option
# that will prevent fsync() from being called in the main process while a
# BGSAVE or BGREWRITEAOF is in progress.
#
# This means that while another child is saving, the durability of Redis is
# the same as "appendfsync no". In practical terms, this means that it is
# possible to lose up to 30 seconds of log in the worst scenario (with the
# default Linux settings).
#
# If you have latency problems turn this to "yes". Otherwise leave it as
# "no" that is the safest pick from the point of view of durability.

no-appendfsync-on-rewrite no

# Automatic rewrite of the append only file.
# Redis is able to automatically rewrite the log file implicitly calling【隱式調用】
# BGREWRITEAOF when the AOF log size grows by the specified percentage【指定的百分比】.
#
# This is how it works: Redis remembers the size of the AOF file after the
# latest rewrite (if no rewrite has happened since the restart, the size of
# the AOF at startup is used).
#
# This base size is compared to the current size. If the current size is
# bigger than the specified percentage, the rewrite is triggered. Also
# you need to specify a minimal size for the AOF file to be rewritten, this
# is useful to avoid rewriting the AOF file even if the percentage increase
# is reached but it is still pretty small.
#
# Specify a percentage of zero in order to disable the automatic AOF
# rewrite feature.

auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb

# An AOF file may be found to be truncated at the end during the Redis
# startup process, when the AOF data gets loaded back into memory.
# This may happen when the system where Redis is running
# crashes, especially when an ext4 filesystem is mounted without the
# data=ordered option (however this can't happen when Redis itself
# crashes or aborts but the operating system still works correctly).
#
# Redis can either exit with an error when this happens, or load as much
# data as possible (the default now) and start if the AOF file is found
# to be truncated at the end. The following option controls this behavior.
#
# If aof-load-truncated is set to yes, a truncated AOF file is loaded and
# the Redis server starts emitting a log to inform the user of the event.
# Otherwise if the option is set to no, the server aborts with an error
# and refuses to start. When the option is set to no, the user requires
# to fix the AOF file using the "redis-check-aof" utility before to restart
# the server.
#
# Note that if the AOF file will be found to be corrupted in the middle
# the server will still exit with an error. This option only applies when
# Redis will try to read more data from the AOF file but not enough bytes
# will be found.
aof-load-truncated yes

# Redis can create append-only base files in either RDB or AOF formats. Using
# the RDB format is always faster and more efficient, and disabling it is only
# supported for backward compatibility purposes.
aof-use-rdb-preamble yes

# Redis supports recording timestamp annotations in the AOF to support restoring
# the data from a specific point-in-time. However, using this capability changes
# the AOF format in a way that may not be compatible with existing AOF parsers.
aof-timestamp-enabled no

################################ SHUTDOWN #####################################

 3.2 AOF 文件格式

AOF文件包含三類文件：基本文件、增量文件與清單文件。其中基本文件一般為rdb格式，不再贅述，下麵主要看下增量文件和清單文件的內容格式。

（1）Redis 協議

 增量文件擴展名為.aof，採用AOF格式。AOF格式其實就是redis通訊協議格式，AOF持久化文件的本質就是基於redis通訊協議的文本，將命令以純文本的方式寫入到文件中。

redis協議規定，redis文本是以行來劃分，每行以\r\n行結束。每一行都有一個消息頭，以表示消息類型。消息頭由六種不同的符號表示，其意義如下：

• （+）表示一個正確的狀態消息；
• （-）表示一個錯誤的消息；
• （*）表示消息體總共有多少行，不包括當前行；
• （$）表示下一行消息數據的長度，不包括換行符長度\r\n；
• （空）表示一個消息數據；
• （:）表示返回一個數值。

（2）查看AOF文件

打開appendonly.aof.1.incr.aof文件，可以看到格式如下：

（3）清單文件

打開appendonly.aof.manifest文件，其格式如下：

 tpye有兩種類型：b（base file） 和 i（incremental file）。

該文件首先會按照seq序號列舉出所有基本文件，基本文件type類型為b，然後再按照seq序號再列舉出所有增量文件，增量文件type為i。

對redis啟動時的數據恢復，也會按照該文件從上到下依次載入他們中的數據。

3.3 Rewrite 機制

隨著使用時間的推移，AOF文件會越來越大。為了防止AOF文件由於太大而占用大量的磁碟空間，降低性能，redis引入了rewrite機制來對AOF文件進行壓縮。

（1） rewrite的定義

所謂rewrite其實就是對AOF文件進行重寫整理。當rewrite開啟後，主進程redis-server創建出一個子進程bgrewriteaof，由該子進程完成rewrite過程。其首先對現有aof文件進行rewrite計算，將計算結果寫入到一個臨時文件，寫入完畢後，再rename該臨時文件為原aof文件名，覆蓋原有文件。

（2）rewrite計算

rewrite計算也稱為rewrite策略。rewrite計算遵循以下策略：

• 讀操作命令不寫入文件；
• 無效命令不寫入文件；
• 過期數據不寫入文件；
• 多條命令合併寫入文件。

（3）手動開啟rewrite

rewrite過程的執行有兩種方式。一種是通過bgrewriteaof命令手動開啟，一種是通過設置條件自動開啟。

手動啟動（觸發）的命令：

> bgrewriteaof

該命令會使主進程redis-server創建出一個子進程bgrewriteaof，由該子進程完成rewrite過程。而在rewrite期間，redis-server仍是可以對外提供讀寫服務的 。

（4）自動開啟rewrite 

其實就是配置文件中的參數，如下：

# Automatic rewrite of the append only file.
# Redis is able to automatically rewrite the log file implicitly calling
# BGREWRITEAOF when the AOF log size grows by the specified percentage.
#
# This is how it works: Redis remembers the size of the AOF file after the
# latest rewrite (if no rewrite has happened since the restart, the size of
# the AOF at startup is used).
#
# This base size is compared to the current size. If the current size is
# bigger than the specified percentage, the rewrite is triggered. Also
# you need to specify a minimal size for the AOF file to be rewritten, this
# is useful to avoid rewriting the AOF file even if the percentage increase
# is reached but it is still pretty small.
#
# Specify a percentage of zero in order to disable the automatic AOF
# rewrite feature.

auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb

學習參閱特別聲明

【Redis視頻從入門到高級】

【https://www.bilibili.com/video/BV1U24y1y7jF?p=11&vd_source=0e347fbc6c2b049143afaa5a15abfc1c】