分析MariaDB初始化腳本mysql_install_db

在初始化MySQL的過程中經常會碰到各種問題，如出現這些問題的原因無非是兩種， 1> 沒有傳遞合適的參數。譬如： [root@localhost ~]# /usr/test/mariadb-10.1.16-linux-x86_64/scripts/mysql_install_db --datad ...

在初始化MySQL的過程中經常會碰到各種問題，如

FATAL ERROR: Could not find ./bin/my_print_defaults

ERROR: 1  Can't create/write to file '/root/test/data/mysql/db.MYI' (Errcode: 13 - Permission denied)

出現這些問題的原因無非是兩種，

1> 沒有傳遞合適的參數。

譬如：

[root@localhost ~]# /usr/test/mariadb-10.1.16-linux-x86_64/scripts/mysql_install_db --datadir=/usr/test/mariadb-10.1.16-linux-x86_64/ --user=mysql &

沒有指定basedir，無法找到my_print_defaults命令

2> 目錄對當前用戶沒有許可權

譬如：

[root@localhost test]# /usr/test/Percona-Server-5.6.31-rel77.0-Linux.x86_64.ssl101/scripts/mysql_install_db --basedir=/usr/test/Percona-Server-5.6.31-rel77.0-Linux.x86_64.ssl101/ --user=mysql

沒有指定datadir，預設是當前目錄下的data目錄。如果放到/usr下執行倒是沒有問題，/root目錄本身的許可權是500的。對於其它用戶，它是沒有許可權在/root目錄及其子目錄創建文件的。

下麵對MariaDB的初始化腳本進行較為詳細的解析。

首先定義初始化變數

這裡面ldata即數據目錄，如果在執行mysql_install_db腳本時，沒有顯示指定--datadir，則datadir預設在當前目錄下的data。

basedir=""
builddir=""
ldata="./data"
langdir=""
srcdir=""

args=""
defaults=""
mysqld_opt=""
user=""

force=0
in_rpm=0
ip_only=0
cross_bootstrap=0

定義usage函數

在使用mysql_install_db --help即輸出的是usage函數的內容

usage()
{
  cat <<EOF
Usage: $0 [OPTIONS]
  --basedir=path       The path to the MariaDB installation directory.
  --builddir=path      If using --srcdir with out-of-directory builds, you
                       will need to set this to the location of the build
                       directory where built files reside.
  --cross-bootstrap    For internal use.  Used when building the MariaDB system
                       tables on a different host than the target.
  --datadir=path       The path to the MariaDB data directory.
  --defaults-extra-file=name
                       Read this file after the global files are read.
  --defaults-file=name Only read default options from the given file name.
  --force              Causes mysql_install_db to run even if DNS does not
                       work.  In that case, grant table entries that
                       normally use hostnames will use IP addresses.
  --help               Display this help and exit.                     
  --ldata=path         The path to the MariaDB data directory. Same as
                       --datadir.
  --no-defaults        Don't read default options from any option file.
  --defaults-file=path Read only this configuration file.
  --rpm                For internal use.  This option is used by RPM files
                       during the MariaDB installation process.
  --skip-name-resolve  Use IP addresses rather than hostnames when creating
                       grant table entries.  This option can be useful if
                       your DNS does not work.
  --srcdir=path        The path to the MariaDB source directory.  This option
                       uses the compiled binaries and support files within the
                       source tree, useful for if you don't want to install
                       MariaDB yet and just want to create the system tables.
  --user=user_name     The login username to use for running mysqld.  Files
                       and directories created by mysqld will be owned by this
                       user.  You must be root to use this option.  By default
                       mysqld runs using your current login name and files and
                       directories that it creates will be owned by you.

All other options are passed to the mysqld program

EOF
  exit 1
}

定義列印函數

s_echo()
{
  if test "$in_rpm" -eq 0 -a "$cross_bootstrap" -eq 0
  then
    echo "$1"
  fi
}

其中$in_rpm對應--rpm參數，$cross_bootstrap對應 --cross-bootstrap參數。

關於這兩個參數的作用，可以參考上面usage函數中的說明。

在腳本執行失敗時，調用的函數

link_to_help()
{
  echo
  echo "The latest information about mysql_install_db is available at"
  echo "https://mariadb.com/kb/en/installing-system-tables-mysql_install_db"
}

定義參數解析函數

parse_arg()
{
  echo "$1" | sed -e 's/^[^=]*=//'
}

其中s是替換，^代表行首定位符，[]代表匹配一組字元里的任意字元，*匹配0個或多個前一字元，[^ ]代表匹配不在指定範圍內的字元。

這個函數實現的效果是截取“=”號後的字元。

譬如，輸入的變數是--basedir=/usr/test，則輸出的結果是/usr/test

定義命令行解析函數

parse_arguments()
{
  # We only need to pass arguments through to the server if we don't
  # handle them here.  So, we collect unrecognized options (passed on
  # the command line) into the args variable.
  pick_args=
  if test "$1" = PICK-ARGS-FROM-ARGV
  then
    pick_args=1
    shift
  fi

  for arg
  do
    case "$arg" in
      --force) force=1 ;;
      --basedir=*) basedir=`parse_arg "$arg"` ;;
      --builddir=*) builddir=`parse_arg "$arg"` ;;
      --srcdir=*)  srcdir=`parse_arg "$arg"` ;;
      --ldata=*|--datadir=*|--data=*) ldata=`parse_arg "$arg"` ;;
      --user=*)
        # Note that the user will be passed to mysqld so that it runs
        # as 'user' (crucial e.g. if log-bin=/some_other_path/
        # where a chown of datadir won't help)
        user=`parse_arg "$arg"` ;;
      --skip-name-resolve) ip_only=1 ;;
      --verbose) verbose=1 ;; # Obsolete
      --rpm) in_rpm=1 ;;
      --help) usage ;;
      --no-defaults|--defaults-file=*|--defaults-extra-file=*)
        defaults="$arg" ;;

      --cross-bootstrap|--windows)
        # Used when building the MariaDB system tables on a different host than
        # the target. The platform-independent files that are created in
        # --datadir on the host can be copied to the target system.
        #
        # The most common use for this feature is in the Windows installer
        # which will take the files from datadir and include them as part of
        # the install package.  See top-level 'dist-hook' make target.
        #
        # --windows is a deprecated alias
        cross_bootstrap=1 ;;

      *)
        if test -n "$pick_args"
        then
          # This sed command makes sure that any special chars are quoted,
          # so the arg gets passed exactly to the server.
          # XXX: This is broken; true fix requires using eval and proper
          # quoting of every single arg ($basedir, $ldata, etc.)
          #args="$args "`echo "$arg" | sed -e 's,\([^a-zA-Z0-9_.-]\),\\\\\1,g'`
          args="$args $arg"
        fi
        ;;
    esac
  done
}

其中，for arg相當於for arg in '"$@"'。

PS：

關於shell腳本中$*，$@和$#三者的區別：

腳本名稱叫test.sh 傳入三個參數: 1 2 3

運行test.sh 1 2 3後

$*為"1 2 3"（一起被引號包住）

$@為"1" "2" "3"（分別被包住）

$#為3（參數數量）

判斷給定的文件是否存在

find_in_basedir()
{
  case "$1" in
    --dir)
      return_dir=1; shift
      ;;
  esac

  file=$1; shift

  for dir in "$@"
  do
    if test -f "$basedir/$dir/$file"
    then
      if test -n "$return_dir"
      then
        echo "$basedir/$dir"
      else
        echo "$basedir/$dir/$file"
      fi
      break
    fi
  done
}

其中 test -f是判斷給定的文件是否存在，test -n代表給定的變數是否為非空。

解析命令行參數

正如它註釋中提到的，第一次解析的參數的目錄在於得到my_print_defaults命令的路徑。

# Ok, let's go.  We first need to parse arguments which are required by
# my_print_defaults so that we can execute it first, then later re-parse
# the command line to add any extra bits that we need.
parse_arguments "$@"

確認my_print_defaults命令的路徑

#
# We can now find my_print_defaults.  This script supports:
#
#   --srcdir=path pointing to compiled source tree
#   --basedir=path pointing to installed binary location
#
# or default to compiled-in locations.
#
if test -n "$srcdir" && test -n "$basedir"
then
  echo "ERROR: Specify either --basedir or --srcdir, not both."
  link_to_help
  exit 1
fi
if test -n "$srcdir"
then
  if test -z "$builddir"
  then
    builddir="$srcdir"
  fi
  print_defaults="$builddir/extra/my_print_defaults"
elif test -n "$basedir"
then
  print_defaults=`find_in_basedir my_print_defaults bin extra`
  if test -z "$print_defaults"
  then
    cannot_find_file my_print_defaults $basedir/bin $basedir/extra
    exit 1
  fi
else
  print_defaults="./bin/my_print_defaults"
fi

首先，判斷srcdir和basedir是否指定，如果兩者都指定的話，則會報錯，在這裡，srcdir指向的是源碼包的編譯路徑，basedir指向的是二進位包的解壓路徑。畢竟是兩種不同的安裝方式，一個是源碼安裝，一個是二進位壓縮包直接解壓，兩者的目錄層次會不一樣。

上述腳本的判斷邏輯是，如果$srcdir指定了，則my_print_defaults命令位於$builddir/extra中。

如果沒有指定$srcdir，而指定了$basedir，則my_print_defaults要麼位於$basedir/bin中，要麼位於$basedir/extra中。如果沒有找到該命令，則直接報錯退出腳本。

如果$srcdir和$basedir都沒有指定，則my_print_defaults預設在當前目錄中的bin目錄下，此時，它假定你是在$basedir上執行該初始化命令的。

判斷my_print_defaults對於當前用戶是否有可執行許可權

if test ! -x "$print_defaults"
then
  cannot_find_file "$print_defaults"
  exit 1
fi

獲取配置文件中[mysqld]和[mysql_install_db]區域的值

# Now we can get arguments from the groups [mysqld] and [mysql_install_db]
# in the my.cfg file, then re-run to merge with command line arguments.
parse_arguments `"$print_defaults" $defaults --mysqld mysql_install_db`

既然能執行my_print_defaults，則可以根據該命令得到配置文件中[mysqld]和[mysql_install_db]區域的值

上面的$defaults是之前定義的--no-defaults，--defaults-file，--defaults-extra-file

譬如如果--defaults-file=/usr/local/mysql，則defaults="--defaults-file=/usr/local/mysql"。

對於下麵這個命令"$print_defaults" $defaults --mysqld mysql_install_db，針對於我本機的環境，它實際上執行的是

# /usr/test/mariadb-10.1.16-linux-x86_64/bin/my_print_defaults --defaults-file=/usr/test/mariadb-10.1.16-linux-x86_64/my.cnf --mysqld mysql_install_db

--port=3308
--basedir=/usr/test/mariadb-10.1.16-linux-x86_64
--datadir=/usr/test/mariadb-10.1.16-linux-x86_64/data
--skip-external-locking
--user=mysql
--key_buffer_size=16K
--max_allowed_packet=1M
--table_open_cache=4
--sort_buffer_size=64K
--read_buffer_size=256K
--read_rnd_buffer_size=256K
--net_buffer_length=2K
--thread_stack=240K
--server-id=1

其中，--mysqld前面不用帶“--”也行。

但是5.6.31 MySQL社區版的my_print_defaults只支持不帶“--”的mysqld，由此可見，兩者在語法方面還是有一定的差別的。

根據上面的輸出的結果，則腳本中的這個命令等價於

parse_arguments --port=3308 --basedir=/usr/test/mariadb-10.1.16-linux-x86_64 --datadir=/usr/test/mariadb-10.1.16-linux-x86_64/data --skip-external-locking --user=mysql --key_buffer_size=16K --max_allowed_packet=1M --table_open_cache=4 --sort_buffer_size=64K --read_buffer_size=256K --read_rnd_buffer_size=256K --net_buffer_length=2K --thread_stack=240K --server-id=1

執行完上面這個命令後，實際上只有三個參數傳遞進來了，--basedir，--datadir，--user

再次執行命令解析函數

parse_arguments PICK-ARGS-FROM-ARGV "$@"

這次調用的目的是捕捉命令行中傳遞的其它參數。

譬如執行如下命令

# /usr/test/mariadb-10.1.16-linux-x86_64/scripts/mysql_install_db --defaults-file=/usr/test/mariadb-10.1.16-linux-x86_64/my.cnf --basedir=/usr/test/mariadb-10.1.16-linux-x86_64/ 123 456 789

則args中的值為' 123 456 789'

定義初始化所需文件的路徑

# Configure paths to support files
if test -n "$srcdir"
then
  basedir="$builddir"
  bindir="$basedir/client"
  extra_bindir="$basedir/extra"
  mysqld="$basedir/sql/mysqld"
  langdir="$basedir/sql/share/english"
  pkgdatadir="$srcdir/scripts"
  scriptdir="$srcdir/scripts"
elif test -n "$basedir"
then
  bindir="$basedir/bin"
  extra_bindir="$bindir"
  mysqld=`find_in_basedir mysqld libexec sbin bin`
  if test -z "$mysqld"
  then
    cannot_find_file mysqld $basedir/libexec $basedir/sbin $basedir/bin
    exit 1
  fi
  langdir=`find_in_basedir --dir errmsg.sys share/english share/mysql/english`
  if test -z "$langdir"
  then
    cannot_find_file errmsg.sys $basedir/share/english $basedir/share/mysql/english
    exit 1
  fi
  pkgdatadir=`find_in_basedir --dir fill_help_tables.sql share share/mysql`
  if test -z "$pkgdatadir"
  then
    cannot_find_file fill_help_tables.sql $basedir/share $basedir/share/mysql
    exit 1
  fi
  scriptdir="$basedir/scripts"
else
  basedir="."
  bindir="./bin"
  extra_bindir="$bindir"
  mysqld="./bin/mysqld"
  pkgdatadir="./share"
  scriptdir="./bin"
fi

也是分三種情況，即指定了srcdir，指定了basedir，或者兩者都沒有指定。

在這裡，說說第二種和第三種情況

若指定了basedir，則會定義三個路徑

1> mysqld，mysqld一般會存放在如下三個路徑中，$basedir/libexec/mysqld，$basedir/sbin/mysqld，$basedir/bin/mysqld

2> langdir，該路徑是errmsg.sys的存放路徑，該文件與mysql的錯誤代碼有關。升級時該文件即需要更新

3> pkgdatadir，該路徑存放mysql庫，performance_schema庫的創建腳本，不僅僅是fill_help_tables.sql。

如果沒有指定srcdir和basedir，則預設將當前路徑設置為basedir。

定義資料庫創建腳本的路徑

在上面路徑確認好的情況下，進一步確認腳本是否存在，mysqld是否有可執行許可權，確認errmsg.sys是否存在

# Set up paths to SQL scripts required for bootstrap
fill_help_tables="$pkgdatadir/fill_help_tables.sql"
create_system_tables="$pkgdatadir/mysql_system_tables.sql"
create_system_tables2="$pkgdatadir/mysql_performance_tables.sql"
fill_system_tables="$pkgdatadir/mysql_system_tables_data.sql"
maria_add_gis_sp="$pkgdatadir/maria_add_gis_sp_bootstrap.sql"

for f in "$fill_help_tables" "$create_system_tables" "$create_system_tables2" "$fill_system_tables" "$maria_add_gis_sp"
do
  if test ! -f "$f"
  then
    cannot_find_file "$f"
    exit 1
  fi
done

if test ! -x "$mysqld"
then
  cannot_find_file "$mysqld"
  exit 1
fi

if test -n "$langdir"
then
  if test ! -f "$langdir/errmsg.sys"
  then
    cannot_find_file "$langdir/errmsg.sys"
    exit 1
  fi
  mysqld_opt="--lc-messages-dir=$langdir/.."
else
  mysqld_opt="--lc-messages=en_US"
fi

確認主機名以及主機名是否有效

它的判斷邏輯是如果--cross-bootstrap，--rpm，--force沒有顯式指定的話，則查看主機名是否能被解析成ip。

首先解析的是主機名，如果沒有解析成功，則解析localhost

# Try to determine the hostname
hostname=`hostname`

# Check if hostname is valid
if test "$cross_bootstrap" -eq 0 -a "$in_rpm" -eq 0 -a "$force" -eq 0
then
  resolved=`"$extra_bindir/resolveip" $hostname 2>&1`
  if test $? -ne 0
  then
    resolved=`"$extra_bindir/resolveip" localhost 2>&1`
    if test $? -ne 0
    then
      echo "Neither host '$hostname' nor 'localhost' could be looked up with"
      echo "'$extra_bindir/resolveip'"
      echo "Please configure the 'hostname' command to return a correct"
      echo "hostname."
      echo "If you want to solve this at a later stage, restart this script"
      echo "with the --force option"
      link_to_help
      exit 1
    fi
    echo "WARNING: The host '$hostname' could not be looked up with resolveip."
    echo "This probably means that your libc libraries are not 100 % compatible"
    echo "with this binary MariaDB version. The MariaDB daemon, mysqld, should work"
    echo "normally with the exception that host name resolving will not work."
    echo "This means that you should use IP addresses instead of hostnames"
    echo "when specifying MariaDB privileges !"
  fi
fi

如果指定了--skip-name-resolve參數，則將主機名解析為IP

if test "$ip_only" -eq 1
then
  hostname=`echo "$resolved" | awk '/ /{print $6}'`
fi

創建數據目錄

# Create database directories
for dir in "$ldata" "$ldata/mysql" "$ldata/test"
do
  if test ! -d "$dir"
  then
    if ! `mkdir -p "$dir"`
    then
      echo "Fatal error Can't create database directory '$dir'"
      link_to_help
      exit 1
    fi
    chmod 700 "$dir"
  fi
  if test -n "$user"
  then
    chown $user "$dir"
    if test $? -ne 0
    then
      echo "Cannot change ownership of the database directories to the '$user'"
      echo "user.  Check that you have the necessary permissions and try again."
      exit 1
    fi
  fi
done

可以看到，會創建三個目錄，datadir，以及datadir下的mysql目錄和test目錄，這個對應mysql庫和test庫。

如果目錄不存在，則創建，並將目錄許可權設置為700，如果指定了--user參數，則將目錄的屬主修改為指定的用戶

如果指定了--user參數，則添加到之前的參數列表中

if test -n "$user"
then
  args="$args --user=$user"
fi

--cross-bootstrap參數是用於跨平臺啟動的，具體可以參考上面parse_arguments函數中對該參數的解釋

如果指定了該參數，則會過濾初始化腳本中有關當前主機名的設置。

# When doing a "cross bootstrap" install, no reference to the current
# host should be added to the system tables.  So we filter out any
# lines which contain the current host name.
if test $cross_bootstrap -eq 1
then
  filter_cmd_line="sed -e '/@current_hostname/d'"
else
  filter_cmd_line="cat"
fi

配置mysqld命令行

感覺MYSQLD_BOOTSTRAP變數出現得莫名其妙，上文中也沒給出任何定義

這樣執行的效果是如果定義了MYSQLD_BOOTSTRAP，則$MYSQLD_BOOTSTRAP值賦給mysqld_bootstrap，如果沒有定義，則$mysqld的值賦給mysqld_bootstrap

# Configure mysqld command line
mysqld_bootstrap="${MYSQLD_BOOTSTRAP-$mysqld}"
mysqld_install_cmd_line()
{
  "$mysqld_bootstrap" $defaults "$mysqld_opt" --bootstrap \
  "--basedir=$basedir" "--datadir=$ldata" --log-warnings=0 --enforce-storage-engine="" \
  $args --max_allowed_packet=8M \
  --net_buffer_length=16K
}

若以如下方式初始化mysql

# /usr/test/mariadb-10.1.16-linux-x86_64/scripts/mysql_install_db --defaults-file=/usr/test/mariadb-10.1.16-linux-x86_64/my.cnf --basedir=/usr/test/mariadb-10.1.16-linux-x86_64/

則上面這個函數相當於

/usr/test/mariadb-10.1.16-linux-x86_64/bin/mysqld --defaults-file=/usr/test/mariadb-10.1.16-linux-x86_64/my.cnf --lc-messages-dir=/usr/test/mariadb-10.1.16-linux-x86_64/share --bootstrap --basedir=/usr/test/mariadb-10.1.16-linux-x86_64/ --datadir=/usr/test/mariadb-10.1.16-linux-x86_64/data --log-warnings=0 --enforce-storage-engine="" --user=mysql --max_allowed_packet=8M --net_buffer_length=16K

執行資料庫初始化腳本

分為三部分

1> 系統表

2> fill_help_tables.sql，該文件用於生成help contents的內容

3> OpenGIS

其中，--bootstrap代表Used by mysql installation scripts。

雖然同樣是執行的mysqld命令，但因為指定了--bootstrap參數，只是執行了資料庫初始化腳本中的命令，並沒有啟動資料庫。

# Create the system and help tables by passing them to "mysqld --bootstrap"
s_echo "Installing MariaDB/MySQL system tables in '$ldata' ..."
if { echo "use mysql;"; cat "$create_system_tables" "$create_system_tables2" "$fill_system_tables"; } | eval "$filter_cmd_line" | mysqld_install_cmd_line > /dev/null
then
  s_echo "OK"
else
  echo
  echo "Installation of system tables failed!  Examine the logs in"
  echo "$ldata for more information."
  echo
  echo "The problem could be conflicting information in an external"
  echo "my.cnf files. You can ignore these by doing:"
  echo
  echo "    shell> $scriptdir/scripts/mysql_install_db --defaults-file=~/.my.cnf"
  echo
  echo "You can also try to start the mysqld daemon with:"
  echo
  echo "    shell> $mysqld --skip-grant --general-log &"
  echo
  echo "and use the command line tool $bindir/mysql"
  echo "to connect to the mysql database and look at the grant tables:"
  echo
  echo "    shell> $bindir/mysql -u root mysql"
  echo "    mysql> show tables;"
  echo
  echo "Try 'mysqld --help' if you have problems with paths.  Using"
  echo "--general-log gives you a log in $ldata that may be helpful."
  link_to_help
  echo "MariaDB is hosted on launchpad; You can find the latest source and"
  echo "email lists at http://launchpad.net/maria"
  echo
  echo "Please check all of the above before submitting a bug report"
  echo "at http://mariadb.org/jira"
  echo
  exit 1
fi

s_echo "Filling help tables..."
if { echo "use mysql;"; cat "$fill_help_tables"; } | mysqld_install_cmd_line > /dev/null
then
  s_echo "OK"
else
  echo
  echo "WARNING: HELP FILES ARE NOT COMPLETELY INSTALLED!"
  echo "The \"HELP\" command might not work properly."
fi

s_echo "Creating OpenGIS required SP-s..."
if { echo "use test;"; cat "$maria_add_gis_sp"; } | mysqld_install_cmd_line > /dev/null
then
  s_echo "OK"
else
  echo
  echo "WARNING: OPENGIS REQUIRED SP-S WERE NOT COMPLETELY INSTALLED!"
  echo "GIS extentions might not work properly."
fi

輸出相關信息

針對的是--cross-bootstrap，--srcdir，--rpm這三個參數。

# Don't output verbose information if running inside bootstrap or using
# --srcdir for testing.  In such cases, there's no end user looking at
# the screen.
if test "$cross_bootstrap" -eq 0 && test -z "$srcdir"
then
  s_echo
  s_echo "To start mysqld at boot time you have to copy"
  s_echo "support-files/mysql.server to the right place for your system"

  echo
  echo "PLEASE REMEMBER TO SET A PASSWORD FOR THE MariaDB root USER !"
  echo "To do so, start the server, then issue the following commands:"
  echo
  echo "'$bindir/mysqladmin' -u root password 'new-password'"
  echo "'$bindir/mysqladmin' -u root -h $hostname password 'new-password'"
  echo
  echo "Alternatively you can run:"
  echo "'$bindir/mysql_secure_installation'"
  echo