安裝inotify [root@server ~]# mkdir -p /home/oldboy/tools 安裝inotify-tools-3.14.tar.gz [root@server tools]# ls -l /proc/sys/fs/inotify/ #出現下麵三個表示支持inotify... ...
安裝inotify
- [root@server ~]# mkdir -p /home/oldboy/tools
安裝inotify-tools-3.14.tar.gz
- [root@server tools]# ls -l /proc/sys/fs/inotify/ #出現下麵三個表示支持inotify
- total 0
- -rw-r--r-- 1 root root 0 Feb 6 15:36 max_queued_events
- -rw-r--r-- 1 root root 0 Feb 6 15:36 max_user_instances
- -rw-r--r-- 1 root root 0 Feb 6 15:36 max_user_watches
解壓:
- [root@server tools]# tar zxf inotify-tools-3.14.tar.gz
編譯安裝:
- [root@server tools]# cd inotify-tools-3.14
- [root@server inotify-tools-3.14]# ./configure --prefix=/usr/local/inotify-tools-3.14
- [root@server inotify-tools-3.14]# echo $?
- 0
- [root@server inotify-tools-3.14]# make && make install
- [root@server inotify-tools-3.14]# echo $? #0表示安裝正確
- 0
- [root@server tools]# ln -s /usr/local/inotify-tools-3.14/ /usr/local/inotify-tools #創建軟連接
- [root@server tools]# ls -l /usr/local/inotify-tools/
- total 16
- drwxr-xr-x. 2 root root 4096 Feb 6 15:43 bin #inotify執行命令(二進位)
- drwxr-xr-x. 3 root root 4096 Feb 6 15:42 include #inotify程式所需要的頭文件
- drwxr-xr-x. 2 root root 4096 Feb 6 15:43 lib #動態鏈接的庫文件
- drwxr-xr-x. 4 root root 4096 Feb 6 15:42 share #幫助文檔
- [root@server inotify-tools]# ll bin/
- total 88
- -rwxr-xr-x. 1 root root 44287 Feb 6 15:43 inotifywait
- -rwxr-xr-x. 1 root root 41409 Feb 6 15:43 inotifywatch
inotifywait:在被監控的文件或目錄上等待特定文件系統事件(open、close、delete等)發生,執行後處於阻塞轉態,適合在shell腳本中使用。
inotifywatch:收集被監控的文件系統使用度統計數據,指文件系統事件發生的次數統計。
inotifywait命令常用參數
- [root@server inotify-tools]# ./bin/inotifywait --help
- inotifywait 3.14
- Wait for a particular event on a file or set of files.
- Usage: inotifywait [ options ] file1 [ file2 ] [ file3 ] [ ... ]
- Options:
- -h|--help Show this help text.
- @<file> Exclude the specified file from being watched.
- --exclude <pattern>
- Exclude all events on files matching the
- extended regular expression <pattern>.
- --excludei <pattern>
- Like --exclude but case insensitive. #排除文件或目錄時,不區分大小寫
- -m|--monitor Keep listening for events forever. Without
- this option, inotifywait will exit after one
- event is received. #始終保持事件監聽狀態
- -d|--daemon Same as --monitor, except run in the background
- logging events to a file specified by --outfile.
- Implies --syslog.
- -r|--recursive Watch directories recursively. #遞歸查詢目錄
- --fromfile <file>
- Read files to watch from <file> or `-' for stdin.
- -o|--outfile <file>
- Print events to <file> rather than stdout.
- -s|--syslog Send errors to syslog rather than stderr.
- -q|--quiet Print less (only print events). #列印很少的信息,僅僅列印監控事件的信息
- -qq Print nothing (not even events).
- --format <fmt> Print using a specified printf-like format
- string; read the man page for more details.
- --timefmt <fmt> strftime-compatible format string for use with
- %T in --format string. #指定事件輸出的格式
- -c|--csv Print events in CSV format.
- -t|--timeout <seconds>
- When listening for a single event, time out after
- waiting for an event for <seconds> seconds.
- If <seconds> is 0, inotifywait will never time out.
- -e|--event <event1> [ -e|--event <event2> ... ]
- Listen for specific event(s). If omitted, all events are
- listened for. #監控事件
- Exit status:
- 0 - An event you asked to watch for was received.
- 1 - An event you did not ask to watch for was received
- (usually delete_self or unmount), or some error occurred.
- 2 - The --timeout option was given and no events occurred
- in the specified interval of time.
- Events:
- access file or directory contents were read
- modify file or directory contents were written
- attrib file or directory attributes changed
- close_write file or directory closed, after being opened in
- writeable mode
- close_nowrite file or directory closed, after being opened in
- read-only mode
- close file or directory closed, regardless of read/write mode
- open file or directory opened
- moved_to file or directory moved to watched directory
- moved_from file or directory moved from watched directory
- move file or directory moved to or from watched directory
- create file or directory created within watched directory
- delete file or directory deleted within watched directory
- delete_self file or directory was deleted
- unmount file system containing file or directory unmounted
測試一:
視窗一:
- [root@server inotify-tools]# /usr/local/inotify-tools/bin/inotifywait -mrq --timefmt '%d/%m/%y %H:%M' --format '%T %w%f' -e create /data #create創建(監控創建事件)
- 06/02/17 16:13 /data/test.txt
- 06/02/17 16:13 /data/test2.txt
- 06/02/17 16:14 /data/1.txt
- 06/02/17 16:14 /data/2.txt
- 06/02/17 16:14 /data/3.txt
- 06/02/17 16:14 /data/4.txt
視窗二:
- [root@server ~]# cd /data/
- [root@server data]# ls
- [root@server data]# touch test.txt
- [root@server data]# touch test2.txt
- [root@server data]# touch {1..4}.txt
測試二:
視窗一:
- [root@server inotify-tools]# /usr/local/inotify-tools/bin/inotifywait -mrq --timefmt '%d/%m/%y %H:%M' --format '%T %w%f' -e delete /data #delete刪除(監控刪除事件)
- 06/02/17 16:17 /data/test.txt
- 06/02/17 16:17 /data/test2.txt
視窗二:
- [root@server data]# rm -f test.txt
- [root@server data]# rm -f test2.txt
同時監控多個事件,事件之間用逗號分隔。
- /usr/local/inotify-tools/bin/inotifywait -mrq --timefmt '%d/%m/%y %H:%M' --format '%T %w%f' -e delete,create,close_write /data
測試:
視窗一:
- [root@server scripts]# cat inotify.sh
- #!/bin/bash
- inotify=/usr/local/inotify-tools/bin/inotifywait
- $inotify -mrq --format '%w%f' -e create,close_write,delete /data \
- |while read file
- do
- cd / &&
- rsync -az ./data --delete [email protected]::backup/ \
- --password-file=/etc/rsync.password
- done
- [root@server scripts]# sh -x inotify.sh
- + inotify=/usr/local/inotify-tools/bin/inotifywait
- + read file
- + /usr/local/inotify-tools/bin/inotifywait -mrq --format %w%f -e create,close_write,delete /data
- + cd /
- + rsync -az ./data --delete [email protected]::backup/ --password-file=/etc/rsync.password
- + read file
- + cd /
- + rsync -az ./data --delete [email protected]::backup/ --password-file=/etc/rsync.password
- + read file
視窗二:
- [root@server data]# touch oldboy.txt
- [root@server data]# touch test.log
備份伺服器:
- [root@backup backup]# ls
- data
- [root@backup backup]# tree
- .
- └── data
- ├── 2.txt
- ├── 3.txt
- ├── 4.txt
- ├── oldboy.txt
- ├── test.log
- └── test.txt
- [root@server scripts]# /bin/sh /server/scripts/inotify.sh & #放入後臺執行
寫入rc.local
應用場景:10—300k小文件併發200—300,不會有延遲。
關鍵參數說明:
在/proc/sys/fs/inotify目錄下有三個文件,對inotify機制有一定的限制。
max_user_watches:設置inotifywait或inotifywatch命令可以監視的文件數量(單進程)。
max_user_instances:設置每個用戶可以運行的inotifywait或inotifywatch命令的進程數。
max_queued_events:設置inotify實例事件(event)隊列可容納的事件數量。
- [root@server scripts]# cat /proc/sys/fs/inotify/max_user_watches # 修改為50000000
- 8192
- [root@server scripts]# cat /proc/sys/fs/inotify/max_user_instances #
- 128
- [root@server scripts]# cat /proc/sys/fs/inotify/max_queued_events #修改為50000000
- 16384
每秒200個文件併發,數據同步幾乎無延遲。
inotify優點:
實時數據同步。
inotify缺點:
1、併發如果大於200個文件(10—100K),同步會有延遲。
2、監控到事件後,調用rsync同步是單進程的(加&併發),sersync多進程同步。
sersync的功能:
1、配置文件。
2、真正的守護進程socker。
3、可以對失敗文件定時重傳(定時任務)。
4、第三方的http介面。
5、預設多線程同步。
高併發數據實時同步方案:
1、文件級別:inotify(sersync)+rsync。
2、文件系統級別:drbd。
3、第三方軟體的同步功能:mysql同步、oracle、mongodb。
4、程式雙寫。
5、業務邏輯解決。