利用mysqlbinlog_flashback閃回丟失數據

来源:https://www.cnblogs.com/aaron8219/archive/2018/06/08/9157585.html
-Advertisement-
Play Games

today,i'll have a test with the open source tool mysqlbinlog_flashback which is released by 58daojia,here's the github address:https://github.com/58da ...


        today,i'll have a test with the open source tool mysqlbinlog_flashback which is released by 58daojia,here's the github address:https://github.com/58daojia-dba/mysqlbinlog_flashback  this tool is similar with the binlog2sql which is also coded by python.let's download and unzip the tool:

 

 1 [root@zlm1 15:42:35 /vagrant]
 2 #cd mysqlbinlog_flashback-master
 3 
 4 [root@zlm1 15:42:46 /vagrant/mysqlbinlog_flashback-master]
 5 #ls -l
 6 total 70
 7 -rwxrwxrwx 1 vagrant vagrant  4766 Dec 19  2016 binlogstream.py.diff
 8 -rwxrwxrwx 1 vagrant vagrant   524 Dec 19  2016 CHANGELOG.txt
 9 -rwxrwxrwx 1 vagrant vagrant  1365 Dec 19  2016 constant.py
10 -rwxrwxrwx 1 vagrant vagrant 13730 Dec 19  2016 flashback.py
11 -rwxrwxrwx 1 vagrant vagrant  1800 Dec 19  2016 func.py
12 drwxrwxrwx 1 vagrant vagrant     0 Jun  8 15:41 internal
13 -rwxrwxrwx 1 vagrant vagrant 10015 Dec 19  2016 joint_sql.py
14 -rwxrwxrwx 1 vagrant vagrant 11357 Dec 19  2016 LICENSE
15 drwxrwxrwx 1 vagrant vagrant     0 Jun  8 15:41 log
16 -rwxrwxrwx 1 vagrant vagrant  9192 Dec 19  2016 mysqlbinlog_back.py
17 -rwxrwxrwx 1 vagrant vagrant  2782 Dec 19  2016 mysql_table.py
18 drwxrwxrwx 1 vagrant vagrant  4096 Jun  8 15:41 pymysqlreplication
19 -rwxrwxrwx 1 vagrant vagrant  5033 Dec 19  2016 README.md
20 drwxrwxrwx 1 vagrant vagrant  4096 Jun  8 15:41 test
21 
22 [root@zlm1 15:42:47 /vagrant/mysqlbinlog_flashback-master]
23 #python mysqlbinlog_back.py --help
24 Traceback (most recent call last):
25   File "mysqlbinlog_back.py", line 12, in <module>
26     from flashback import Parameter,deal_all_event,generate_create_table,convert_datetime_to_timestamp
27   File "/vagrant/mysqlbinlog_flashback-master/flashback.py", line 13, in <module>
28     from pymysqlreplication import BinLogStreamReader
29   File "/vagrant/mysqlbinlog_flashback-master/pymysqlreplication/__init__.py", line 23, in <module>
30     from .binlogstream import BinLogStreamReader
31   File "/vagrant/mysqlbinlog_flashback-master/pymysqlreplication/binlogstream.py", line 3, in <module>
32     import pymysql
33 ImportError: No module named pymysql
34 
35 [root@zlm1 15:43:06 /vagrant/mysqlbinlog_flashback-master]
36 #python --version
37 Python 2.7.5
38 
39 [root@zlm1 16:02:51 /vagrant/mysqlbinlog_flashback-master]
40 #

 

    the tool need python 2.6,mine is 2.7.5,i'll go to the python official website to download it,Python 2.6.9 is the final version of 2.6.x since Oct.29 2013.here's the address:https://www.python.org/download/releases/2.6.9/,below is the directory structure in Python 2.6.9:

 

 1 [root@zlm1 16:29:12 ~/Python-2.6.9]
 2 #ls -l
 3 total 788
 4 -rwxr-xr-x  1 root root 380641 Oct 29  2013 configure
 5 -rw-r--r--  1 root root 112213 Oct 29  2013 configure.in
 6 drwxrwxr-x 23 root root   4096 Oct 29  2013 Demo
 7 drwxrwxr-x 17 root root   4096 Oct 29  2013 Doc
 8 drwxrwxr-x  2 root root     20 Oct 29  2013 Grammar
 9 drwxrwxr-x  2 root root   4096 Oct 29  2013 Include
10 -rwxr-xr-x  1 root root   7122 Oct 29  2013 install-sh
11 drwxrwxr-x 43 root root   8192 Oct 29  2013 Lib
12 -rw-r--r--  1 root root  14411 Oct 29  2013 LICENSE
13 drwxrwxr-x 11 root root   4096 Oct 29  2013 Mac
14 -rw-r--r--  1 root root  39341 Oct 29  2013 Makefile.pre.in
15 drwxrwxr-x  5 root root   4096 Oct 29  2013 Misc
16 drwxrwxr-x  8 root root   4096 Oct 29  2013 Modules
17 drwxrwxr-x  3 root root   4096 Oct 29  2013 Objects
18 drwxrwxr-x  2 root root   4096 Oct 29  2013 Parser
19 drwxrwxr-x 10 root root   4096 Oct 29  2013 PC
20 drwxrwxr-x  2 root root   4096 Oct 29  2013 PCbuild
21 -rw-r--r--  1 root root  29620 Oct 29  2013 pyconfig.h.in
22 drwxrwxr-x  2 root root   4096 Oct 29  2013 Python
23 -rw-r--r--  1 root root  55033 Oct 29  2013 README
24 drwxrwxr-x  5 root root    142 Oct 29  2013 RISCOS
25 -rw-r--r--  1 root root  87576 Oct 29  2013 setup.py
26 drwxrwxr-x 20 root root   4096 Oct 29  2013 Tools
27 
28 [root@zlm1 16:29:13 ~/Python-2.6.9]
29 #./configure && make && make install
30 ... -- Omitted
31  [root@zlm1 16:38:41 ~/Python-2.6.9]
32 #python --version
33 Python 2.6.9
34 
35 [root@zlm1 16:38:49 ~/Python-2.6.9]
36 #[root@zlm1 16:35:40 ~/mysqlbinlog_flashback-master]
37 #pwd
38 /root/mysqlbinlog_flashback-master
39 
40 [root@zlm1 16:39:21 ~/mysqlbinlog_flashback-master]
41 #python mysqlbinlog_back.py --help
42 Traceback (most recent call last):
43   File "mysqlbinlog_back.py", line 12, in <module>
44     from flashback import Parameter,deal_all_event,generate_create_table,convert_datetime_to_timestamp
45   File "/root/mysqlbinlog_flashback-master/flashback.py", line 13, in <module>
46     from pymysqlreplication import BinLogStreamReader
47   File "/root/mysqlbinlog_flashback-master/pymysqlreplication/__init__.py", line 23, in <module>
48     from .binlogstream import BinLogStreamReader
49   File "/root/mysqlbinlog_flashback-master/pymysqlreplication/binlogstream.py", line 3, in <module>
50     import pymysql
51 ImportError: No module named pymysql
52 
53 [root@zlm1 16:39:38 ~/mysqlbinlog_flashback-master]
54 #

 

 it still appear the identical issue,what's wrong?once more i check the readme.md it says besides the python version,"pymysql" module is also necessary.

 

1 [root@zlm1 16:59:16 ~/pip-10.0.1]
2 #pip install pymysql
3 Traceback (most recent call last):
4   File "/usr/bin/pip", line 5, in <module>
5     from pkg_resources import load_entry_point
6 ImportError: No module named pkg_resources
7 
8 [root@zlm1 16:59:33 ~/pip-10.0.1]
9 #

 

  1 [root@zlm1 17:10:16 ~/PyMySQL-0.8.1]
  2 #ls -l
  3 total 52
  4 -rw-r--r-- 1 501 games 8006 May  7 12:22 CHANGELOG
  5 -rw-r--r-- 1 501 games  320 May 18  2016 example.py
  6 -rw-r--r-- 1 501 games 1070 Nov 27  2013 LICENSE
  7 -rw-r--r-- 1 501 games   84 Jan 14  2015 MANIFEST.in
  8 -rw-r--r-- 1 501 games 6051 May  7 12:27 PKG-INFO
  9 drwxr-xr-x 4 501 games 4096 May  7 12:27 pymysql
 10 drwxr-xr-x 2 501 games  101 May  7 12:27 PyMySQL.egg-info
 11 -rw-r--r-- 1 501 games 3879 May  7 12:10 README.rst
 12 -rwxr-xr-x 1 501 games  704 Aug 29  2016 runtests.py
 13 -rw-r--r-- 1 501 games  145 May  7 12:27 setup.cfg
 14 -rwxr-xr-x 1 501 games 1491 May  7 12:26 setup.py
 15 -rw-r--r-- 1 501 games  184 Jan 13  2017 tox.ini
 16 
 17 [root@zlm1 17:10:17 ~/PyMySQL-0.8.1]
 18 #python setup.py install
 19 Traceback (most recent call last):
 20   File "setup.py", line 3, in <module>
 21     from setuptools import setup, find_packages
 22 ImportError: No module named setuptools
 23 
 24 [root@zlm1 17:10:32 ~/PyMySQL-0.8.1]
 25 #chown root.root -R *
 26 
 27 [root@zlm1 17:11:23 ~/PyMySQL-0.8.1]
 28 #ls -l
 29 total 52
 30 -rw-r--r-- 1 root root 8006 May  7 12:22 CHANGELOG
 31 -rw-r--r-- 1 root root  320 May 18  2016 example.py
 32 -rw-r--r-- 1 root root 1070 Nov 27  2013 LICENSE
 33 -rw-r--r-- 1 root root   84 Jan 14  2015 MANIFEST.in
 34 -rw-r--r-- 1 root root 6051 May  7 12:27 PKG-INFO
 35 drwxr-xr-x 4 root root 4096 May  7 12:27 pymysql
 36 drwxr-xr-x 2 root root  101 May  7 12:27 PyMySQL.egg-info
 37 -rw-r--r-- 1 root root 3879 May  7 12:10 README.rst
 38 -rwxr-xr-x 1 root root  704 Aug 29  2016 runtests.py
 39 -rw-r--r-- 1 root root  145 May  7 12:27 setup.cfg
 40 -rwxr-xr-x 1 root root 1491 May  7 12:26 setup.py
 41 -rw-r--r-- 1 root root  184 Jan 13  2017 tox.ini
 42 
 43 [root@zlm1 17:11:25 ~/PyMySQL-0.8.1]
 44 #python setup.py install
 45 Traceback (most recent call last):
 46   File "setup.py", line 3, in <module>
 47     from setuptools import setup, find_packages
 48 ImportError: No module named setuptools  -- It seems another module name "setuptools" is needed.
 49 
 50 [root@zlm1 17:22:25 ~/setuptools-39.2.0]
 51 #ls -l
 52 total 192
 53 -rwxrwxrwx 1 root root   1669 May 19 13:18 bootstrap.py
 54 -rwxrwxrwx 1 root root 113612 May 19 13:18 CHANGES.rst
 55 -rwxrwxrwx 1 root root    223 May 19 13:18 conftest.py
 56 drwxrwxrwx 4 root root   4096 Jun  8 17:18 docs
 57 -rwxrwxrwx 1 root root    126 May 19 13:18 easy_install.py
 58 -rwxrwxrwx 1 root root  10317 May 19 13:18 launcher.c
 59 -rwxrwxrwx 1 root root   1078 May 19 13:18 LICENSE
 60 -rwxrwxrwx 1 root root    430 May 19 13:18 MANIFEST.in
 61 -rwxrwxrwx 1 root root   1640 May 19 13:18 msvc-build-launcher.cmd
 62 -rwxrwxrwx 1 root root   1438 May 19 13:18 pavement.py
 63 -rwxrwxrwx 1 root root   3160 May 19 13:19 PKG-INFO
 64 drwxrwxrwx 5 root root    103 Jun  8 17:18 pkg_resources
 65 -rwxrwxrwx 1 root root    493 May 19 13:18 pytest.ini
 66 -rwxrwxrwx 1 root root   1545 May 19 13:18 README.rst
 67 -rwxrwxrwx 1 root root    434 May 19 13:19 setup.cfg
 68 -rwxrwxrwx 1 root root   7365 May 19 13:18 setup.py
 69 drwxrwxrwx 6 root root   4096 Jun  8 17:18 setuptools
 70 drwxrwxrwx 2 root root    143 Jun  8 17:18 setuptools.egg-info
 71 drwxrwxrwx 2 root root     46 Jun  8 17:18 tests
 72 -rwxrwxrwx 1 root root    662 May 19 13:18 towncrier_template.rst
 73 -rwxrwxrwx 1 root root   1391 May 19 13:18 tox.ini
 74 
 75 [root@zlm1 17:22:26 ~/setuptools-39.2.0]
 76 #python setup.py install
 77 running install
 78 running bdist_egg
 79 running egg_info
 80 writing requirements to setuptools.egg-info/requires.txt
 81 writing setuptools.egg-info/PKG-INFO
 82 writing top-level names to setuptools.egg-info/top_level.txt
 83 writing dependency_links to setuptools.egg-info/dependency_links.txt
 84 writing entry points to setuptools.egg-info/entry_points.txt
 85 reading manifest file 'setuptools.egg-info/SOURCES.txt'
 86 reading manifest template 'MANIFEST.in'
 87 writing manifest file 'setuptools.egg-info/SOURCES.txt'
 88 installing library code to build/bdist.linux-x86_64/egg
 89 running install_lib
 90 running build_py
 91 creating build
 92 creating build/lib
 93 ... -- Omitted.
 94  
 95 root@zlm1 17:25:21 ~/PyMySQL-0.8.1]
 96 #python setup.py install
 97 /usr/lib64/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'project_urls'
 98   warnings.warn(msg)
 99 running install
100 running bdist_egg
101 running egg_info
102 writing PyMySQL.egg-info/PKG-INFO
103 writing top-level names to PyMySQL.egg-info/top_level.txt
104 writing dependency_links to PyMySQL.egg-info/dependency_links.txt
105 reading manifest file 'PyMySQL.egg-info/SOURCES.txt'
106 reading manifest template 'MANIFEST.in'
107 writing manifest file 'PyMySQL.egg-info/SOURCES.txt'
108 installing library code to build/bdist.linux-x86_64/egg
109 running install_lib
110 running build_py
111 creating build
112 creating build/lib
113 creating build/lib/pymysql
114 ... -- Omitted.
115  
116 --Finally the PyMySQL was installed successfully.

 

 1 [root@zlm1 17:31:37 ~/mysqlbinlog_flashback-master]
 2 #python mysqlbinlog_back.py --help
 3 ===log will also  write to .//mysqlbinlog_flashback.log===
 4 Usage: python mysqlbinlog_back.py [options]
 5 sample1:python  mysqlbinlog_back.py --host="127.0.0.1" --username="root" --port=43306 --password="" --schema=test --table="test5"
 6 sample2:python mysqlbinlog_back.py --host="127.0.0.1" --username="root" --port=43306 --password="" --schema=test --table="test5,test6" --binlog_end_time="2016-11-05 11:27:13" --binlog_start_file_name="mysql-bin.000024"  --binlog_start_file_position=4 --binlog_start_time="2016-11-04 11:27:13"  --skip_delete  --skip_insert --add_schema_name
 7 sample3:python mysqlbinlog_back.py  --host="127.0.0.1" --username="root" --port=43306 --password="" --schema=test --table="test5,test6" --binlog_start_file_name="mysql-bin.000022"
 8 
 9 Options:
10   -h, --help            show this help message and exit
11   -H HOST, --host=HOST  mandatory,mysql hostname
12   -P PORT, --port=PORT  mysql port,default 3306
13   -u USERNAME, --username=USERNAME
14                         mandatory,username
15   -p PASSWORD, --password=PASSWORD
16                         password
17   -s SCHEMA, --schema=SCHEMA
18                         mandatory,mysql schema
19   -t TABLES, --tables=TABLES
20                         mandatory,mysql tables,suport multiple tables,use
21                         comma as separator
22   -N BINLOG_END_TIME, --binlog_end_time=BINLOG_END_TIME
23                         binlog end time,format yyyy-mm-dd hh24:mi:ss,default
24                         is current time
25   -S BINLOG_START_FILE_NAME, --binlog_start_file_name=BINLOG_START_FILE_NAME
26                         binlog start file name,default is current logfile of
27                         db
28   -L BINLOG_START_FILE_POSITION, --binlog_start_file_position=BINLOG_START_FILE_POSITION
29                         binlog start file name
30   -E BINLOG_START_TIME, --binlog_start_time=BINLOG_START_TIME
31                         binlog start time,format yyyy-mm-dd hh24:mi:ss
32   -l OUTPUT_FILE_PATH, --output_file_path=OUTPUT_FILE_PATH
33                         file path that sql generated,,default ./log
34   -I, --skip_insert     skip insert(WriteRowsEvent) event
35   -U, --skip_update     skip update(UpdateRowsEvent) event
36   -D, --skip_delete     skip delete(DeleteRowsEvent) event
37   -a, --add_schema_name
38                         add schema name for flashback sql
39   -v, --version         version info
View Code

 

    okay,now the tool mysqlbinlog_flashback can te used normally,let's begin our data-loss flashback test:

 

  1 root@localhost:mysql3306.sock [zlm]05:39:39>create table test(
  2     -> id bigint not null auto_increment,
  3     -> name varchar(20) not null default '',
  4     -> primary key(id)
  5     -> ) engine=innodb default charset=utf8mb4;
  6 Query OK, 0 rows affected (0.05 sec)
  7 
  8 root@localhost:mysql3306.sock [zlm]05:39:45>delimiter //
  9 root@localhost:mysql3306.sock [zlm]05:39:51>create procedure pro_insert()
 10     -> begin
 11     -> declare id int;
 12     -> set id = 100000;
 13     -> while id>0 do
 14     -> insert into test(name) values ('aaron8219');  
 15     -> set id=id-1;               
 16     -> end while;
 17     -> end //
 18 Query OK, 0 rows affected (0.05 sec)
 19 
 20 root@localhost:mysql3306.sock [zlm]05:39:51>delimiter ;
 21 root@localhost:mysql3306.sock [zlm]05:39:53>call pro_insert();
 22 Query OK, 1 row affected (8.87 sec)
 23 
 24 root@localhost:mysql3306.sock [zlm]05:40:40>select count(*) from test;
 25 +----------+
 26 | count(*) |
 27 +----------+
 28 |   100000 |
 29 +----------+
 30 1 row in set (0.03 sec)
 31 
 32 root@localhost:mysql3306.sock [zlm]05:41:02>flush logs;
 33 Query OK, 0 rows affected (0.12 sec)
 34 
 35 root@localhost:mysql3306.sock [zlm]05:42:03>show binary log;
 36 ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 

您的分享是我們最大的動力!

-Advertisement-
Play Games
更多相關文章
  • 最近在做項目過程中,經常會遇到生成訂單流水號唯一性的應用,也有不少同事也請教我對唯一性的 "流水號"的問題,根據個人所見,一般生成的規則都是根據日期來進行操作,我目前在做的一個項目:”報告卡“上的編碼就是根據 ”yyyyMMdd+四位數字(累加)“的格式生成,一般對這一類的操作最好是在存儲過程中生成 ...
  • 使用Vim編輯器寫腳本時,經常會遇到多重迴圈語句,習慣上會用tab鍵來補齊。這時設置tab鍵占用的長度,可以調節界面的鬆緊度,使其達到令人滿意的效果。 在針對個別用戶和所有用戶來設置時,與編輯SSH相應的配置文件十分類似。 一.設置當前用戶的Tab鍵的長度 編輯配置文件~/.vimrc即可 保存並退 ...
  • 1. 使用系統和軟體 2. 必要的包安裝 3. 安裝Jdk【java8】 3.1. 軟體安裝 3.2. 環境變數 4. 安裝maven 4.1. 軟體安裝 4.2. 環境變數 5. 安裝ant 5.1. 軟體安裝 5.2. 環境變數 6. 安裝findbugs 6.1. 軟體安裝 6.2. 環境變數 ...
  • 網盤下載:http://tadown.com/fs/9yi0be9nschue6b67/ ...
  • load average 的含義平均負載(load average)是指系統的運行隊列的平均利用率,也可以認為是可運行進程的平均數。 top命令中load average顯示的是最近1分鐘、5分鐘和15分鐘的系統平均負載。系統平均負載表示 系統平均負載被定義為在特定時間間隔內運行隊列中(在CPU上運 ...
  • linux 查看系統當前時間,修改時間1. 查看時間和日期命令 : "date"2.設置時間和日期例如:將系統日期設定成2018年6月8日的命令命令 : "date -s 06/08/2018"將系統時間設定成下午10點46分03秒的命令命令 : "date -s 10:46:03" ...
  • 一、列表 list 列表是有序的,列表元素可以被修改 list之灰魔法 1、列表格式: 中括弧括起來, 逗號分隔每個元素, 列表中的元素可以是數字、字元串、列表、布爾值、等所有的都能放進去 “集合“,”內部可以放置任何東西 li=['alex', '小名', [12, 23, 45], 23, 23 ...
  • RDD是spark的核心,先感性的先認識RDD,大體上對RDD進行了分類操作 ...
一周排行
    -Advertisement-
    Play Games
  • 移動開發(一):使用.NET MAUI開發第一個安卓APP 對於工作多年的C#程式員來說,近來想嘗試開發一款安卓APP,考慮了很久最終選擇使用.NET MAUI這個微軟官方的框架來嘗試體驗開發安卓APP,畢竟是使用Visual Studio開發工具,使用起來也比較的順手,結合微軟官方的教程進行了安卓 ...
  • 前言 QuestPDF 是一個開源 .NET 庫,用於生成 PDF 文檔。使用了C# Fluent API方式可簡化開發、減少錯誤並提高工作效率。利用它可以輕鬆生成 PDF 報告、發票、導出文件等。 項目介紹 QuestPDF 是一個革命性的開源 .NET 庫,它徹底改變了我們生成 PDF 文檔的方 ...
  • 項目地址 項目後端地址: https://github.com/ZyPLJ/ZYTteeHole 項目前端頁面地址: ZyPLJ/TreeHoleVue (github.com) https://github.com/ZyPLJ/TreeHoleVue 目前項目測試訪問地址: http://tree ...
  • 話不多說,直接開乾 一.下載 1.官方鏈接下載: https://www.microsoft.com/zh-cn/sql-server/sql-server-downloads 2.在下載目錄中找到下麵這個小的安裝包 SQL2022-SSEI-Dev.exe,運行開始下載SQL server; 二. ...
  • 前言 隨著物聯網(IoT)技術的迅猛發展,MQTT(消息隊列遙測傳輸)協議憑藉其輕量級和高效性,已成為眾多物聯網應用的首選通信標準。 MQTTnet 作為一個高性能的 .NET 開源庫,為 .NET 平臺上的 MQTT 客戶端與伺服器開發提供了強大的支持。 本文將全面介紹 MQTTnet 的核心功能 ...
  • Serilog支持多種接收器用於日誌存儲,增強器用於添加屬性,LogContext管理動態屬性,支持多種輸出格式包括純文本、JSON及ExpressionTemplate。還提供了自定義格式化選項,適用於不同需求。 ...
  • 目錄簡介獲取 HTML 文檔解析 HTML 文檔測試參考文章 簡介 動態內容網站使用 JavaScript 腳本動態檢索和渲染數據,爬取信息時需要模擬瀏覽器行為,否則獲取到的源碼基本是空的。 本文使用的爬取步驟如下: 使用 Selenium 獲取渲染後的 HTML 文檔 使用 HtmlAgility ...
  • 1.前言 什麼是熱更新 游戲或者軟體更新時,無需重新下載客戶端進行安裝,而是在應用程式啟動的情況下,在內部進行資源或者代碼更新 Unity目前常用熱更新解決方案 HybridCLR,Xlua,ILRuntime等 Unity目前常用資源管理解決方案 AssetBundles,Addressable, ...
  • 本文章主要是在C# ASP.NET Core Web API框架實現向手機發送驗證碼簡訊功能。這裡我選擇是一個互億無線簡訊驗證碼平臺,其實像阿裡雲,騰訊雲上面也可以。 首先我們先去 互億無線 https://www.ihuyi.com/api/sms.html 去註冊一個賬號 註冊完成賬號後,它會送 ...
  • 通過以下方式可以高效,並保證數據同步的可靠性 1.API設計 使用RESTful設計,確保API端點明確,並使用適當的HTTP方法(如POST用於創建,PUT用於更新)。 設計清晰的請求和響應模型,以確保客戶端能夠理解預期格式。 2.數據驗證 在伺服器端進行嚴格的數據驗證,確保接收到的數據符合預期格 ...