(以下情況僅針對StoneDB 1.0版本不支持的部分DML和DDL操作,StoneDB 2.0及以上版本將無需此類操作) 主從複製中,主庫的任何更新都會同步到從庫,如果從庫不想重做主庫的某個更新動作,可以使用以下兩種方法進行規避。當然,最終帶來的影響是主從環境數據不一致的問題。 以下的測試環境中, ...
(以下情況僅針對StoneDB 1.0版本不支持的部分DML和DDL操作,StoneDB 2.0及以上版本將無需此類操作)
主從複製中,主庫的任何更新都會同步到從庫,如果從庫不想重做主庫的某個更新動作,可以使用以下兩種方法進行規避。當然,最終帶來的影響是主從環境數據不一致的問題。
以下的測試環境中,主庫是 InnoDB,從庫是 StoneDB,在主庫做從庫不支持的 DML 或者 DDL。
從庫執行 GTID 的空事務
###主庫
mysql> show create table ttt\G
*************************** 1. row ***************************
Table: ttt
Create Table: CREATE TABLE `ttt` (
`id` int(11) NOT NULL,
`name` varchar(5) DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4
1 row in set (0.00 sec)
mysql> select * from ttt;
+----+------+
| id | name |
+----+------+
| 1 | AAA |
| 2 | BBB |
| 3 | CCC |
+----+------+
3 rows in set (0.00 sec)
###從庫
mysql> show create table ttt\G
*************************** 1. row ***************************
Table: ttt
Create Table: CREATE TABLE `ttt` (
`id` int(11) NOT NULL,
`name` varchar(5) DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=STONEDB DEFAULT CHARSET=utf8mb4
1 row in set (0.02 sec)
mysql> select * from ttt;
+----+------+
| id | name |
+----+------+
| 1 | AAA |
| 2 | BBB |
| 3 | CCC |
+----+------+
3 rows in set (0.00 sec)
###主庫
mysql> delete from ttt where id=3;
Query OK, 1 row affected (0.00 sec)
mysql> select * from ttt;
+----+------+
| id | name |
+----+------+
| 1 | AAA |
| 2 | BBB |
+----+------+
2 rows in set (0.00 sec)
###從庫
mysql> show slave status\G
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: 192.168.30.101
Master_User: u_repl
Master_Port: 33306
Connect_Retry: 60
Master_Log_File: binlog.000002
Read_Master_Log_Pos: 1053
Relay_Log_File: ub01-relay-bin.000002
Relay_Log_Pos: 993
Relay_Master_Log_File: binlog.000002
Slave_IO_Running: Yes
Slave_SQL_Running: No
Replicate_Do_DB:
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
Last_Errno: 1031
Last_Error: Error 'Table storage engine for 'ttt' doesn't have this option' on query. Default database: 'db'. Query: 'delete from ttt where id=3'
Skip_Counter: 0
Exec_Master_Log_Pos: 786
Relay_Log_Space: 1466
Until_Condition: None
Until_Log_File:
Until_Log_Pos: 0
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master: NULL
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno: 0
Last_IO_Error:
Last_SQL_Errno: 1031
Last_SQL_Error: Error 'Table storage engine for 'ttt' doesn't have this option' on query. Default database: 'db'. Query: 'delete from ttt where id=3'
Replicate_Ignore_Server_Ids:
Master_Server_Id: 101
Master_UUID: ae40cabd-efb2-11ec-ac20-44a84203989a
Master_Info_File: /data/stonedb/install/data/master.info
SQL_Delay: 0
SQL_Remaining_Delay: NULL
Slave_SQL_Running_State:
Master_Retry_Count: 86400
Master_Bind:
Last_IO_Error_Timestamp:
Last_SQL_Error_Timestamp: 220729 02:26:29
Master_SSL_Crl:
Master_SSL_Crlpath:
Retrieved_Gtid_Set: ae40cabd-efb2-11ec-ac20-44a84203989a:1-4
Executed_Gtid_Set: 4ddecc1a-ee49-11ec-96fe-f219e7257407:1,
ae40cabd-efb2-11ec-ac20-44a84203989a:1-3
Auto_Position: 1
Replicate_Rewrite_DB:
Channel_Name:
Master_TLS_Version:
1 row in set (0.00 sec)
mysql> select * from ttt;
+----+------+
| id | name |
+----+------+
| 1 | AAA |
| 2 | BBB |
| 3 | CCC |
+----+------+
3 rows in set (0.00 sec)
主庫執行 delete後,由於 StoneDB 不支持 delete,從庫會有報錯,並且主從複製中斷。
下一步需要在主庫找到執行 delete操作的gtid值。
###主庫
mysql> show binary logs;
+---------------+-----------+
| Log_name | File_size |
+---------------+-----------+
| binlog.000001 | 177 |
| binlog.000002 | 1053 |
+---------------+-----------+
2 rows in set (0.00 sec)
mysql> show binlog events in '/data/stonedb/install/binlog/binlog.000002';
+---------------+------+----------------+-----------+-------------+-------------------------------------------------------------------+
| Log_name | Pos | Event_type | Server_id | End_log_pos | Info |
+---------------+------+----------------+-----------+-------------+-------------------------------------------------------------------+
| binlog.000002 | 4 | Format_desc | 101 | 123 | Server ver: 5.7.36-StoneDB-log, Binlog ver: 4 |
| binlog.000002 | 123 | Previous_gtids | 101 | 154 | |
| binlog.000002 | 154 | Gtid | 101 | 219 | SET @@SESSION.GTID_NEXT= 'ae40cabd-efb2-11ec-ac20-44a84203989a:1' |
| binlog.000002 | 219 | Query | 101 | 307 | create database db |
| binlog.000002 | 307 | Gtid | 101 | 372 | SET @@SESSION.GTID_NEXT= 'ae40cabd-efb2-11ec-ac20-44a84203989a:2' |
| binlog.000002 | 372 | Query | 101 | 494 | use `db`; create table ttt(id int primary key,name varchar(5)) |
| binlog.000002 | 494 | Gtid | 101 | 559 | SET @@SESSION.GTID_NEXT= 'ae40cabd-efb2-11ec-ac20-44a84203989a:3' |
| binlog.000002 | 559 | Query | 101 | 634 | BEGIN |
| binlog.000002 | 634 | Query | 101 | 755 | use `db`; insert into ttt values(1,'AAA'),(2,'BBB'),(3,'CCC') |
| binlog.000002 | 755 | Xid | 101 | 786 | COMMIT /* xid=20 */ |
| binlog.000002 | 786 | Gtid | 101 | 851 | SET @@SESSION.GTID_NEXT= 'ae40cabd-efb2-11ec-ac20-44a84203989a:4' |
| binlog.000002 | 851 | Query | 101 | 926 | BEGIN |
| binlog.000002 | 926 | Query | 101 | 1022 | use `db`; delete from ttt where id=3 |
| binlog.000002 | 1022 | Xid | 101 | 1053 | COMMIT /* xid=28 */ |
+---------------+------+----------------+-----------+-------------+-------------------------------------------------------------------+
14 rows in set (0.00 sec)
如果是在生產環境找主庫 delete 操作的 gtid 值,需要知道哪個時間點,然後用 mysqlbinlog 解析binlog。
這裡由於是做測試,可以簡單快遞的找到 delete 操作的 gtid 值,ae40cabd-efb2-11ec-ac20-44a84203989a:4。
gtid 值由參數 server_uuid 和事務 id 組成,標識一個這個操作的唯一性。
###從庫
set gtid_next='ae40cabd-efb2-11ec-ac20-44a84203989a:4';
begin;
commit;
set gtid_next=automatic;
start slave;
mysql> show slave status\G
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: 192.168.30.101
Master_User: u_repl
Master_Port: 33306
Connect_Retry: 60
Master_Log_File: binlog.000002
Read_Master_Log_Pos: 1053
Relay_Log_File: ub01-relay-bin.000002
Relay_Log_Pos: 1260
Relay_Master_Log_File: binlog.000002
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
Replicate_Do_DB:
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
Last_Errno: 0
Last_Error:
Skip_Counter: 0
Exec_Master_Log_Pos: 1053
Relay_Log_Space: 1466
Until_Condition: None
Until_Log_File:
Until_Log_Pos: 0
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno: 0
Last_IO_Error:
Last_SQL_Errno: 0
Last_SQL_Error:
Replicate_Ignore_Server_Ids:
Master_Server_Id: 101
Master_UUID: ae40cabd-efb2-11ec-ac20-44a84203989a
Master_Info_File: /data/stonedb/install/data/master.info
SQL_Delay: 0
SQL_Remaining_Delay: NULL
Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates
Master_Retry_Count: 86400
Master_Bind:
Last_IO_Error_Timestamp:
Last_SQL_Error_Timestamp:
Master_SSL_Crl:
Master_SSL_Crlpath:
Retrieved_Gtid_Set: ae40cabd-efb2-11ec-ac20-44a84203989a:1-4
Executed_Gtid_Set: 4ddecc1a-ee49-11ec-96fe-f219e7257407:1,
ae40cabd-efb2-11ec-ac20-44a84203989a:1-4
Auto_Position: 1
Replicate_Rewrite_DB:
Channel_Name:
Master_TLS_Version:
1 row in set (0.00 sec)
mysql> select * from ttt;
+----+------+
| id | name |
+----+------+
| 1 | AAA |
| 2 | BBB |
| 3 | CCC |
+----+------+
3 rows in set (0.00 sec)
利用 gtid 跳過一個空事務後,主從複製的線程已經正常啟動,但由於 StoneDB 不支持 delete,現在主從環境數據是不一致的。
###主庫
mysql> insert into ttt values(4,'DDD');
Query OK, 1 row affected (0.00 sec)
mysql> select * from ttt;
+----+------+
| id | name |
+----+------+
| 1 | AAA |
| 2 | BBB |
| 4 | DDD |
+----+------+
3 rows in set (0.00 sec)
###從庫
mysql> select * from ttt;
+----+------+
| id | name |
+----+------+
| 1 | AAA |
| 2 | BBB |
| 3 | CCC |
| 4 | DDD |
+----+------+
4 rows in set (0.00 sec)
主從複製的線程啟動後,主庫的更新,從庫同步正常。
如果覺得在主庫找 delete 的 gtid 值麻煩,在主庫執行 delete 前,可以指定 delete 的 gtid 值。在從庫還是根據這個 gtid 值執行空事務。
###
mysql> show variables like 'server_uuid';
+---------------+--------------------------------------+
| Variable_name | Value |
+---------------+--------------------------------------+
| server_uuid | ae40cabd-efb2-11ec-ac20-44a84203989a |
+---------------+--------------------------------------+
1 row in set (0.01 sec)
mysql> set gtid_next='ae40cabd-efb2-11ec-ac20-44a84203989a:100';
Query OK, 0 rows affected (0.00 sec)
mysql> begin;
Query OK, 0 rows affected (0.00 sec)
mysql> delete from ttt where id=1;
Query OK, 1 row affected (0.00 sec)
mysql> commit;
Query OK, 0 rows affected (0.00 sec)
mysql> set gtid_next=automatic;
Query OK, 0 rows affected (0.00 sec)
###從庫
set gtid_next='ae40cabd-efb2-11ec-ac20-44a84203989a:100';
begin;
commit;
set gtid_next=automatic;
start slave;
關閉當前線程的binlog
###主庫
mysql> show create table ttt\G
*************************** 1. row ***************************
Table: ttt
Create Table: CREATE TABLE `ttt` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`name` varchar(5) DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=3 DEFAULT CHARSET=utf8mb4
1 row in set (0.00 sec)
mysql> select * from ttt;
+----+------+
| id | name |
+----+------+
| 1 | AAA |
| 2 | BBB |
+----+------+
2 rows in set (0.00 sec)
###從庫
mysql> show create table ttt\G
*************************** 1. row ***************************
Table: ttt
Create Table: CREATE TABLE `ttt` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`name` varchar(5) DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=STONEDB DEFAULT CHARSET=utf8mb4
1 row in set (0.00 sec)
mysql> select * from ttt;
+----+------+
| id | name |
+----+------+
| 1 | AAA |
| 2 | BBB |
+----+------+
2 rows in set (0.00 sec)
###主庫
mysql> set sql_log_bin=off;
Query OK, 0 rows affected (0.00 sec)
mysql> alter table ttt modify name varchar(10);
Query OK, 0 rows affected (0.00 sec)
Records: 0 Duplicates: 0 Warnings: 0
mysql> show create table ttt\G
*************************** 1. row ***************************
Table: ttt
Create Table: CREATE TABLE `ttt` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`name` varchar(10) DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=3 DEFAULT CHARSET=utf8mb4
1 row in set (0.00 sec)
###從庫
mysql> show slave status\G
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: 192.168.30.101
Master_User: u_repl
Master_Port: 33306
Connect_Retry: 60
Master_Log_File: binlog.000002
Read_Master_Log_Pos: 2288
Relay_Log_File: ub01-relay-bin.000002
Relay_Log_Pos: 2495
Relay_Master_Log_File: binlog.000002
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
Replicate_Do_DB:
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
Last_Errno: 0
Last_Error:
Skip_Counter: 0
Exec_Master_Log_Pos: 2288
Relay_Log_Space: 2701
Until_Condition: None
Until_Log_File:
Until_Log_Pos: 0
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno: 0
Last_IO_Error:
Last_SQL_Errno: 0
Last_SQL_Error:
Replicate_Ignore_Server_Ids:
Master_Server_Id: 101
Master_UUID: ae40cabd-efb2-11ec-ac20-44a84203989a
Master_Info_File: /data/stonedb/install/data/master.info
SQL_Delay: 0
SQL_Remaining_Delay: NULL
Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates
Master_Retry_Count: 86400
Master_Bind:
Last_IO_Error_Timestamp:
Last_SQL_Error_Timestamp:
Master_SSL_Crl:
Master_SSL_Crlpath:
Retrieved_Gtid_Set: ae40cabd-efb2-11ec-ac20-44a84203989a:1-8:100
Executed_Gtid_Set: 4ddecc1a-ee49-11ec-96fe-f219e7257407:1-3,
ae40cabd-efb2-11ec-ac20-44a84203989a:1-8:100
Auto_Position: 1
Replicate_Rewrite_DB:
Channel_Name:
Master_TLS_Version:
1 row in set (0.00 sec)
mysql> show create table ttt\G
*************************** 1. row ***************************
Table: ttt
Create Table: CREATE TABLE `ttt` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`name` varchar(5) DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=STONEDB DEFAULT CHARSET=utf8mb4
1 row in set (0.00 sec)
主庫關閉當前線程的binlog,對錶做DDL,將欄位 name 的長度擴大。
主從複製正常,從庫表的欄位 name 的長度不變。
###主庫
開啟新的線程,註意一點是開啟新的線程!!!
mysql> insert into ttt(name) values('CCC');
Query OK, 1 row affected (0.00 sec)
mysql> select * from ttt;
+----+------+
| id | name |
+----+------+
| 1 | AAA |
| 2 | BBB |
| 3 | CCC |
+----+------+
3 rows in set (0.00 sec)
###從庫
mysql> select * from ttt;
+----+------+
| id | name |
+----+------+
| 1 | AAA |
| 2 | BBB |
| 3 | CCC |
+----+------+
3 rows in set (0.00 sec)
sql_log_bin=off,關閉的是當前線程的binlog,不影響其他線程的任何更新。
以上兩種方法都可以規避從庫不想重做主庫的某個更新動作,目的是讓從庫遇到不支持的操作時可以讓主從複製的線程正常工作,但帶來的問題是主從環境數據不一致。