- 背景:
針對海量(liang)增長模式(shi)的(de)(de)(de)行業(ye)關系型(xing)數據,往(wang)往(wang)采(cai)用(yong)分(fen)(fen)(fen)布式(shi)數據庫模式(shi),為了提升分(fen)(fen)(fen)布式(shi)數據庫的(de)(de)(de)高性能(neng)、高可(ke)靠性、高可(ke)擴展性生(sheng)產要求,我(wo)們往(wang)往(wang)采(cai)用(yong)分(fen)(fen)(fen)庫分(fen)(fen)(fen)表分(fen)(fen)(fen)片設計,實(shi)現(xian)業(ye)務數據的(de)(de)(de)分(fen)(fen)(fen)片存(cun)儲。在(zai)分(fen)(fen)(fen)片存(cun)儲中(zhong),一(yi)(yi)般(ban)會(hui)(hui)采(cai)用(yong)分(fen)(fen)(fen)片鍵(jian)(jian)方式(shi),指定(ding)某(mou)些字(zi)段(duan)(duan)為分(fen)(fen)(fen)片鍵(jian)(jian)字(zi)段(duan)(duan),并通過分(fen)(fen)(fen)片鍵(jian)(jian)字(zi)段(duan)(duan)進(jin)行分(fen)(fen)(fen)片位(wei)(wei)置(zhi)的(de)(de)(de)定(ding)位(wei)(wei),實(shi)現(xian)數據的(de)(de)(de)快(kuai)速(su)讀寫。那么,在(zai)實(shi)際業(ye)務中(zhong),根(gen)據上篇所描(miao)述(shu),業(ye)務實(shi)體(ti)之間往(wang)往(wang)存(cun)在(zai)關聯(lian)關系,為了實(shi)現(xian)關聯(lian)關系實(shi)體(ti)的(de)(de)(de)快(kuai)速(su)查(cha)詢定(ding)位(wei)(wei),會(hui)(hui)建立分(fen)(fen)(fen)片鍵(jian)(jian)關系,實(shi)現(xian)實(shi)體(ti)間快(kuai)速(su)查(cha)找(zhao)定(ding)位(wei)(wei),那么,這種做法會(hui)(hui)帶(dai)來另外一(yi)(yi)個(ge)問題:當(dang)其中(zhong)一(yi)(yi)個(ge)分(fen)(fen)(fen)片鍵(jian)(jian)實(shi)體(ti)發生(sheng)變更的(de)(de)(de)時(shi)候,如(ru)以客(ke)戶身份證(zheng)(zheng)編號為分(fen)(fen)(fen)片鍵(jian)(jian)時(shi)對所在(zai)身份證(zheng)(zheng)進(jin)行批量(liang)割接或(huo)者客(ke)戶合并時(shi),可(ke)能(neng)會(hui)(hui)產生(sheng)大量(liang)的(de)(de)(de)跨分(fen)(fen)(fen)片變更事(shi)務處理。
- 分析:
根據(ju)上述所述,針(zhen)對分布式數據(ju)庫中(zhong)(zhong)存在關(guan)聯關(guan)系(xi)的分片(pian)鍵(jian)(jian)關(guan)系(xi)表(biao)(biao),當其(qi)中(zhong)(zhong)一個(ge)分片(pian)鍵(jian)(jian)發生(sheng)批量遷移(yi)時,則可能會(hui)造成批量跨(kua)分片(pian)變更事(shi)務處(chu)理,所以(yi),為了避免實體表(biao)(biao)中(zhong)(zhong)分片(pian)鍵(jian)(jian)一處(chu)修改引起分片(pian)鍵(jian)(jian)關(guan)系(xi)表(biao)(biao)中(zhong)(zhong)多(duo)處(chu)跨(kua)分片(pian)修改事(shi)務,可以(yi)考慮增加分片(pian)鍵(jian)(jian)尋址表(biao)(biao),用于跟蹤實體表(biao)(biao)中(zhong)(zhong)分片(pian)鍵(jian)(jian)的變更軌跡。
例如,仍(reng)以客(ke)戶資料表為例:
1)建客戶分片表:
CREATE TABLE ` customer ` (
`customer_id` varchar(20) not NULL,
`field_1` varchar(20) DEFAULT NULL,
`field_2` int(11) DEFAULT '1',
`shard1` bigint(20) DEFAULT NULL,
PRIMARY KEY (`customer_id `)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
然后(hou)設(she)置分(fen)片鍵及其分(fen)片算法(fa)
sharding @@table name=' customer ' set type='sharding' and sharding_func='shard-XXX-FUNC' and sharding_id='shard1' and dn='dn1…';
2)建客(ke)戶關(guan)聯(lian)關(guan)系分片表:
創(chuang)建(jian)客(ke)戶關聯關系分片庫表:
CREATE TABLE ` customer_rela ` (
`customer_id_a` varchar(20) not NULL, --關聯的(de)A端客戶標識
`field_1` varchar(20) DEFAULT NULL,
`field_2` int(11) DEFAULT '1',
`shard1` bigint(20) DEFAULT NULL,--本表記錄的分片鍵
`customer_id_z` varchar(20) not NULL, --關(guan)聯的Z端客戶標識
` customer_id_z_shard1 ` bigint(20) DEFAULT NULL,--關聯(lian)的Z端客戶所在分片鍵
PRIMARY KEY (`customer_id_a `,`customer_id_z `)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
然(ran)后設置關聯(lian)表的分片鍵及其分片算法(fa)
sharding @@table name=' customer_rela ' set type='sharding' and sharding_func='shard-XXX-FUNC' and sharding_id='shard1' and dn='dn1…';
當需要創(chuang)建兩(liang)(liang)個客戶的(de)關聯關系時,可以從兩(liang)(liang)端分(fen)別(bie)創(chuang)建兩(liang)(liang)條記錄(lu),記錄(lu)雙(shuang)向(xiang)的(de)關聯關系,同時記錄(lu)雙(shuang)方客戶所(suo)在的(de)分(fen)片鍵標識(shi)
3)增加客戶尋址(zhi)表(biao)customer_key_addr
CREATE TABLE ` customer_key_addr` (
`customer_id` varchar(20) not NULL, --當前客戶標(biao)識
`newshard1` bigint(20) DEFAULT NULL, --當前客戶分(fen)片(pian)(pian)鍵變(bian)更(geng)后的新的分(fen)片(pian)(pian)鍵
`shard1` bigint(20) DEFAULT NULL,--本(ben)表記錄的原有分(fen)片鍵(jian)
PRIMARY KEY (`customer_id `)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
然后設置客戶尋(xun)址表(biao)的分片鍵及其分片算法(fa)
sharding @@table name=' customer_key_addr ' set type='sharding' and sharding_func='shard-XXX-FUNC' and sharding_id='shard1' and dn='dn1…';
4)為了(le)實現(xian)在尋址表(biao)場(chang)景下的客(ke)戶(hu)信息(xi)快速方便(bian)查詢,可以封裝(zhuang)公(gong)共的客(ke)戶(hu)資料查詢方法:
首(shou)先(xian)根據客戶標識以及分片鍵(jian)從客戶表進(jin)行查詢:
Select * from customer where shard1 = ? and customer_id = ?
如(ru)果(guo)上述記錄是因為分片鍵(jian)變更(geng)處(chu)于已遷移注銷(xiao)狀(zhuang)態,則再從(cong)客戶尋址表中查找:
Select * from customer_key_addr where shard1 = ? and customer_id = ?
根據(ju)上(shang)述(shu)記錄獲取新的(de)分(fen)片(pian)鍵標識(shi),然后根據(ju)新分(fen)片(pian)鍵標識(shi)按照上(shang)述(shu)邏輯(ji)再從(cong)客戶表中(zhong)查找。
通過上述分(fen)片(pian)鍵(jian)(jian)尋址表(biao)方式,在用戶對(dui)客(ke)戶分(fen)片(pian)鍵(jian)(jian)進行(xing)變更時(shi),可以實現對(dui)客(ke)戶分(fen)片(pian)記錄的快速追蹤(zong)訪(fang)問處(chu)理。
- 結尾:
在(zai)實際分布式大型應用中,業務實體數(shu)(shu)(shu)據(ju)(ju)之間關系(xi)非常復雜,而且變(bian)更頻繁,那么(me),在(zai)海量(liang)數(shu)(shu)(shu)據(ju)(ju)的(de)(de)(de)分庫分表(biao)存儲場景下,如何(he)實現對分片(pian)數(shu)(shu)(shu)據(ju)(ju)的(de)(de)(de)高效讀寫(xie)變(bian)更處理以及保持數(shu)(shu)(shu)據(ju)(ju)一(yi)致性顯(xian)得尤為(wei)重要。本文通(tong)過(guo)分片(pian)鍵尋址表(biao)方式,進(jin)行分片(pian)鍵數(shu)(shu)(shu)據(ju)(ju)的(de)(de)(de)追(zhui)蹤處理,即避免在(zai)分片(pian)鍵變(bian)更時(shi)可能引起大量(liang)跨分片(pian)事務處理,又能實現數(shu)(shu)(shu)據(ju)(ju)的(de)(de)(de)快速追(zhui)蹤訪問,提高應用系(xi)統讀寫(xie)效率。