亚欧色一区w666天堂,色情一区二区三区免费看,少妇特黄A片一区二区三区,亚洲人成网站999久久久综合,国产av熟女一区二区三区

  • 發布文章
  • 消息中心
點贊
收藏
評論
分享
原創

淺談分布式數據庫場景下分庫分表技術實踐(三)

2023-08-25 09:05:21
12
0
  1. 背景:

     針對海量(liang)增長模式(shi)的(de)(de)(de)行業(ye)關系型(xing)數據,往(wang)往(wang)采(cai)用(yong)分(fen)(fen)(fen)布式(shi)數據庫模式(shi),為了提升分(fen)(fen)(fen)布式(shi)數據庫的(de)(de)(de)高性能(neng)、高可(ke)靠性、高可(ke)擴展性生(sheng)產要求,我(wo)們往(wang)往(wang)采(cai)用(yong)分(fen)(fen)(fen)庫分(fen)(fen)(fen)表分(fen)(fen)(fen)片設計,實(shi)現(xian)業(ye)務數據的(de)(de)(de)分(fen)(fen)(fen)片存(cun)儲。在(zai)分(fen)(fen)(fen)片存(cun)儲中(zhong),一(yi)(yi)般(ban)會(hui)(hui)采(cai)用(yong)分(fen)(fen)(fen)片鍵(jian)(jian)方式(shi),指定(ding)某(mou)些字(zi)段(duan)(duan)為分(fen)(fen)(fen)片鍵(jian)(jian)字(zi)段(duan)(duan),并通過分(fen)(fen)(fen)片鍵(jian)(jian)字(zi)段(duan)(duan)進(jin)行分(fen)(fen)(fen)片位(wei)(wei)置(zhi)的(de)(de)(de)定(ding)位(wei)(wei),實(shi)現(xian)數據的(de)(de)(de)快(kuai)速(su)讀寫。那么,在(zai)實(shi)際業(ye)務中(zhong),根(gen)據上篇所描(miao)述(shu),業(ye)務實(shi)體(ti)之間往(wang)往(wang)存(cun)在(zai)關聯(lian)關系,為了實(shi)現(xian)關聯(lian)關系實(shi)體(ti)的(de)(de)(de)快(kuai)速(su)查(cha)詢定(ding)位(wei)(wei),會(hui)(hui)建立分(fen)(fen)(fen)片鍵(jian)(jian)關系,實(shi)現(xian)實(shi)體(ti)間快(kuai)速(su)查(cha)找(zhao)定(ding)位(wei)(wei),那么,這種做法會(hui)(hui)帶(dai)來另外一(yi)(yi)個(ge)問題:當(dang)其中(zhong)一(yi)(yi)個(ge)分(fen)(fen)(fen)片鍵(jian)(jian)實(shi)體(ti)發生(sheng)變更的(de)(de)(de)時(shi)候,如(ru)以客(ke)戶身份證(zheng)(zheng)編號為分(fen)(fen)(fen)片鍵(jian)(jian)時(shi)對所在(zai)身份證(zheng)(zheng)進(jin)行批量(liang)割接或(huo)者客(ke)戶合并時(shi),可(ke)能(neng)會(hui)(hui)產生(sheng)大量(liang)的(de)(de)(de)跨分(fen)(fen)(fen)片變更事(shi)務處理。

  1. 分析:

     根據(ju)上述所述,針(zhen)對分布式數據(ju)庫中(zhong)(zhong)存在關(guan)聯關(guan)系(xi)的分片(pian)鍵(jian)(jian)關(guan)系(xi)表(biao)(biao),當其(qi)中(zhong)(zhong)一個(ge)分片(pian)鍵(jian)(jian)發生(sheng)批量遷移(yi)時,則可能會(hui)造成批量跨(kua)分片(pian)變更事(shi)務處(chu)理,所以(yi),為了避免實體表(biao)(biao)中(zhong)(zhong)分片(pian)鍵(jian)(jian)一處(chu)修改引起分片(pian)鍵(jian)(jian)關(guan)系(xi)表(biao)(biao)中(zhong)(zhong)多(duo)處(chu)跨(kua)分片(pian)修改事(shi)務,可以(yi)考慮增加分片(pian)鍵(jian)(jian)尋址表(biao)(biao),用于跟蹤實體表(biao)(biao)中(zhong)(zhong)分片(pian)鍵(jian)(jian)的變更軌跡。

例如,仍(reng)以客(ke)戶資料表為例:

1)建客戶分片表:

      CREATE TABLE ` customer ` (

  `customer_id` varchar(20) not  NULL,

  `field_1` varchar(20) DEFAULT NULL,

  `field_2` int(11) DEFAULT '1',

  `shard1` bigint(20) DEFAULT NULL,

  PRIMARY KEY (`customer_id `)

) ENGINE=InnoDB DEFAULT CHARSET=utf8;

   ;     然后(hou)設(she)置分(fen)片鍵及其分(fen)片算法(fa)

        sharding @@table name=' customer ' set type='sharding' and sharding_func='shard-XXX-FUNC' and sharding_id='shard1' and dn='dn1…';

2)建客(ke)戶關(guan)聯(lian)關(guan)系分片表:

        創(chuang)建(jian)客(ke)戶關聯關系分片庫表:

              CREATE TABLE ` customer_rela ` (

  `customer_id_a` varchar(20) not  NULL, --關聯的(de)A端客戶標識

  `field_1` varchar(20) DEFAULT NULL,

  `field_2` int(11) DEFAULT '1',

  `shard1` bigint(20) DEFAULT NULL,--本表記錄的分片鍵

   `customer_id_z` varchar(20) not  NULL, --關(guan)聯的Z端客戶標識

   ` customer_id_z_shard1 ` bigint(20) DEFAULT NULL,--關聯(lian)的Z端客戶所在分片鍵

 

  PRIMARY KEY (`customer_id_a `,`customer_id_z `)

) ENGINE=InnoDB DEFAULT CHARSET=utf8;

        然(ran)后設置關聯(lian)表的分片鍵及其分片算法(fa)

        sharding @@table name=' customer_rela ' set type='sharding' and sharding_func='shard-XXX-FUNC' and sharding_id='shard1' and dn='dn1…';

當需要創(chuang)建兩(liang)(liang)個客戶的(de)關聯關系時,可以從兩(liang)(liang)端分(fen)別(bie)創(chuang)建兩(liang)(liang)條記錄(lu),記錄(lu)雙(shuang)向(xiang)的(de)關聯關系,同時記錄(lu)雙(shuang)方客戶所(suo)在的(de)分(fen)片鍵標識(shi)

3)增加客戶尋址(zhi)表(biao)customer_key_addr

CREATE TABLE ` customer_key_addr` (

  `customer_id` varchar(20) not  NULL, --當前客戶標(biao)識

  `newshard1` bigint(20) DEFAULT NULL,  --當前客戶分(fen)片(pian)(pian)鍵變(bian)更(geng)后的新的分(fen)片(pian)(pian)鍵

  `shard1` bigint(20) DEFAULT NULL,--本(ben)表記錄的原有分(fen)片鍵(jian)

  PRIMARY KEY (`customer_id `)

) ENGINE=InnoDB DEFAULT CHARSET=utf8;

        然后設置客戶尋(xun)址表(biao)的分片鍵及其分片算法(fa)

    ;    sharding @@table name=' customer_key_addr ' set type='sharding' and sharding_func='shard-XXX-FUNC' and sharding_id='shard1' and dn='dn1…';

4)為了(le)實現(xian)在尋址表(biao)場(chang)景下的客(ke)戶(hu)信息(xi)快速方便(bian)查詢,可以封裝(zhuang)公(gong)共的客(ke)戶(hu)資料查詢方法:

        首(shou)先(xian)根據客戶標識以及分片鍵(jian)從客戶表進(jin)行查詢:

        Select *  from customer where shard1 = ? and customer_id = ?

    如(ru)果(guo)上述記錄是因為分片鍵(jian)變更(geng)處(chu)于已遷移注銷(xiao)狀(zhuang)態,則再從(cong)客戶尋址表中查找:

     Select * from customer_key_addr where shard1 = ? and customer_id = ?

    根據(ju)上(shang)述(shu)記錄獲取新的(de)分(fen)片(pian)鍵標識(shi),然后根據(ju)新分(fen)片(pian)鍵標識(shi)按照上(shang)述(shu)邏輯(ji)再從(cong)客戶表中(zhong)查找。

 

通過上述分(fen)片(pian)鍵(jian)(jian)尋址表(biao)方式,在用戶對(dui)客(ke)戶分(fen)片(pian)鍵(jian)(jian)進行(xing)變更時(shi),可以實現對(dui)客(ke)戶分(fen)片(pian)記錄的快速追蹤(zong)訪(fang)問處(chu)理。

  1. 結尾:

     在(zai)實際分布式大型應用中,業務實體數(shu)(shu)(shu)據(ju)(ju)之間關系(xi)非常復雜,而且變(bian)更頻繁,那么(me),在(zai)海量(liang)數(shu)(shu)(shu)據(ju)(ju)的(de)(de)(de)分庫分表(biao)存儲場景下,如何(he)實現對分片(pian)數(shu)(shu)(shu)據(ju)(ju)的(de)(de)(de)高效讀寫(xie)變(bian)更處理以及保持數(shu)(shu)(shu)據(ju)(ju)一(yi)致性顯(xian)得尤為(wei)重要。本文通(tong)過(guo)分片(pian)鍵尋址表(biao)方式,進(jin)行分片(pian)鍵數(shu)(shu)(shu)據(ju)(ju)的(de)(de)(de)追(zhui)蹤處理,即避免在(zai)分片(pian)鍵變(bian)更時(shi)可能引起大量(liang)跨分片(pian)事務處理,又能實現數(shu)(shu)(shu)據(ju)(ju)的(de)(de)(de)快速追(zhui)蹤訪問,提高應用系(xi)統讀寫(xie)效率。

0條評論
作者已關閉評論
鄧****強
8文章數
0粉絲數(shu)
鄧****強
8 文(wen)章 | 0 粉絲
原(yuan)創

淺談分布式數據庫場景下分庫分表技術實踐(三)

2023-08-25 09:05:21
12
0
  1. 背景:

     針(zhen)對(dui)海量(liang)增長(chang)模(mo)式(shi)(shi)的(de)(de)行(xing)(xing)業(ye)關系(xi)(xi)型數(shu)據(ju),往往采用(yong)分(fen)(fen)(fen)(fen)(fen)布式(shi)(shi)數(shu)據(ju)庫模(mo)式(shi)(shi),為(wei)了(le)提升分(fen)(fen)(fen)(fen)(fen)布式(shi)(shi)數(shu)據(ju)庫的(de)(de)高性(xing)能(neng)、高可(ke)靠性(xing)、高可(ke)擴(kuo)展(zhan)性(xing)生產(chan)要求,我們(men)往往采用(yong)分(fen)(fen)(fen)(fen)(fen)庫分(fen)(fen)(fen)(fen)(fen)表分(fen)(fen)(fen)(fen)(fen)片(pian)(pian)(pian)(pian)設計,實(shi)現業(ye)務數(shu)據(ju)的(de)(de)分(fen)(fen)(fen)(fen)(fen)片(pian)(pian)(pian)(pian)存(cun)儲。在分(fen)(fen)(fen)(fen)(fen)片(pian)(pian)(pian)(pian)存(cun)儲中,一(yi)(yi)般會(hui)采用(yong)分(fen)(fen)(fen)(fen)(fen)片(pian)(pian)(pian)(pian)鍵(jian)方式(shi)(shi),指(zhi)定(ding)(ding)某些字段(duan)為(wei)分(fen)(fen)(fen)(fen)(fen)片(pian)(pian)(pian)(pian)鍵(jian)字段(duan),并通過(guo)分(fen)(fen)(fen)(fen)(fen)片(pian)(pian)(pian)(pian)鍵(jian)字段(duan)進行(xing)(xing)分(fen)(fen)(fen)(fen)(fen)片(pian)(pian)(pian)(pian)位(wei)(wei)置的(de)(de)定(ding)(ding)位(wei)(wei),實(shi)現數(shu)據(ju)的(de)(de)快(kuai)速(su)讀(du)寫。那么,在實(shi)際業(ye)務中,根據(ju)上篇(pian)所描述,業(ye)務實(shi)體(ti)之間(jian)往往存(cun)在關聯關系(xi)(xi),為(wei)了(le)實(shi)現關聯關系(xi)(xi)實(shi)體(ti)的(de)(de)快(kuai)速(su)查(cha)詢定(ding)(ding)位(wei)(wei),會(hui)建立分(fen)(fen)(fen)(fen)(fen)片(pian)(pian)(pian)(pian)鍵(jian)關系(xi)(xi),實(shi)現實(shi)體(ti)間(jian)快(kuai)速(su)查(cha)找定(ding)(ding)位(wei)(wei),那么,這種做法會(hui)帶來另外一(yi)(yi)個問題:當其(qi)中一(yi)(yi)個分(fen)(fen)(fen)(fen)(fen)片(pian)(pian)(pian)(pian)鍵(jian)實(shi)體(ti)發生變更的(de)(de)時(shi)(shi)候,如以客戶(hu)身(shen)份證編號(hao)為(wei)分(fen)(fen)(fen)(fen)(fen)片(pian)(pian)(pian)(pian)鍵(jian)時(shi)(shi)對(dui)所在身(shen)份證進行(xing)(xing)批量(liang)割(ge)接或者(zhe)客戶(hu)合并時(shi)(shi),可(ke)能(neng)會(hui)產(chan)生大量(liang)的(de)(de)跨(kua)分(fen)(fen)(fen)(fen)(fen)片(pian)(pian)(pian)(pian)變更事務處理。

  1. 分析:

     根據(ju)(ju)上述所(suo)述,針對分(fen)布式(shi)數據(ju)(ju)庫中(zhong)(zhong)(zhong)存在關(guan)(guan)聯關(guan)(guan)系(xi)(xi)的分(fen)片(pian)(pian)(pian)鍵(jian)關(guan)(guan)系(xi)(xi)表(biao),當其(qi)中(zhong)(zhong)(zhong)一個(ge)分(fen)片(pian)(pian)(pian)鍵(jian)發生(sheng)批量遷移時,則(ze)可能會(hui)造成批量跨(kua)分(fen)片(pian)(pian)(pian)變(bian)更(geng)事務處理,所(suo)以,為了避免實(shi)體(ti)表(biao)中(zhong)(zhong)(zhong)分(fen)片(pian)(pian)(pian)鍵(jian)一處修改(gai)引起分(fen)片(pian)(pian)(pian)鍵(jian)關(guan)(guan)系(xi)(xi)表(biao)中(zhong)(zhong)(zhong)多處跨(kua)分(fen)片(pian)(pian)(pian)修改(gai)事務,可以考(kao)慮(lv)增加分(fen)片(pian)(pian)(pian)鍵(jian)尋址表(biao),用(yong)于(yu)跟蹤實(shi)體(ti)表(biao)中(zhong)(zhong)(zhong)分(fen)片(pian)(pian)(pian)鍵(jian)的變(bian)更(geng)軌跡。

例如,仍(reng)以客戶資料表(biao)為(wei)例:

1)建客戶分片表:

      CREATE TABLE ` customer ` (

  `customer_id` varchar(20) not  NULL,

  `field_1` varchar(20) DEFAULT NULL,

  `field_2` int(11) DEFAULT '1',

  `shard1` bigint(20) DEFAULT NULL,

  PRIMARY KEY (`customer_id `)

) ENGINE=InnoDB DEFAULT CHARSET=utf8;

        然后設置分片(pian)鍵及其分片(pian)算(suan)法(fa)

        sharding @@table name=' customer ' set type='sharding' and sharding_func='shard-XXX-FUNC' and sharding_id='shard1' and dn='dn1…';

2)建客戶(hu)關聯關系分(fen)片表(biao):

        創建客戶關聯關系分(fen)片庫表:

              CREATE TABLE ` customer_rela ` (

  `customer_id_a` varchar(20) not  NULL, --關聯的A端客戶(hu)標識

  `field_1` varchar(20) DEFAULT NULL,

  `field_2` int(11) DEFAULT '1',

  `shard1` bigint(20) DEFAULT NULL,--本表記錄的分片鍵(jian)

   `customer_id_z` varchar(20) not  NULL, --關聯的Z端(duan)客戶(hu)標識

   ` customer_id_z_shard1 ` bigint(20) DEFAULT NULL,--關聯的(de)Z端(duan)客戶所在分片鍵

 

  PRIMARY KEY (`customer_id_a `,`customer_id_z `)

) ENGINE=InnoDB DEFAULT CHARSET=utf8;

   ;    ; 然后設置關聯表的分(fen)片(pian)(pian)鍵及其分(fen)片(pian)(pian)算法

        sharding @@table name=' customer_rela ' set type='sharding' and sharding_func='shard-XXX-FUNC' and sharding_id='shard1' and dn='dn1…';

當需要創(chuang)建兩(liang)個客戶(hu)的關聯關系(xi)時,可以從兩(liang)端分(fen)別創(chuang)建兩(liang)條記錄(lu),記錄(lu)雙向的關聯關系(xi),同時記錄(lu)雙方客戶(hu)所在的分(fen)片鍵標識(shi)

3)增(zeng)加(jia)客戶(hu)尋(xun)址表customer_key_addr

CREATE TABLE ` customer_key_addr` (

  `customer_id` varchar(20) not  NULL, --當前客戶標識

  `newshard1` bigint(20) DEFAULT NULL,  --當前(qian)客戶分片(pian)鍵(jian)變更后的新的分片(pian)鍵(jian)

  `shard1` bigint(20) DEFAULT NULL,--本表記錄的原(yuan)有(you)分(fen)片鍵

  PRIMARY KEY (`customer_id `)

) ENGINE=InnoDB DEFAULT CHARSET=utf8;

        然(ran)后(hou)設置客戶尋址表的分片鍵及其分片算法

        sharding @@table name=' customer_key_addr ' set type='sharding' and sharding_func='shard-XXX-FUNC' and sharding_id='shard1' and dn='dn1…';

4)為了實(shi)現在尋(xun)址表場(chang)景下的(de)客戶信息快速(su)方便(bian)查詢,可以(yi)封裝公共的(de)客戶資料查詢方法:

    ; ;   首先(xian)根據(ju)客戶標識(shi)以及分片鍵從客戶表進行(xing)查詢:

        Select *  from customer where shard1 = ? and customer_id = ?

    如果上述記錄是因(yin)為(wei)分(fen)片鍵變更處(chu)于已遷移注銷狀態,則再(zai)從客戶尋址(zhi)表中(zhong)查找:

     Select * from customer_key_addr where shard1 = ? and customer_id = ?

    根據(ju)上述記錄獲取(qu)新(xin)的分片鍵標識,然后根據(ju)新(xin)分片鍵標識按照上述邏輯再從客戶表(biao)中(zhong)查找。

 

通過上述分(fen)片(pian)(pian)鍵尋址表方(fang)式,在用戶(hu)對(dui)客戶(hu)分(fen)片(pian)(pian)鍵進行變更(geng)時,可以實現(xian)對(dui)客戶(hu)分(fen)片(pian)(pian)記錄的(de)快速(su)追蹤訪問處理。

  1. 結尾:

     在實(shi)際分(fen)布式大(da)型(xing)應用中,業務實(shi)體數(shu)(shu)(shu)據(ju)(ju)之間關系非常復(fu)雜,而(er)且變更(geng)頻繁,那么,在海量數(shu)(shu)(shu)據(ju)(ju)的分(fen)庫分(fen)表(biao)存儲場(chang)景(jing)下,如何(he)實(shi)現對分(fen)片數(shu)(shu)(shu)據(ju)(ju)的高效讀(du)寫變更(geng)處(chu)理以(yi)及保持數(shu)(shu)(shu)據(ju)(ju)一致性顯(xian)得尤為重要。本文(wen)通過(guo)分(fen)片鍵(jian)尋址表(biao)方式,進行分(fen)片鍵(jian)數(shu)(shu)(shu)據(ju)(ju)的追(zhui)蹤(zong)處(chu)理,即避免在分(fen)片鍵(jian)變更(geng)時可能引起(qi)大(da)量跨分(fen)片事務處(chu)理,又(you)能實(shi)現數(shu)(shu)(shu)據(ju)(ju)的快(kuai)速(su)追(zhui)蹤(zong)訪問,提高應用系統讀(du)寫效率。

文章來自個人專欄
文(wen)章 | 訂閱
0條評論
作者已關閉評論
作者已關閉評論
0
0