銆愯浆銆慔Base涓璟ookeeper锛孯egionServer锛孧aster锛孋lient涔嬮棿鍏崇郴
Posted
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了銆愯浆銆慔Base涓璟ookeeper锛孯egionServer锛孧aster锛孋lient涔嬮棿鍏崇郴相关的知识，希望对你有一定的参考价值。
鏍囩锛?a href='http://www.mamicode.com/so/1/%e7%8a%b6%e6%80%81' title='鐘舵€?>鐘舵€?/a> 涓婁笅 鍐呭 manage 鏁版嵁鍚屾 淇℃伅 鏉ユ簮 img 浣跨敤
鍦?.0涔嬪墠HDFS涓彧鏈変竴涓狽ameNode锛屼絾瀵逛簬鍦ㄧ嚎鐨勫簲鐢ㄥ彧鏈変竴涓狽ameNode鏄笉瀹夊叏鐨勶紝鏁呭湪2.0涓NameNode杩涜鎶借薄锛屾娊璞℃垚NamService鍏朵笅鍖呭惈鏈夊涓狽ameNode锛?em>浣嗗彧鏈変竴涓繍琛屽湪娲昏穬鐘舵€侊紝鍥犳闇€瑕亃ookeeper杩涜閫変妇鍜岃嚜鍔ㄨ浆鎹€備竴鏃ctive褰撴帀涔嬪悗zookeeper浼氳嚜瀹氳繘琛屽垏鎹㈠皢standby鍒囨崲涓篴ctive銆?/strong>
鍥剧墖鏉ユ簮锛?a href="https://link.jianshu.com?t=https://issues.apache.org/jira/browse/HDFS-1623" rel="nofollow" target="_blank">HDFS-1623璁捐鏂囨。
鍥剧墖浣滆€咃細 Sanjay Radia, Suresh Srinivas
濡備笂鍥撅紝姣忎竴涓繍琛孨ameNode鐨勬満鍣ㄤ笂閮戒細杩愯涓€涓狥ailoverController Active杩涚▼锛岀敤浜庣洃鎺ameNode锛屽嵆FailoverController涓嶯ameNode 浜岃€呮槸杩愯鍦ㄥ悓涓€鍙拌妭鐐逛笂鐨勩€?/strong>
1銆戙€佸涓婏紝绯荤粺涓瑼ctive NameNode涓€鏃︽暟鎹彂鐢熷彉鍖?Active NameNode浼氬皢鍙樺寲锛坋dits鏂囦欢锛夊啓鍏ュ埌浠嬭川涓紝閫氬父浠嬭川浣跨敤锛?/p>
1銆丯FS缃戠粶鏂囦欢绯荤粺锛?/p>
2銆佷緷璧栦簬zookeeper鐨凧ournalNode,褰撴満鍣ㄤ笉鏄緢澶氭椂鍙互浣跨敤JournalNode瀛樺偍edits锛?strong>鑰屼竴鏃︿粙璐ㄤ腑鏁版嵁鍙戠敓鍙樺寲锛孲tandby鐨凬ameNode浼氬疄鏃跺悓姝ヤ粙璐ㄤ腑鐨勬暟鎹紝鍥犳ActiveNameNode涓庡浜嶴tandby鐨凬ameNode鏄疄鏃舵暟鎹悓姝ョ殑锛?/em>
2銆戙€丗ailoverController Active杩涚▼浼氬疄鏃剁洃鎺ctive鐨凬ameNode骞舵妸浠栫殑淇℃伅姹囨姤缁?strong>zookeeper锛?strong>鐢辨涓嶉毦鐞嗚В涓轰粈涔堣灏咶ailoverController涓嶯ameNode 瀹夎鍦ㄥ悓涓€涓妭鐐逛笂浜嗗惂锛屽洜涓篎ailoverController闇€瑕佸疄鏃剁殑鐩戞帶NameNode灏嗕簩鑰呮斁鍦ㄥ悓涓€涓妭鐐逛笂锛夛紝鑻ctive NameNode娌℃湁鍙戠敓鏁呴殰锛孎ailoverController Active杩涚▼浼氭瘡闅斾竴娈垫椂闂村皢鐩戞帶Actice NameNode鐨勭姸鎬侊紝骞跺皢Active NameNode鐨勪俊鎭彂閫佺粰zookeeper銆傝嫢涓€鏃︾洃鎺у埌Active NameNode鍑虹幇闂锛屽氨浼氬皢淇℃伅姹囨姤缁?strong>zookeeper锛岃€屽彟涓€涓猄tandby FailoverController杩涚▼涓巃ctive FailoverController杩涚▼鏄暟鎹悓姝ョ殑锛堥€氳繃zookeeper瀹炵幇锛夊洜姝わ紝Standby FailoverController杩涚▼浼氬悓鏃剁煡閬揂ctive NameNode褰撴帀浜嗭紝姝ゆ椂浼氱粰瀹冪洃鎺х殑Standby NameNode鑺傜偣鍙戦€佷俊鎭紝浣垮叾鎴愪负Active NameNode銆?/p>
CDH5.7.1鐗堟湰鐨刪dfs鍦▃ookeeper鐨勭洰褰曚负锛?hadoop-ha/nameservice1/{ActiveBreadCrumb/ActiveStandbyElectorLock}
hadoop-ha/Namespace锛堥泦缇ゅ綋鍓嶄负nameservice1 -- 瀵瑰簲hdfs鐨凬ameNode Nameservice閰嶇疆锛?nameservice1 鐩綍涓嬫湁锛欰ctiveBreadCrumb ActiveStandbyElectorLock 鍏跺唴瀹瑰潎涓簄ameservice1 - hadoop6 锛坔adoop6涓哄綋鍓峚ctiveNameNode hadoop7涓簊tandbyNameNode锛?/p>
yarn鍦▃ookeeper涓婄殑鑺傜偣锛?yarn-leader-election/yarnRM/{ActiveBreadCrumb/ActiveStandbyElectorLock} 璁拌浇resourceManager鐨凙ctive鑺傜偣鏈嶅姟鍣ㄥ湴鍧€
/rmstore/ZKRMStateRoot/{RMAppRoot,AMRMTokenSecretManagerRoot,EpochNode,RMDTSecretManagerRoot,RMVersionNode} 鍏朵腑RMAppRoot淇濈暀浜嗘墍鏈夊巻鍙茬殑鎻愪氦鍒皔arn涓婄殑搴旂敤绋嬪簭鐨勫厓鏁版嵁銆?/p>
/hbase/[...rs,master,balancer,namespace,hbaseid,table ...]
rs淇濈暀浜唕egion server鐨勫叆鍙?鍏跺瓙鑺傜偣瀵瑰簲region server 鐨?regionServer:60030/rs-status 涓?鐨?RegionServer鎻忚堪-ServerName锛堜竴鑸潵璇碊ataNode鑺傜偣涓婇兘浼氬畨瑁卹egion server 绉诲姩璁＄畻鑰屼笉鏄Щ鍔ㄦ暟鎹級
balancer璁板綍浜嗗叾璐熻浇鏁版嵁 namespace 璁板綍浜唄base鐨勫懡鍚嶇┖闂?涓€涓懡鍚嶇┖闂村寘鍚玠efault鍜宧base锛屽懡鍚嶇┖闂村唴鍒涘缓hbase琛?/p>
hbaseid璁板綍鐫€hbase cluster id锛宼able閲岀殑瀛愯妭鐐逛负hbase涓婄殑琛ㄣ€倀able-lock 涓婇攣鐨則able琛?/p>
涓巏afka鐨勭粨鍚?-- kafka鐩綍鐨勯厤缃湪 kafka-閰嶇疆鐨刏ookeeper Root -- zookeeper.chroot 濡傛灉涓虹┖鍦ㄩ粯璁ゅ湪zookeeper鐨勬牴鐩綍涓嬬洿鎺ュ垱寤篴dmin绛夎妭鐐癸紝濡傛灉娣诲姞/kafka鐨勮瘽锛屽垯鍦?kafka鐨勮妭鐐逛笅鍒涘缓涓€绯诲垪鑺傜偣銆?/p>
/kafka/{admin,isr_change_notification,controller_epoch,consumers,brokers,config,controller} 涓簁afka鎻愪緵鏈嶅姟鍏朵腑admin鑺傜偣鍖呭惈deletetopics鍏跺瓙鑺傜偣涓烘墍鏈夊垹闄よ繃寰梩opic銆俰sr_change_notification瀹炵幇kafka鍒嗗尯鍚屾鐩戝惉浠诲姟銆俢onsumers璁板綍浜嗗綋鍓嶇殑娑堣垂鑰呫€俠rokers鍖呭惈{ids,topics,seqid},ids鍖呭惈瀛愯妭鐐逛负brokerid 姝ゆ椂涓?70 --瀵瑰簲鐫€kafka閰嶇疆涓殑Kafka Broker 鐨凚roker ID -- broker.id 锛堝綋鍓峩afka闆嗙兢鍙湁涓€涓猙roker 鍏禸rokerid涓?70锛? brokerid涓?70锛岃鑺傜偣鐨勫唴瀹逛负{"jmx_port":"9393","timestamp","","endpoints":["PLAINTEXT://10.2.5.64:9092"],"host":"10.2.5.64","version":2,"port":9092} topics涓嬮潰鐨勮妭鐐瑰搴旂殑鏄湁澶氬皯涓猼opic锛屾瘡涓猼opic鐨勫瓙鑺傜偣涓簆artitions,partitions鐨勫瓙鑺傜偣涓哄垎鍖虹紪鍙穥0,1,2,...},姣忎釜鍒嗗尯缂栧彿瀛愯妭鐐逛负state锛宻tate鑺傜偣涓嬫病鏈夊瓙鑺傜偣浜嗐€俿tate鐨勬暟鎹负{"controller_epoch":25,"leader":170(brokerid),"version":1,"leader_epoch":1,"isr":[170]} isr--瀵瑰簲鐨勬槸璇ュ垎鍖烘暟鎹鍦ㄥ悓姝ョ姸鎬佸壇鏈殑brokerid
zookeeper涓嶩base
Client瀹㈡埛绔?/strong>銆丮aster銆丷egion閮戒細閫氳繃蹇冭烦鏈哄埗锛圧PC閫氫俊锛変笌zookeeper淇濇寔鑱旂郴銆?/strong>
褰撳湪Hbase涓彃鍏ユ垨璇诲彇鏁版嵁鏃?a target="_blank">娴佺▼濡備笅锛?/a>
1銆?/strong>鍦–lient涓啓涓€涓狫ava绫昏繍琛岋紝瀹㈡埛绔?/strong>鍙渶瑕佽繛鎺?strong>zookeeper锛?strong>瀹㈡埛绔?/strong>浼氫粠zookeeper涓緱鍒癛egionserver鐨勬槧灏勪俊鎭紝涔嬪悗瀹㈡埛绔?/strong>浼氱洿鎺ヨ繛鎺ュ埌Region Server锛?/p>
2銆?/strong>RegionServer鍦ㄥ惎鍔ㄤ箣鍚庝細鍚?strong>zookeeper姹囨姤淇℃伅锛堥€氳繃蹇冭烦RPC锛夛細鏈韩鏈夊灏慠egion锛屾湁鍝簺鏁版嵁锛屽綋鍓嶆満鍣ㄧ殑杩愯鐘跺喌绛夌瓑銆?/p>
3銆?/strong>master 鍚姩鍚庝篃浼氬悜zookeeper姹囨姤淇℃伅锛屽苟涓斾粠zookeeper涓緱鍒癛egion Server鐨勪竴浜涗俊鎭€備緥濡傚綋涓€鍙癛egion Server褰撴帀涔嬪悗锛?strong>zookeeper浼氬緱鐭ワ紝涔嬪悗Master涔熶細閫氳繃zookeeper寰楀埌璇egion Server褰撴帀鐨勪俊鎭€?/p>
4銆?/strong>褰?strong>瀹㈡埛绔?/strong>Client鍦ㄥ仛DDL锛堝垱寤猴紝淇敼锛屽垹闄よ〃锛夋椂锛屼細閫氳繃zookeeper鑾峰彇鍒癕aster鐨勫湴鍧€锛岃€孧aster涓繚瀛樹簡琛ㄧ殑鍏冩暟鎹俊鎭紝涔嬪悗Client灏卞彲浠ョ洿鎺ヤ笌Master杩涜閫氫俊锛岃繘琛岃〃鐨凞DL鎿嶄綔 5銆?/strong>褰揜egion涓暟鎹笉鏂澶э紝MAster浼氬悜Region Serve鍙戦€佹寚浠わ紝璁插垎鍓插嚭鏉ョ殑Region杩涜杞Щ锛?/p>
Zookeeper锛?/strong> 1銆戜繚璇佷换浣曟椂鍊欙紝闆嗙兢涓彧鏈変竴涓椿璺冪殑master锛屽洜涓轰负淇濊瘉瀹夊叏鎬т細鍚姩澶氫釜Master 2銆戝瓨鍌ㄦ墍鏈塕egion鐨勫鍧€鍏ュ彛銆傜煡閬撻偅涓猂egion鍦ㄥ摢鍙版満鍣ㄤ笂銆?/p>
3銆戝疄鏃剁洃鎺egion Server鐨勭姸鎬侊紝灏哛egion Server鐨勪笂涓嬬嚎鐨勪俊鎭眹鎶ョ粰HMaster銆傦紙鍥犱负姣忛棿闅斾竴娈垫椂闂达紝RegionServer涓嶮aster閮戒細zookeeper鍙戦€佸績璺充俊鎭級锛孯egion Server涓嶇洿鎺ュ悜Master鍙戦€佷俊鎭殑鍘熷洜鏄负浜嗗噺灏慚aster鐨勫帇鍔涘洜涓哄彧鏈変竴涓椿璺冪殑Master锛屾墍鏈夌殑RegionServer鍚屾椂鍚戜粬姹囨姤淇℃伅锛屽帇鍔涘お澶с€傝€岃嫢鏈?00鍙癛egionServer鏃讹紝Region Server鍙互鍒嗘瘡10鍙板悜涓€涓?strong>zookeeper 4銆戝瓨鍌℉base鐨勫厓鏁版嵁锛圫chema锛夊寘鎷紝鐭ラ亾鏁翠釜Hbase闆嗙兢涓湁鍝簺Table,姣忎釜 Table 鏈夊摢浜沜olumn family锛堝垪鏃忥級 Client Client鍖呭惈浜嗚闂瓾base鐨勬帴鍙ｏ紝Client缁存姢杩欎簺Cache鏉ュ姞蹇Hbase鐨勮闂紝姣斿Region鐨勪綅缃俊鎭紝zookeeper锛?strong>zookeeper
Master
Master鏈変互涓嬬壒鐐癸細
1銆佷负RegionServer鍒嗛厤Region
2銆佽礋璐egionServer鐨勮礋杞藉潎琛?/p>
3銆佸彂鐜板け鏁堢殑RegionServer骞堕噸鏂板垎閰嶅叾涓婄殑Region
4銆丠DFS涓婄殑鍨冨溇鏂囦欢鍥炴敹
5銆佸鐞哠chema鏇存柊璇锋眰
RegionServer鏈変互涓嬪嚑鐐癸細
1銆丷egionServer缁存姢Master鍒嗛厤缁欎粬鐨?Region锛屽鐞嗗杩欎簺Region鐨処O璇锋眰
2銆丷egionServer璐熻矗鍒囧垎鍦ㄨ繍琛岃繃绋嬩腑鍙樺緱杩囧ぇ鐨凴egion锛?/p>
鍥犳鍙湅鍑猴紝Client璁块棶Hbase涓婄殑鏁版嵁鐨勮繃绋嬪苟涓嶉渶瑕丮aster鐨勫弬涓庯紙瀵诲潃璁块棶zookeeper鍜孯egion Serve锛?a target="_blank">鍐欐暟鎹?/strong>璁块棶Region Server锛塎aster浠呬粎缁存姢鐫€table鍜孯egion鐨勫厓鏁版嵁淇℃伅銆傝礋杞藉緢浣庛€?/a>
浣滆€咃細鏃跺緟鍚?br />閾炬帴锛歨ttps://www.jianshu.com/p/4d3f31bf158d
鏉ユ簮锛氱畝涔?br />钁椾綔鏉冨綊浣滆€呮墍鏈夈€傚晢涓氳浆杞借鑱旂郴浣滆€呰幏寰楁巿鏉冿紝闈炲晢涓氳浆杞借娉ㄦ槑鍑哄銆?/div>
以上是关于銆愯浆銆慔Base涓璟ookeeper锛孯egionServer锛孧aster锛孋lient涔嬮棿鍏崇郴的主要内容，如果未能解决你的问题，请参考以下文章
銆愯浆銆?springBoot---蹇€熷垱寤洪」鐩紝鍒濊Вjackson
銆愯浆銆戙€怱alesfoece銆慉pproval Process 鍦?Apex 涓殑浣跨敤
銆愯浆銆?springboot鏁村悎redis-sentinel鏀寔Cache娉ㄨВ