hadoop闆嗙兢鍩烘湰杩涚▼

Posted 2022-09-10

鏍囩锛?a href='http://www.mamicode.com/so/1/%e9%80%9a%e4%bf%a1' title='閫氫俊'>閫氫俊   cat   oop   cti   over   瀹堟姢杩涚▼   杩愯   瑙掕壊   str   

 

namenode

銆€銆€namenode 鐩稿綋浜庝竴涓瀵艰€咃紝璐熻矗璋冨害 姣斿浣犻渶瑕佸瓨涓€涓?40m鐨勬枃浠?濡傛灉鎸夌収64m鍒嗗潡 閭ｄ箞namenode灏变細鎶婅繖10涓潡锛堣繖閲屼笉鑰冭檻鍓湰锛夊垎閰嶅埌闆嗙兢涓殑datanode涓?骞惰褰曞浜庡叧绯?銆傚綋浣犺涓嬭浇杩欎釜鏂囦欢鐨勬椂鍊檔amenode灏辩煡閬撳湪閭ｄ簺鑺傜偣涓婄粰浣犲彇杩欎簺鏁版嵁浜嗐€?/p>

 

QuorumPeerMain 

銆€銆€QuorumPeerMain Zookeeper闆嗙兢鍚姩鐨勫叆鍙ｇ被鏄疩uorumPeerMain鏉ュ姞杞介厤缃惎鍔≦uorumPeer绾跨▼銆傞鍏堟垜浠潵鐪嬩笅QuorumPeer锛?璋锋瓕缈昏瘧quorum鏄硶瀹氫汉鏁帮紝瀹氶鐨勬剰鎬濓紝 peer鏄绛夌殑鎰忔€濓紝閭ｄ箞QuorumPeer涓璹uorum浠ｈ〃鐨勬剰鎬濆氨鏄瘡涓獄ookeeper闆嗙兢鍚姩鐨勬椂鍊欓泦缇や腑zookeeper鏈嶅姟鏁伴噺灏卞凡缁忕‘瀹氫簡锛屽湪姣忎釜zookeeper鐨勯厤缃枃浠朵腑閰嶇疆闆嗙兢涓殑鎵€鏈夋満鍣ㄤ簨瀹炰笂杩欎篃纭畾zookeeper鍦ㄨ繍琛屼腑鏄笉鑳藉姩鎬佹墿瀹圭殑锛屽繀椤诲仠涓嬫湇鍔′慨鏀归厤缃墠鍙互锛決uorumPeer涓璸eer浠ｈ〃灏辨槸闆嗕腑姣忎釜zookeeper瑙掕壊鏄绛夌殑娌℃湁涓讳粠涔嬪垎锛屾瘡涓獄ookeeper鏈嶅姟閮藉彲浠ユ垚涓簂eader锛?follower锛宱bserver銆?/p>

 

journalnode

journalnode 涓や釜NameNode涓轰簡鏁版嵁鍚屾锛屼細閫氳繃涓€缁勭О浣淛ournalNodes鐨勭嫭绔嬭繘绋嬭繘琛岀浉浜掗€氫俊銆傚綋active鐘舵€佺殑NameNode鐨勫懡鍚嶇┖闂存湁浠讳綍淇敼鏃讹紝浼氬憡鐭ュぇ閮ㄥ垎鐨凧ournalNodes杩涚▼銆俿tandby鐘舵€佺殑NameNode鏈夎兘鍔涜鍙朖Ns涓殑鍙樻洿淇℃伅锛屽苟涓斾竴鐩寸洃鎺dit log鐨勫彉鍖栵紝鎶婂彉鍖栧簲鐢ㄤ簬鑷繁鐨勫懡鍚嶇┖闂淬€俿tandby鍙互纭繚鍦ㄩ泦缇ゅ嚭閿欐椂锛屽懡鍚嶇┖闂寸姸鎬佸凡缁忓畬鍏ㄥ悓姝ヤ簡

ZKFailoverController 

銆€銆€ZKFailoverController 鏄痟adoop-2.7.0涓璈DFS NameNode HA瀹炵幇鐨勪腑蹇冪粍浠讹紝瀹冭礋璐ｆ暣浣撶殑鏁呴殰杞Щ鎺у埗绛夈€傚畠鏄竴涓畧鎶よ繘绋嬶紝閫氳繃main()鏂规硶鍚姩锛岀户鎵胯嚜ZKFailoverController銆?/p>

 

resourcemanager

銆€銆€resourcemanager 鍦╕ARN涓紝ResourceManager璐熻矗闆嗙兢涓墍鏈夎祫婧愮殑缁熶竴绠＄悊鍜屽垎閰嶏紝瀹冩帴鏀舵潵鑷悇涓妭鐐癸紙NodeManager锛夌殑璧勬簮姹囨姤淇℃伅锛屽苟鎶婅繖浜涗俊鎭寜鐓т竴瀹氱殑绛栫暐鍒嗛厤缁欏悇涓簲鐢ㄧ▼搴忥紙瀹為檯涓婃槸ApplicationManager锛夈€?/p>

 

DataNode

銆€銆€DataNode 瑙ｉ噴Datanode鏄枃浠剁郴缁熺殑宸ヤ綔鑺傜偣锛屼粬浠牴鎹鎴风鎴栬€呮槸namenode鐨勮皟搴﹀瓨鍌ㄥ拰妫€绱㈡暟鎹紝骞朵笖瀹氭湡鍚憂amenode鍙戦€佷粬浠墍瀛樺偍鐨勫潡(block)鐨勫垪琛ㄣ€?/p>

 

nodemanager

銆€銆€nodemanager 鏄痀ARN涓瘡涓妭鐐逛笂鐨勪唬鐞嗭紝瀹冪鐞唄adoop闆嗙兢涓崟涓绠楄妭鐐癸紝鍖呮嫭涓嶳esourceManger淇濇寔閫氫俊锛岀洃鐫ontainer鐨勭敓鍛藉懆鏈熺鐞嗭紝鐩戞帶姣忎釜Container鐨勮祫婧愪娇鐢紙鍐呭瓨銆丆PU绛夛級鎯呭喌锛岃拷韪妭鐐瑰仴搴风姸鍐碉紝绠＄悊鏃ュ織鍜屼笉鍚屽簲鐢ㄧ▼搴忕敤鍒扮殑闄勫睘鏈嶅姟銆?/p>