锘??xml version="1.0" encoding="utf-8" standalone="yes"?>国内精品视频一区,亚洲小少妇裸体bbw,一本色道久久88综合日韩精品http://www.shnenglu.com/koson/zh-cnMon, 29 Sep 2025 22:21:04 GMTMon, 29 Sep 2025 22:21:04 GMT60ms 絎旂粡鏀扲Phttp://www.shnenglu.com/koson/archive/2010/09/27/127823.htmlkosonkosonSun, 26 Sep 2010 18:23:00 GMThttp://www.shnenglu.com/koson/archive/2010/09/27/127823.htmlhttp://www.shnenglu.com/koson/comments/127823.htmlhttp://www.shnenglu.com/koson/archive/2010/09/27/127823.html#Feedback3http://www.shnenglu.com/koson/comments/commentRss/127823.htmlhttp://www.shnenglu.com/koson/services/trackbacks/127823.html絎旇瘯鐨勫唴瀹逛笉鏄緢闅撅紝涓嬮潰鏄竴浜涜褰曪細
閫夋嫨棰橈細
1.涓や釜闀垮害鍚勪負N鐨勬湁搴忔暟緇勮繘琛屽悎騫訛紝姹傚彲鑳界殑鏈澶氱殑姣旇緝嬈℃暟錛岋紙2n-1錛?br>2.涓や釜闀垮害涓篘鐨勬湁搴忔暟緇勶紝瑕佹眰鍦ㄨ繖涓や釜鏁扮粍涓帓絎琋鐨勫厓绱狅紝鏈灝忕殑鏃墮棿澶嶆潅搴︼紵 錛?O(logn)錛岀被浼間簩鍒嗘悳绱級
3.閫嗘嘗鍏拌〃杈懼紡姹傚鹼紝錛堢珶鐒剁敾浜嗗緢涔呯殑鍚庣紑琛ㄨ揪寮忔病鐢誨嚭鏉ワ紝鐪熸澂鍏楓傘傜洿鎺ユ眰鍊煎氨琛屼簡錛?br>4.涓涓叧浜庝簩鍙夋爲鐨勯棶棰橈紝澶ф剰鏄鍦ㄤ簩鍙夋爲鏌ユ壘鏌愪釜鍏冪礌錛屾眰閫夐」緇欏嚭鐨勬煡鎵懼簭鍒楀摢涓笉鍙兘鍑虹幇錛燂紙鑰冨療浜屽弶鏍戠殑鎬ц川錛?br>5.excell鐨勫垪琛ㄧず濡侫B...Z, AA AB ....ZZ, AAA AAB .... ZZZ, 姹侱EF鐨勫崄榪涘埗鍊鹼紙姹?6榪涘埗鐨勫鹼紝鐩存帴璁$畻錛?br>6.鍑芥暟鎸囬拡鏁扮粍鐨勫啓娉曢棶棰樸傘?br>7.铏氬嚱鏁伴棶棰橈紝澶ф剰鏄熀綾誨畾涔変簡涓涓繚鎶ゆ垚鍛橈紝鏋勯犲嚱鏁板垵濮嬪寲涓?錛岃繕瀹氫箟浜嗕竴涓櫄鍑芥暟錛屽熀綾繪槸灝嗘垚鍛?-錛岃屽瓙綾誨彧閲嶅畾涔変簡铏氬嚱鏁幫紝灝嗘垚鍛?+錛屼富鍑芥暟閲岋紝new浜嗕竴涓瓙綾誨璞★紝鐒跺悗瀹氫箟涓涓熀綾繪寚閽堟寚鍚戞瀵硅薄錛屽張瀹氫箟浜嗕竴涓熀綾誨紩鐢ㄦ寚鍚戞鍩虹被鎸囬拡鎸囧悜鐨勫璞★紝鐒跺悗鍒嗗埆璋冪敤浜嗚櫄鍑芥暟錛岃姹傚熀綾誨畾涔夌殑鎴愬憳鐨勫箋?br>8.緇欏嚭涓孌電▼搴忥紝瑕佹眰杈撳嚭鍊鹼紝鐩存帴璁$畻銆傜▼搴忛噷璁$畻瀛楃鏁扮粍 char a[]={'a','b','c'}鐨勯暱搴﹂噰鐢╯izeof(a)/sizeof(a[0])鐨勬柟娉曘?br>9.鎸囧嚭緇欏嚭閫夐」涓笉鍙兘瀛樺偍鍦ㄦ爤涓殑鏄傘傘傦紙鍏ㄥ眬闈欐佸彉閲忥紝鏀懼湪闈欐佸尯涓級
10.緇欏嚭char *p="hello world", char a[]="byebye",strncpy(p,a,6),闂繖涓▼搴忚繍琛屽悗p鐨勭粨鏋滄槸浠涔堬紵(榪欓噷*p鏄竴涓瓧絎︿覆甯擱噺錛屼笉鑳藉瀹冪殑鍏冪礌榪涜淇敼錛屾墍浠ョ▼搴忓湪榪愯鏃朵細鍑洪敊)
涓昏棰樼紪紼嬮錛?br>澶ф剰鏄粰鍑轟竴涓暟緇勶紝榪欎釜鏁扮粍姣忎釜鍏冪礌閮戒笉鍚岋紝騫朵笖鍙兘鏄崌搴忕殑錛屾垨鑰呮槸鍗囧簭+鏃嬭漿鍚庣殑緇撴灉錛屼緥濡?,2,3,4,5,鎴栬?4,5,1,2,3 鎴栬?3,4,5,1,2絳夌瓑錛?br>鐒跺悗緇欎竴涓暟錛岃鎵懼嚭榪欎釜鏁板湪鎵緇欐暟緇勪腑鐨勭儲寮曞兼垨鑰呰繑鍥?1錛岃姹傚鏉傚害蹇呴』灝忎簬o(n)銆?br>鐩稿姣旇緝綆鍗曞惂錛岄鍏堟槸鍒ゆ柇鏄惁鏄粠宸﹀埌鍙蟲湁鍗囧簭鐨勶紝鑻ユ槸錛屽垯鐢ㄤ簩鍒嗘煡鎵撅紝澶嶆潅搴︿負o(logn),濡傛灉涓嶆槸錛屽垯鏍規嵁瑕佹壘鐨勫間笌絎竴涓兼瘮杈冪殑緇撴灉錛屽湪宸﹀崐閮ㄥ垎鎴栧彸鍗婇儴鍒嗘煡鎵捐繖涓暟錛屾槗鐭ワ紝鏌ユ壘嬈℃暟鑲畾灝忎簬n錛屽洜鑰屽鏉傚害絎﹀悎瑕佹眰銆?br>絎簩涓皬棰樻槸瑕佺粰鍑轟竴浜涙祴璇曟暟鎹茍鍔犱互璇存槑銆?br>姝e紡鎵懼伐鐨勭涓鍦洪潰璇曪紝涓嶆槸寰堥『鍒╋紝鐗規璁板綍錛屾敀涓婻P, ^.^



koson 2010-09-27 02:23 鍙戣〃璇勮
]]>
嫻烽噺鏁版嵁闈㈣瘯棰樻暣鐞嗭紙杞級http://www.shnenglu.com/koson/archive/2010/09/23/127442.htmlkosonkosonThu, 23 Sep 2010 12:42:00 GMThttp://www.shnenglu.com/koson/archive/2010/09/23/127442.htmlhttp://www.shnenglu.com/koson/comments/127442.htmlhttp://www.shnenglu.com/koson/archive/2010/09/23/127442.html#Feedback0http://www.shnenglu.com/koson/comments/commentRss/127442.htmlhttp://www.shnenglu.com/koson/services/trackbacks/127442.html1. 緇欏畾a銆乥涓や釜鏂囦歡錛屽悇瀛樻斁50浜夸釜url錛屾瘡涓猽rl鍚勫崰64瀛楄妭錛屽唴瀛橀檺鍒舵槸4G錛岃浣犳壘鍑篴銆乥鏂囦歡鍏卞悓鐨剈rl錛?/strong>

鏂規1錛氬彲浠ヤ及璁℃瘡涓枃浠跺畨鐨勫ぇ灝忎負50G×64=320G錛岃繙榪滃ぇ浜庡唴瀛橀檺鍒剁殑4G銆傛墍浠ヤ笉鍙兘灝嗗叾瀹屽叏鍔犺澆鍒板唴瀛樹腑澶勭悊銆傝冭檻閲囧彇鍒嗚屾不涔嬬殑鏂規硶銆?/p>

s 閬嶅巻鏂囦歡a錛屽姣忎釜url姹傚彇clip_image002錛岀劧鍚庢牴鎹墍鍙栧緱鐨勫煎皢url鍒嗗埆瀛樺偍鍒?a name=baidusnap0>1000涓皬鏂囦歡錛堣涓?a >clip_image004錛変腑銆傝繖鏍鋒瘡涓皬鏂囦歡鐨勫ぇ綰︿負300M銆?/p>

s 閬嶅巻鏂囦歡b錛岄噰鍙栧拰a鐩稿悓鐨勬柟寮忓皢url鍒嗗埆瀛樺偍鍒?strong style="BACKGROUND-COLOR: #ffff66; COLOR: black">1000鍚勫皬鏂囦歡錛堣涓?a >clip_image006錛夈傝繖鏍?strong style="BACKGROUND-COLOR: #ff66ff; COLOR: black">澶勭悊鍚庯紝鎵鏈夊彲鑳界浉鍚岀殑url閮藉湪瀵瑰簲鐨勫皬鏂囦歡錛?a >clip_image008錛変腑錛屼笉瀵瑰簲鐨勫皬鏂囦歡涓嶅彲鑳芥湁鐩稿悓鐨剈rl銆傜劧鍚庢垜浠彧瑕佹眰鍑?strong style="BACKGROUND-COLOR: #ffff66; COLOR: black">1000瀵瑰皬鏂囦歡涓浉鍚岀殑url鍗沖彲銆?/p>

s 姹傛瘡瀵瑰皬鏂囦歡涓浉鍚岀殑url鏃訛紝鍙互鎶婂叾涓竴涓皬鏂囦歡鐨剈rl瀛樺偍鍒癶ash_set涓傜劧鍚庨亶鍘嗗彟涓涓皬鏂囦歡鐨勬瘡涓猽rl錛岀湅鍏舵槸鍚﹀湪鍒氭墠鏋勫緩鐨刪ash_set涓紝濡傛灉鏄紝閭d箞灝辨槸鍏卞悓鐨剈rl錛屽瓨鍒版枃浠墮噷闈㈠氨鍙互浜嗐?/p>

鏂規2錛氬鏋滃厑璁告湁涓瀹氱殑閿欒鐜囷紝鍙互浣跨敤Bloom filter錛?G鍐呭瓨澶ф鍙互琛ㄧず340浜縝it銆傚皢鍏朵腑涓涓枃浠朵腑鐨剈rl浣跨敤Bloom filter鏄犲皠涓鴻繖340浜縝it錛岀劧鍚庢尐涓鍙栧彟澶栦竴涓枃浠剁殑url錛屾鏌ユ槸鍚︿笌Bloom filter錛屽鏋滄槸錛岄偅涔堣url搴旇鏄叡鍚岀殑url錛堟敞鎰忎細鏈変竴瀹氱殑閿欒鐜囷級銆?/p>

2. 鏈?0涓枃浠訛紝姣忎釜鏂囦歡1G錛屾瘡涓枃浠剁殑姣忎竴琛屽瓨鏀劇殑閮芥槸鐢ㄦ埛鐨剄uery錛屾瘡涓枃浠剁殑query閮藉彲鑳介噸澶嶃傝姹備綘鎸夌収query鐨勯搴︽帓搴忋?/strong>

鏂規1錛?/p>

s 欏哄簭璇誨彇10涓枃浠訛紝鎸夌収hash(query)%10鐨勭粨鏋滃皢query鍐欏叆鍒板彟澶?0涓枃浠訛紙璁頒負clip_image010錛変腑銆傝繖鏍鋒柊鐢熸垚鐨勬枃浠舵瘡涓殑澶у皬澶х害涔?G錛堝亣璁緃ash鍑芥暟鏄殢鏈虹殑錛夈?/p>

s 鎵句竴鍙板唴瀛樺湪2G宸﹀彸鐨勬満鍣紝渚濇瀵?a >clip_image010[1]鐢╤ash_map(query, query_count)鏉ョ粺璁℃瘡涓猶uery鍑虹幇鐨勬鏁般傚埄鐢ㄥ揩閫?鍫?褰掑茍鎺掑簭鎸夌収鍑虹幇嬈℃暟榪涜鎺掑簭銆傚皢鎺掑簭濂界殑query鍜屽搴旂殑query_cout杈撳嚭鍒版枃浠朵腑銆傝繖鏍峰緱鍒頒簡10涓帓濂藉簭鐨勬枃浠訛紙璁頒負clip_image012錛夈?/p>

s 瀵?a >clip_image012[1]榪?0涓枃浠惰繘琛屽綊騫舵帓搴忥紙鍐呮帓搴忎笌澶栨帓搴忕浉緇撳悎錛夈?/p>

鏂規2錛?/p>

涓鑸琿uery鐨勬婚噺鏄湁闄愮殑錛屽彧鏄噸澶嶇殑嬈℃暟姣旇緝澶氳屽凡錛屽彲鑳藉浜庢墍鏈夌殑query錛屼竴嬈℃у氨鍙互鍔犲叆鍒板唴瀛樹簡銆傝繖鏍鳳紝鎴戜滑灝卞彲浠ラ噰鐢╰rie鏍?hash_map絳夌洿鎺ユ潵緇熻姣忎釜query鍑虹幇鐨勬鏁幫紝鐒跺悗鎸夊嚭鐜版鏁板仛蹇?鍫?褰掑茍鎺掑簭灝卞彲浠ヤ簡銆?/p>

鏂規3錛?/p>

涓庢柟妗?綾諱技錛屼絾鍦ㄥ仛瀹宧ash錛屽垎鎴愬涓枃浠跺悗錛屽彲浠ヤ氦緇欏涓枃浠舵潵澶勭悊錛岄噰鐢ㄥ垎甯冨紡鐨勬灦鏋勬潵澶勭悊錛堟瘮濡侻apReduce錛夛紝鏈鍚庡啀榪涜鍚堝茍銆?/p>

3. 鏈変竴涓?G澶у皬鐨勪竴涓枃浠訛紝閲岄潰姣忎竴琛屾槸涓涓瘝錛岃瘝鐨勫ぇ灝忎笉瓚呰繃16瀛楄妭錛屽唴瀛橀檺鍒跺ぇ灝忔槸1M銆傝繑鍥為鏁版渶楂樼殑100涓瘝銆?/strong>

鏂規1錛氶『搴忚鏂囦歡涓紝瀵逛簬姣忎釜璇峹錛屽彇clip_image014錛岀劧鍚庢寜鐓ц鍊煎瓨鍒?000涓皬鏂囦歡錛堣涓?a >clip_image016錛変腑銆傝繖鏍鋒瘡涓枃浠跺ぇ姒傛槸200k宸﹀彸銆傚鏋滃叾涓殑鏈夌殑鏂囦歡瓚呰繃浜?M澶у皬錛岃繕鍙互鎸夌収綾諱技鐨?strong style="BACKGROUND-COLOR: #880000; COLOR: white">鏂規硶緇х畫寰涓嬪垎錛岀煡閬撳垎瑙e緱鍒扮殑灝忔枃浠剁殑澶у皬閮戒笉瓚呰繃1M銆傚姣忎釜灝忔枃浠訛紝緇熻姣忎釜鏂囦歡涓嚭鐜扮殑璇嶄互鍙婄浉搴旂殑棰戠巼錛堝彲浠ラ噰鐢╰rie鏍?hash_map絳夛級錛屽茍鍙栧嚭鍑虹幇棰戠巼鏈澶х殑100涓瘝錛堝彲浠ョ敤鍚?strong style="BACKGROUND-COLOR: #ffff66; COLOR: black">100涓粨鐐圭殑鏈灝忓爢錛夛紝騫舵妸100璇嶅強鐩稿簲鐨勯鐜囧瓨鍏ユ枃浠訛紝榪欐牱鍙堝緱鍒頒簡5000涓枃浠躲備笅涓姝ュ氨鏄妸榪?000涓枃浠惰繘琛屽綊騫訛紙綾諱技涓庡綊騫舵帓搴忥級鐨勮繃紼嬩簡銆?/p>

4. 嫻烽噺鏃ュ織鏁版嵁錛屾彁鍙栧嚭鏌愭棩璁塊棶鐧懼害嬈℃暟鏈澶氱殑閭d釜IP銆?/strong>

鏂規1錛氶鍏堟槸榪欎竴澶╋紝騫朵笖鏄闂櫨搴︾殑鏃ュ織涓殑IP鍙栧嚭鏉ワ紝閫愪釜鍐欏叆鍒頒竴涓ぇ鏂囦歡涓傛敞鎰忓埌IP鏄?2浣嶇殑錛屾渶澶氭湁clip_image018涓狪P銆傚悓鏍峰彲浠ラ噰鐢ㄦ槧灝勭殑鏂規硶錛屾瘮濡傛ā1000錛屾妸鏁翠釜澶ф枃浠舵槧灝勪負1000涓皬鏂囦歡錛屽啀鎵懼嚭姣忎釜灝忔枃涓嚭鐜伴鐜囨渶澶х殑IP錛堝彲浠ラ噰鐢╤ash_map榪涜棰戠巼緇熻錛岀劧鍚庡啀鎵懼嚭棰戠巼鏈澶х殑鍑犱釜錛夊強鐩稿簲鐨勯鐜囥傜劧鍚庡啀鍦ㄨ繖1000涓渶澶х殑IP涓紝鎵懼嚭閭d釜棰戠巼鏈澶х殑IP錛屽嵆涓烘墍姹傘?/p>

5. 鍦?.5浜夸釜鏁存暟涓壘鍑轟笉閲嶅鐨勬暣鏁幫紝鍐呭瓨涓嶈凍浠ュ綰寵繖2.5浜夸釜鏁存暟銆?/strong>

鏂規1錛氶噰鐢?-Bitmap錛堟瘡涓暟鍒嗛厤2bit錛?0琛ㄧず涓嶅瓨鍦紝01琛ㄧず鍑虹幇涓嬈★紝10琛ㄧず澶氭錛?1鏃犳剰涔夛級榪涜錛屽叡闇鍐呭瓨clip_image020鍐呭瓨錛岃繕鍙互鎺ュ彈銆傜劧鍚庢壂鎻忚繖2.5浜夸釜鏁存暟錛屾煡鐪婤itmap涓浉瀵瑰簲浣嶏紝濡傛灉鏄?0鍙?1錛?1鍙?0錛?0淇濇寔涓嶅彉銆傛墍鎻忓畬浜嬪悗錛屾煡鐪媌itmap錛屾妸瀵瑰簲浣嶆槸01鐨勬暣鏁拌緭鍑哄嵆鍙?/p>

鏂規2錛氫篃鍙噰鐢ㄤ笂棰樼被浼肩殑鏂規硶錛岃繘琛屽垝鍒嗗皬鏂囦歡鐨?strong style="BACKGROUND-COLOR: #880000; COLOR: white">鏂規硶銆傜劧鍚庡湪灝忔枃浠朵腑鎵懼嚭涓嶉噸澶嶇殑鏁存暟錛屽茍鎺掑簭銆傜劧鍚庡啀榪涜褰掑茍錛屾敞鎰忓幓闄ら噸澶嶇殑鍏冪礌銆?/p>

6. 嫻烽噺鏁版嵁鍒嗗竷鍦?strong style="BACKGROUND-COLOR: #ffff66; COLOR: black">100鍙扮數鑴戜腑錛屾兂涓姙娉曢珮鏍$粺璁″嚭榪欐壒鏁版嵁鐨凾OP10銆?/strong>

鏂規1錛?/p>

s 鍦ㄦ瘡鍙扮數鑴戜笂姹傚嚭TOP10錛屽彲浠ラ噰鐢ㄥ寘鍚?0涓厓绱犵殑鍫嗗畬鎴愶紙TOP10灝忥紝鐢ㄦ渶澶у爢錛孴OP10澶э紝鐢ㄦ渶灝忓爢錛夈傛瘮濡傛眰TOP10澶э紝鎴戜滑棣栧厛鍙栧墠10涓厓绱犺皟鏁存垚鏈灝忓爢錛屽鏋滃彂鐜幫紝鐒跺悗鎵弿鍚庨潰鐨勬暟鎹紝騫朵笌鍫嗛《鍏冪礌姣旇緝錛屽鏋滄瘮鍫嗛《鍏冪礌澶э紝閭d箞鐢ㄨ鍏冪礌鏇挎崲鍫嗛《錛岀劧鍚庡啀璋冩暣涓烘渶灝忓爢銆傛渶鍚庡爢涓殑鍏冪礌灝辨槸TOP10澶с?/p>

s 姹傚嚭姣忓彴鐢佃剳涓婄殑TOP10鍚庯紝鐒跺悗鎶婅繖100鍙扮數鑴戜笂鐨凾OP10緇勫悎璧鋒潵錛屽叡1000涓暟鎹紝鍐嶅埄鐢ㄤ笂闈㈢被浼肩殑鏂規硶姹傚嚭TOP10灝卞彲浠ヤ簡銆?/p>

7. 鎬庝箞鍦ㄦ搗閲忔暟鎹腑鎵懼嚭閲嶅嬈℃暟鏈澶氱殑涓涓紵

鏂規1錛氬厛鍋歨ash錛岀劧鍚庢眰妯℃槧灝勪負灝忔枃浠訛紝姹傚嚭姣忎釜灝忔枃浠朵腑閲嶅嬈℃暟鏈澶氱殑涓涓紝騫惰褰曢噸澶嶆鏁般傜劧鍚庢壘鍑轟笂涓姝ユ眰鍑虹殑鏁版嵁涓噸澶嶆鏁版渶澶氱殑涓涓氨鏄墍姹傦紙鍏蜂綋鍙傝冨墠闈㈢殑棰橈級銆?/p>

8. 涓婂崈涓囨垨涓婁嚎鏁版嵁錛堟湁閲嶅錛夛紝緇熻鍏朵腑鍑虹幇嬈℃暟鏈澶氱殑閽盢涓暟鎹?/strong>

鏂規1錛氫笂鍗冧竾鎴栦笂浜跨殑鏁版嵁錛岀幇鍦ㄧ殑鏈哄櫒鐨勫唴瀛樺簲璇ヨ兘瀛樹笅銆傛墍浠ヨ冭檻閲囩敤hash_map/鎼滅儲浜屽弶鏍?綰㈤粦鏍戠瓑鏉ヨ繘琛岀粺璁℃鏁般傜劧鍚庡氨鏄彇鍑哄墠N涓嚭鐜版鏁版渶澶氱殑鏁版嵁浜嗭紝鍙互鐢ㄧ6棰樻彁鍒扮殑鍫嗘満鍒跺畬鎴愩?/p>

9. 1000涓囧瓧絎︿覆錛屽叾涓湁浜涙槸閲嶅鐨勶紝闇瑕佹妸閲嶅鐨勫叏閮ㄥ幓鎺夛紝淇濈暀娌℃湁閲嶅鐨勫瓧絎︿覆銆傝鎬庝箞璁捐鍜屽疄鐜幫紵

鏂規1錛氳繖棰樼敤trie鏍戞瘮杈冨悎閫傦紝hash_map涔熷簲璇ヨ兘琛屻?/p>

10. 涓涓枃鏈枃浠訛紝澶х害鏈変竴涓囪錛屾瘡琛屼竴涓瘝錛岃姹傜粺璁″嚭鍏朵腑鏈棰戠箒鍑虹幇鐨勫墠10涓瘝錛岃緇欏嚭鎬濇兂錛岀粰鍑烘椂闂村鏉傚害鍒嗘瀽銆?/strong>

鏂規1錛氳繖棰樻槸鑰冭檻鏃墮棿鏁堢巼銆傜敤trie鏍戠粺璁℃瘡涓瘝鍑虹幇鐨勬鏁幫紝鏃墮棿澶嶆潅搴︽槸O(n*le)錛坙e琛ㄧず鍗曡瘝鐨勫鉤鍑嗛暱搴︼級銆傜劧鍚庢槸鎵懼嚭鍑虹幇鏈棰戠箒鐨勫墠10涓瘝錛屽彲浠ョ敤鍫嗘潵瀹炵幇錛屽墠闈㈢殑棰樹腑宸茬粡璁插埌浜嗭紝鏃墮棿澶嶆潅搴︽槸O(n*lg10)銆傛墍浠ユ葷殑鏃墮棿澶嶆潅搴︼紝鏄疧(n*le)涓嶰(n*lg10)涓緝澶х殑鍝竴涓?/p>

11. 涓涓枃鏈枃浠訛紝鎵懼嚭鍓?0涓粡甯稿嚭鐜扮殑璇嶏紝浣嗚繖嬈℃枃浠舵瘮杈冮暱錛岃鏄笂浜胯鎴栧崄浜胯錛屾諱箣鏃犳硶涓嬈¤鍏ュ唴瀛橈紝闂渶浼樿В銆?/strong>

鏂規1錛氶鍏堟牴鎹敤hash騫舵眰妯★紝灝嗘枃浠跺垎瑙d負澶氫釜灝忔枃浠訛紝瀵逛簬鍗曚釜鏂囦歡鍒╃敤涓婇鐨?strong style="BACKGROUND-COLOR: #880000; COLOR: white">鏂規硶姹傚嚭姣忎釜鏂囦歡浠朵腑10涓渶甯稿嚭鐜扮殑璇嶃傜劧鍚庡啀榪涜褰掑茍澶勭悊錛屾壘鍑烘渶緇堢殑10涓渶甯稿嚭鐜扮殑璇嶃?/p>

12. 100w涓暟涓壘鍑烘渶澶х殑100涓暟銆?/strong>

鏂規1錛氬湪鍓嶉潰鐨勯涓紝鎴戜滑宸茬粡鎻愬埌浜嗭紝鐢ㄤ竴涓惈100涓厓绱犵殑鏈灝忓爢瀹屾垚銆傚鏉傚害涓篛(100w*lg100)銆?/p>

鏂規2錛氶噰鐢ㄥ揩閫熸帓搴忕殑鎬濇兂錛屾瘡嬈″垎鍓蹭箣鍚庡彧鑰冭檻姣旇醬澶х殑涓閮ㄥ垎錛岀煡閬撴瘮杞村ぇ鐨勪竴閮ㄥ垎鍦ㄦ瘮100澶氱殑鏃跺欙紝閲囩敤浼犵粺鎺掑簭綆楁硶鎺掑簭錛屽彇鍓?strong style="BACKGROUND-COLOR: #ffff66; COLOR: black">100涓傚鏉傚害涓篛(100w*100)銆?/p>

鏂規3錛氶噰鐢ㄥ眬閮ㄦ窐姹版硶銆傞夊彇鍓?strong style="BACKGROUND-COLOR: #ffff66; COLOR: black">100涓厓绱狅紝騫舵帓搴忥紝璁頒負搴忓垪L銆傜劧鍚庝竴嬈℃壂鎻忓墿浣欑殑鍏冪礌x錛屼笌鎺掑ソ搴忕殑100涓厓绱犱腑鏈灝忕殑鍏冪礌姣旓紝濡傛灉姣旇繖涓渶灝忕殑瑕佸ぇ錛岄偅涔堟妸榪欎釜鏈灝忕殑鍏冪礌鍒犻櫎錛屽茍鎶妜鍒╃敤鎻掑叆鎺掑簭鐨勬濇兂錛屾彃鍏ュ埌搴忓垪L涓備緷嬈″驚鐜紝鐭ラ亾鎵弿浜嗘墍鏈夌殑鍏冪礌銆傚鏉傚害涓篛(100w*100)銆?/p>

13. 瀵繪壘鐑棬鏌ヨ錛?/strong>

鎼滅儲寮曟搸浼氶氳繃鏃ュ織鏂囦歡鎶婄敤鎴鋒瘡嬈℃绱嬌鐢ㄧ殑鎵鏈夋绱覆閮借褰曚笅鏉ワ紝姣忎釜鏌ヨ涓茬殑闀垮害涓?-255瀛楄妭銆傚亣璁劇洰鍓嶆湁涓鍗冧竾涓褰曪紝榪欎簺鏌ヨ涓茬殑閲嶅璇繪瘮杈冮珮錛岃櫧鐒舵繪暟鏄?鍗冧竾錛屼絾鏄鏋滃幓闄ら噸澶嶅拰錛屼笉瓚呰繃3鐧句竾涓備竴涓煡璇覆鐨勯噸澶嶅害瓚婇珮錛岃鏄庢煡璇㈠畠鐨勭敤鎴瘋秺澶氾紝涔熷氨瓚婄儹闂ㄣ傝浣犵粺璁℃渶鐑棬鐨?0涓煡璇覆錛岃姹備嬌鐢ㄧ殑鍐呭瓨涓嶈兘瓚呰繃1G銆?/p>

(1) 璇鋒弿榪頒綘瑙e喅榪欎釜闂鐨勬濊礬錛?/p>

(2) 璇風粰鍑轟富瑕佺殑澶勭悊嫻佺▼錛岀畻娉曪紝浠ュ強綆楁硶鐨勫鏉傚害銆?/p>

鏂規1錛氶噰鐢╰rie鏍戯紝鍏抽敭瀛楀煙瀛樿鏌ヨ涓插嚭鐜扮殑嬈℃暟錛屾病鏈夊嚭鐜頒負0銆傛渶鍚庣敤10涓厓绱犵殑鏈灝忔帹鏉ュ鍑虹幇棰戠巼榪涜鎺掑簭銆?/p>

14. 涓鍏辨湁N涓満鍣紝姣忎釜鏈哄櫒涓婃湁N涓暟銆傛瘡涓満鍣ㄦ渶澶氬瓨O(N)涓暟騫跺瀹冧滑鎿嶄綔銆傚浣曟壘鍒?a >clip_image022涓暟涓殑涓暟錛?/strong>

鏂規1錛氬厛澶т綋浼拌涓涓嬭繖浜涙暟鐨勮寖鍥達紝姣斿榪欓噷鍋囪榪欎簺鏁伴兘鏄?2浣嶆棤絎﹀彿鏁存暟錛堝叡鏈?a >clip_image018[1]涓級銆傛垜浠妸0鍒?a >clip_image024鐨勬暣鏁板垝鍒嗕負N涓寖鍥存錛屾瘡涓鍖呭惈clip_image026涓暣鏁般傛瘮濡傦紝絎竴涓浣?鍒?a >clip_image028錛岀浜屾涓?a >clip_image026[1]鍒?a >clip_image030錛?#8230;錛岀N涓涓?a >clip_image032鍒?a >clip_image024[1]銆傜劧鍚庯紝鎵弿姣忎釜鏈哄櫒涓婄殑N涓暟錛屾妸灞炰簬絎竴涓尯孌電殑鏁版斁鍒扮涓涓満鍣ㄤ笂錛屽睘浜庣浜屼釜鍖烘鐨勬暟鏀懼埌絎簩涓満鍣ㄤ笂錛?#8230;錛屽睘浜庣N涓尯孌電殑鏁版斁鍒扮N涓満鍣ㄤ笂銆傛敞鎰忚繖涓繃紼嬫瘡涓満鍣ㄤ笂瀛樺偍鐨勬暟搴旇鏄疧(N)鐨勩備笅闈㈡垜浠緷嬈$粺璁℃瘡涓満鍣ㄤ笂鏁扮殑涓暟錛屼竴嬈$瘡鍔狅紝鐩村埌鎵懼埌絎琸涓満鍣紝鍦ㄨ鏈哄櫒涓婄瘡鍔犵殑鏁板ぇ浜庢垨絳変簬clip_image034錛岃屽湪絎琸-1涓満鍣ㄤ笂鐨勭瘡鍔犳暟灝忎簬clip_image034[1]錛屽茍鎶婅繖涓暟璁頒負x銆傞偅涔堟垜浠鎵劇殑涓綅鏁板湪絎琸涓満鍣ㄤ腑錛屾帓鍦ㄧclip_image036浣嶃傜劧鍚庢垜浠絎琸涓満鍣ㄧ殑鏁版帓搴忥紝騫舵壘鍑虹clip_image036[1]涓暟錛屽嵆涓烘墍姹傜殑涓綅鏁般傚鏉傚害鏄?a >clip_image038鐨勩?/p>

鏂規2錛氬厛瀵規瘡鍙版満鍣ㄤ笂鐨勬暟榪涜鎺掑簭銆傛帓濂藉簭鍚庯紝鎴戜滑閲囩敤褰掑茍鎺掑簭鐨勬濇兂錛屽皢榪橬涓満鍣ㄤ笂鐨勬暟褰掑茍璧鋒潵寰楀埌鏈緇堢殑鎺掑簭銆傛壘鍒扮clip_image034[2]涓究鏄墍姹傘傚鏉傚害鏄?a >clip_image040鐨勩?/p>

15. 鏈澶ч棿闅欓棶棰?/strong>

緇欏畾n涓疄鏁?a >clip_image042錛屾眰鐫n涓疄鏁板湪瀹炶醬涓婂悜閲?涓暟涔嬮棿鐨勬渶澶у樊鍊鹼紝瑕佹眰綰挎х殑鏃墮棿綆楁硶銆?/p>

鏂規1錛氭渶鍏堟兂鍒扮殑鏂規硶灝辨槸鍏堝榪檔涓暟鎹繘琛屾帓搴忥紝鐒跺悗涓閬嶆壂鎻忓嵆鍙‘瀹氱浉閭葷殑鏈澶ч棿闅欍備絾璇?strong style="BACKGROUND-COLOR: #880000; COLOR: white">鏂規硶涓嶈兘婊¤凍綰挎ф椂闂寸殑瑕佹眰銆傛晠閲囧彇濡備笅鏂規硶錛?/p>

s 鎵懼埌n涓暟鎹腑鏈澶у拰鏈灝忔暟鎹甿ax鍜宮in銆?/p>

s 鐢╪-2涓偣絳夊垎鍖洪棿[min, max]錛屽嵆灝哰min, max]絳夊垎涓簄-1涓尯闂達紙鍓嶉棴鍚庡紑鍖洪棿錛夛紝灝嗚繖浜涘尯闂寸湅浣滄《錛岀紪鍙蜂負clip_image044錛屼笖妗?a >clip_image046鐨勪笂鐣屽拰妗秈+1鐨勪笅灞婄浉鍚岋紝鍗蟲瘡涓《鐨勫ぇ灝忕浉鍚屻傛瘡涓《鐨勫ぇ灝忎負錛?a >clip_image048銆傚疄闄呬笂錛岃繖浜涙《鐨勮竟鐣屾瀯鎴愪簡涓涓瓑宸暟鍒楋紙棣栭」涓簃in錛屽叕宸負clip_image050錛夛紝涓旇涓哄皢min鏀懼叆絎竴涓《錛屽皢max鏀懼叆絎琻-1涓《銆?/p>

s 灝唍涓暟鏀懼叆n-1涓《涓細灝嗘瘡涓厓绱?a >clip_image052鍒嗛厤鍒版煇涓《錛堢紪鍙蜂負index錛夛紝鍏朵腑clip_image054錛屽茍姹傚嚭鍒嗗埌姣忎釜妗剁殑鏈澶ф渶灝忔暟鎹?/p>

s 鏈澶ч棿闅欙細闄ゆ渶澶ф渶灝忔暟鎹甿ax鍜宮in浠ュ鐨刵-2涓暟鎹斁鍏-1涓《涓紝鐢辨娊灞夊師鐞嗗彲鐭ヨ嚦灝戞湁涓涓《鏄┖鐨勶紝鍙堝洜涓烘瘡涓《鐨勫ぇ灝忕浉鍚岋紝鎵浠ユ渶澶ч棿闅欎笉浼氬湪鍚屼竴妗朵腑鍑虹幇錛屼竴瀹氭槸鏌愪釜妗剁殑涓婄晫鍜屾皵鍊欐煇涓《鐨勪笅鐣屼箣闂撮殭錛屼笖璇ラ噺絳掍箣闂寸殑妗訛紙鍗充究濂藉湪璇ヨ繛涓究濂戒箣闂寸殑妗訛級涓瀹氭槸絀烘《銆備篃灝辨槸璇達紝鏈澶ч棿闅欏湪妗秈鐨勪笂鐣屽拰妗秊鐨勪笅鐣屼箣闂翠駭鐢?a >clip_image056銆備竴閬嶆壂鎻忓嵆鍙畬鎴愩?/p>

16. 灝嗗涓泦鍚堝悎騫舵垚娌℃湁浜ら泦鐨勯泦鍚堬細緇欏畾涓涓瓧絎︿覆鐨勯泦鍚堬紝鏍煎紡濡傦細clip_image058銆傝姹傚皢鍏朵腑浜ら泦涓嶄負絀虹殑闆嗗悎鍚堝茍錛岃姹傚悎騫跺畬鎴愮殑闆嗗悎涔嬮棿鏃犱氦闆嗭紝渚嬪涓婁緥搴旇緭鍑?a >clip_image060銆?/strong>

(1) 璇鋒弿榪頒綘瑙e喅榪欎釜闂鐨勬濊礬錛?/p>

(2) 緇欏嚭涓昏鐨?strong style="BACKGROUND-COLOR: #ff66ff; COLOR: black">澶勭悊嫻佺▼錛岀畻娉曪紝浠ュ強綆楁硶鐨勫鏉傚害錛?/p>

(3) 璇鋒弿榪板彲鑳界殑鏀硅繘銆?/p>

鏂規1錛氶噰鐢ㄥ茍鏌ラ泦銆傞鍏堟墍鏈夌殑瀛楃涓查兘鍦ㄥ崟鐙殑騫舵煡闆嗕腑銆傜劧鍚庝緷鎵弿姣忎釜闆嗗悎錛岄『搴忓悎騫跺皢涓や釜鐩擱偦鍏冪礌鍚堝茍銆備緥濡傦紝瀵逛簬clip_image062錛岄鍏堟煡鐪媋aa鍜宐bb鏄惁鍦ㄥ悓涓涓茍鏌ラ泦涓紝濡傛灉涓嶅湪錛岄偅涔堟妸瀹冧滑鎵鍦ㄧ殑騫舵煡闆嗗悎騫訛紝鐒跺悗鍐嶇湅bbb鍜宑cc鏄惁鍦ㄥ悓涓涓茍鏌ラ泦涓紝濡傛灉涓嶅湪錛岄偅涔堜篃鎶婂畠浠墍鍦ㄧ殑騫舵煡闆嗗悎騫躲傛帴涓嬫潵鍐嶆壂鎻忓叾浠栫殑闆嗗悎錛屽綋鎵鏈夌殑闆嗗悎閮芥壂鎻忓畬浜嗭紝騫舵煡闆嗕唬琛ㄧ殑闆嗗悎渚挎槸鎵姹傘傚鏉傚害搴旇鏄疧(NlgN)鐨勩傛敼榪涚殑璇濓紝棣栧厛鍙互璁板綍姣忎釜鑺傜偣鐨勬牴緇撶偣錛屾敼榪涙煡璇€傚悎騫剁殑鏃跺欙紝鍙互鎶婂ぇ鐨勫拰灝忕殑榪涜鍚堬紝榪欐牱涔熷噺灝戝鏉傚害銆?/p>

17. 鏈澶у瓙搴忓垪涓庢渶澶у瓙鐭╅樀闂

鏁扮粍鐨勬渶澶у瓙搴忓垪闂錛氱粰瀹氫竴涓暟緇勶紝鍏朵腑鍏冪礌鏈夋錛屼篃鏈夎礋錛屾壘鍑哄叾涓竴涓繛緇瓙搴忓垪錛屼嬌鍜屾渶澶с?/p>

鏂規1錛氳繖涓棶棰樺彲浠ュ姩鎬佽鍒掔殑鎬濇兂瑙e喅銆傝clip_image064琛ㄧず浠ョi涓厓绱?a >clip_image066緇撳熬鐨勬渶澶у瓙搴忓垪錛岄偅涔堟樉鐒?a >clip_image068銆傚熀浜庤繖涓鐐瑰彲浠ュ緢蹇敤浠g爜瀹炵幇銆?/p>

鏈澶у瓙鐭╅樀闂錛氱粰瀹氫竴涓煩闃碉紙浜岀淮鏁扮粍錛夛紝鍏朵腑鏁版嵁鏈夊ぇ鏈夊皬錛岃鎵句竴涓瓙鐭╅樀錛屼嬌寰楀瓙鐭╅樀鐨勫拰鏈澶э紝騫惰緭鍑鴻繖涓拰銆?/p>

鏂規1錛氬彲浠ラ噰鐢ㄤ笌鏈澶у瓙搴忓垪綾諱技鐨勬濇兂鏉ヨВ鍐熾傚鏋滄垜浠‘瀹氫簡閫夋嫨絎琲鍒楀拰絎琷鍒椾箣闂寸殑鍏冪礌錛岄偅涔堝湪榪欎釜鑼冨洿鍐咃紝鍏跺疄灝辨槸涓涓渶澶у瓙搴忓垪闂銆傚浣曠‘瀹氱i鍒楀拰絎琷鍒楀彲浠ヨ瘝鐢ㄦ毚鎼滅殑鏂規硶榪涜銆?/p>

koson 2010-09-23 20:42 鍙戣〃璇勮
]]>
struct緇撴瀯浣撶殑鏁版嵁瀵歸綈http://www.shnenglu.com/koson/archive/2010/09/22/127341.htmlkosonkosonWed, 22 Sep 2010 06:49:00 GMThttp://www.shnenglu.com/koson/archive/2010/09/22/127341.htmlhttp://www.shnenglu.com/koson/comments/127341.htmlhttp://www.shnenglu.com/koson/archive/2010/09/22/127341.html#Feedback0http://www.shnenglu.com/koson/comments/commentRss/127341.htmlhttp://www.shnenglu.com/koson/services/trackbacks/127341.htmlstruct
{
   char a;
   int b;
   short c;
};
姣忎釜鎴愬憳璧峰鍦板潃蹇呴』涓虹浉搴旂被鍨嬬殑澶у皬鐨勫嶆暟錛屽a,璧峰鍦板潃涓?錛屼負sizeof(char)=1鐨勫嶆暟錛岃屽埌b鐨勬椂鍊欙紝鍥犱負榪欐椂鍦板潃涓?錛屼笉鏄痵izeof(int)=4鐨勫嶆暟錛屾墍浠ョ紪璇戝櫒浼氭坊鍔?涓瓧鑺傚ぇ灝忥紝姝ゆ椂鍦板潃涓?錛屼綔涓篵鐨勮搗濮嬪湴鍧錛宐鍗犵敤4涓瓧鑺傦紝鎵浠ユ鏃跺唴瀛樺湴鍧涓?錛?鏄痵izeof(short)=2鐨勫嶆暟錛屾墍浠鐨勮搗濮嬪湴鍧涓?錛屽崰涓や釜瀛楄妭錛屾墍浠ユ鏃舵葷殑鍒嗛厤鍐呭瓨澶у皬涓?0錛屼絾鏄洜涓烘暣涓粨鏋勪綋鐨勬渶澶ф垚鍛樹負sizeof(int)=4,鑰?0涓嶄負4鐨勫嶆暟錛屾墍浠ヨ娣誨姞涓や釜瀛楄妭涓?2.
浣跨敤#pragma pack(n)鍙互鎸囧畾緙栬瘧鍣ㄦ寜n涓瓧鑺備綔涓哄榻愭柟寮忥紝榪欎釜鎸囧畾鍙閭d簺澶т簬n涓瓧鑺傜殑鎴愬憳鏈夋晥錛屽涓婁緥錛岃嫢鎸囧畾浜嗕互2涓瓧鑺備綔涓哄榻愶紝鍒檃鐨勮搗濮嬪湴鍧涓?錛屾帴鐫鍒癰,鍥犱負sizeof(int)=4 < 2,鎵浠ヨ鎸囧畾鐨勫榻愭柟寮忥紝姝ゆ椂鍦板潃涓?錛屼笉涓?鐨勫嶆暟錛屾墍浠ヨ鍔?錛屾墍浠鐨勮搗濮嬪湴鍧涓?錛屽崰鐢?涓瓧鑺傚ぇ灝忎箣鍚庡湴鍧涓?錛屾帴鐫鏄痗錛屽洜涓?鏄?鐨勫嶆暟錛屾墍浠?涓篶鐨勮搗濮嬪湴鍧錛屽崰鐢ㄤ袱涓瓧鑺傦紝鎬葷殑澶у皬涓?



koson 2010-09-22 14:49 鍙戣〃璇勮
]]>
XP涓媀irtualbox铏氭嫙Ubuntu鍏變韓鏂囦歡澶硅緗紙杞級http://www.shnenglu.com/koson/archive/2010/09/15/126626.htmlkosonkosonTue, 14 Sep 2010 16:57:00 GMThttp://www.shnenglu.com/koson/archive/2010/09/15/126626.htmlhttp://www.shnenglu.com/koson/comments/126626.htmlhttp://www.shnenglu.com/koson/archive/2010/09/15/126626.html#Feedback0http://www.shnenglu.com/koson/comments/commentRss/126626.htmlhttp://www.shnenglu.com/koson/services/trackbacks/126626.html 

    1. 瀹夎澧炲己鍔熻兘鍖?Guest Additions)

    瀹夎濂?u>ubuntu鍚庯紝榪愯Ubuntu騫剁櫥褰曘傜劧鍚庡湪VirtualBox鐨勮彍鍗曢噷閫夋嫨"璁懼(Devices)" -> "瀹夎澧炲己鍔熻兘鍖?Install Guest Additions)"銆?/p>

    浣犱細鍙戠幇鍦║buntu妗岄潰涓婂鍑轟竴涓厜鐩樺浘鏍囷紝榪欏紶鍏夌洏榛樿琚嚜鍔ㄥ姞杞藉埌浜嗘枃浠跺す/media/cdom0銆傝繘鍏ュ懡浠よ緇堢錛岃緭鍏ワ細

    cd /media/cdom0

    sudo ./VboxLinuxAdditions.run

    寮濮嬪畨瑁呭伐鍏峰寘銆傚畨瑁呭畬姣曞悗浼氭彁紺鴻閲嶅惎Ubuntu銆?/p>

    2. 璁劇疆鍏變韓鏂囦歡澶?/p>

    閲嶅惎瀹屾垚鍚庣偣鍑?璁懼(Devices)" -> 鍏變韓鏂囦歡澶?Shared Folders)鑿滃崟錛屾坊鍔犱竴涓叡浜枃浠跺す錛岄夐」鍥哄畾鍜屼復鏃舵槸鎸囪鏂囦歡澶規槸鍚︽槸鎸佷箙鐨勩傚叡浜悕鍙互浠誨彇涓涓嚜宸卞枩嬈㈢殑錛屾瘮濡?gongxiang"錛屽敖閲忎嬌鐢ㄨ嫳鏂囧悕縐般?/p>

    3. 鎸傝澆鍏變韓鏂囦歡澶?/p>

    閲嶆柊榪涘叆铏氭嫙Ubuntu錛屽湪鍛戒護琛岀粓绔笅杈撳叆錛?/p>

    sudo mkdir /mnt/shared

    sudo mount -t vboxsf gongxiang /mnt/shared

    鍏朵腑"gongxiang"鏄箣鍓嶅垱寤虹殑鍏變韓鏂囦歡澶圭殑鍚嶅瓧銆侽K錛岀幇鍦║buntu鍜屼富鏈哄彲浠ヤ簰浼犳枃浠朵簡銆?/p>

    鍋囧鎮ㄤ笉鎯蟲瘡涓嬈¢兘鎵嬪姩鎸傝澆錛屽彲浠ュ湪/etc/fstab涓坊鍔犱竴欏?/p>

    gongxiang /mnt/shared vboxsf rw,gid=100,uid=1000,auto 0 0

    榪欐牱灝辮兘澶熻嚜鍔ㄦ寕杞戒簡銆?/p>

    4. 鍗歌澆鐨勮瘽浣跨敤涓嬮潰鐨勫懡浠わ細

    sudo umount -f /mnt/shared

    娉ㄦ剰錛?/p>

    鍏變韓鏂囦歡澶圭殑鍚嶇О鍗冧竾涓嶈鍜屾寕杞界偣鐨勫悕縐扮浉鍚屻傛瘮濡傦紝涓婇潰鐨勬寕杞界偣鏄?mnt/shared錛屽鏋滃叡浜枃浠跺す鐨勫悕瀛椾篃鏄痵hared鐨勮瘽錛屽湪鎸傝澆鐨勬椂鍊欏氨浼氬嚭鐜板涓嬬殑閿欒淇℃伅(鐪?a >http://www.virtualbox.org/ticket/2265)錛?/p>

    /sbin/mount.vboxsf: mounting failed with the error: Protocol error

    鍘熷洜鍒嗘瀽鍙互鐪婽ips on running Sun Virtualbox鐨凷hared Folder on a Linux Guest鑺傘?/p>

koson 2010-09-15 00:57 鍙戣〃璇勮
]]>
linux涓媡ar gz bz2 tgz z絳変紬澶氬帇緙╂枃浠剁殑瑙e帇鏂規硶錛堣漿錛?/title><link>http://www.shnenglu.com/koson/archive/2010/09/15/126625.html</link><dc:creator>koson</dc:creator><author>koson</author><pubDate>Tue, 14 Sep 2010 16:56:00 GMT</pubDate><guid>http://www.shnenglu.com/koson/archive/2010/09/15/126625.html</guid><wfw:comment>http://www.shnenglu.com/koson/comments/126625.html</wfw:comment><comments>http://www.shnenglu.com/koson/archive/2010/09/15/126625.html#Feedback</comments><slash:comments>0</slash:comments><wfw:commentRss>http://www.shnenglu.com/koson/comments/commentRss/126625.html</wfw:commentRss><trackback:ping>http://www.shnenglu.com/koson/services/trackbacks/126625.html</trackback:ping><description><![CDATA[<p>鍦ㄥ叿浣撴葷粨鍚勭被鍘嬬緝鏂囦歡涔嬪墠鍛紝棣栧厛瑕佸紕娓呬袱涓蹇碉細鎵撳寘鍜屽帇緙┿傛墦鍖呮槸鎸囧皢涓澶у爢鏂囦歡鎴栫洰褰曚粈涔堢殑鍙樻垚涓涓葷殑鏂囦歡錛屽帇緙╁垯鏄皢涓涓ぇ鐨勬枃浠墮氳繃涓浜涘帇緙╃畻娉曞彉鎴愪竴涓皬鏂囦歡銆備負浠涔堣鍖哄垎榪欎袱涓蹇靛憿錛熷叾瀹炶繖婧愪簬Linux涓殑寰堝鍘嬬緝紼嬪簭鍙兘閽堝涓涓枃浠惰繘琛屽帇緙╋紝榪欐牱褰撲綘鎯寵鍘嬬緝涓澶у爢鏂囨椂錛屼綘灝卞緱鍏堝熷姪鍙﹀畠鐨勫伐鍏峰皢榪欎竴澶у爢鏂囦歡鍏堟墦鎴愪竴涓寘錛岀劧鍚庡啀灝卞師鏉ョ殑鍘嬬緝紼嬪簭榪涜鍘嬬緝銆?br>銆銆Linux涓嬫渶甯哥敤鐨勬墦鍖呯▼搴忓氨鏄痶ar浜嗭紝浣跨敤tar紼嬪簭鎵撳嚭鏉ョ殑鍖呮垜浠父縐頒負tar鍖咃紝tar鍖呮枃浠剁殑鍛戒護閫氬父閮芥槸浠?tar緇撳熬鐨勩傜敓鎴恡ar鍖呭悗錛屽氨鍙互鐢ㄥ叾瀹冪殑紼嬪簭鏉ヨ繘琛屽帇緙╀簡錛屾墍浠ラ鍏堝氨鏉ヨ璁瞭ar鍛戒護鐨勫熀鏈敤娉曪細<br>銆銆tar鍛戒護鐨勯夐」鏈夊緢澶?鐢╩an tar鍙互鏌ョ湅鍒?錛屼絾甯哥敤鐨勫氨閭d箞鍑犱釜閫夐」錛屼笅闈㈡潵涓句緥璇存槑涓涓嬶細<br>       # tar -cf all.tar *.jpg<br>銆銆榪欐潯鍛戒護鏄皢鎵鏈?jpg鐨勬枃浠舵墦鎴愪竴涓悕涓篴ll.tar鐨勫寘銆?c鏄〃紺轟駭鐢熸柊鐨勫寘錛?f鎸囧畾鍖呯殑鏂囦歡鍚嶃?br>       # tar -rf all.tar *.gif<br>銆銆榪欐潯鍛戒護鏄皢鎵鏈?gif鐨勬枃浠跺鍔犲埌all.tar鐨勫寘閲岄潰鍘匯?r鏄〃紺哄鍔犳枃浠剁殑鎰忔濄?br>       # tar -uf all.tar logo.gif<br>銆銆榪欐潯鍛戒護鏄?鏇存柊鍘熸潵tar鍖卆ll.tar涓璴ogo.gif鏂囦歡錛?u鏄〃紺烘洿鏂版枃浠剁殑鎰忔濄?br>       # tar -tf all.tar<br>銆銆榪欐潯鍛戒護鏄垪鍑篴ll.tar鍖呬腑鎵鏈夋枃浠訛紝-t鏄垪鍑烘枃浠剁殑鎰忔?br>       # tar -xf all.tar<br>銆銆榪欐潯鍛戒護鏄В鍑篴ll.tar鍖呬腑鎵鏈夋枃浠訛紝-x鏄В寮鐨勬剰鎬?br>       浠ヤ笂灝辨槸tar鐨勬渶鍩烘湰鐨勭敤娉曘備負浜嗘柟渚跨敤鎴峰湪鎵撳寘瑙e寘鐨勫悓鏃跺彲浠ュ帇緙╂垨瑙e帇鏂囦歡錛宼ar鎻愪緵浜嗕竴縐嶇壒孌婄殑鍔熻兘銆傝繖灝辨槸tar鍙互鍦ㄦ墦鍖呮垨瑙e寘鐨勫悓鏃惰皟鐢ㄥ叾瀹冪殑鍘嬬緝紼嬪簭錛屾瘮濡傝皟鐢╣zip銆乥zip2絳夈?br>      1) tar璋冪敤gzip<br>       gzip鏄疓NU緇勭粐寮鍙戠殑涓涓帇緙╃▼搴忥紝.gz緇撳熬鐨勬枃浠跺氨鏄痝zip鍘嬬緝鐨勭粨鏋溿備笌gzip鐩稿鐨勮В鍘嬬▼搴廹unzip銆倀ar涓嬌鐢?z榪欎釜鍙傛暟鏉ヨ皟鐢╣zip銆備笅闈㈡潵涓句緥璇存槑涓涓嬶細<br>       # tar -czf all.tar.gz *.jpg<br>銆銆榪欐潯鍛戒護鏄皢鎵鏈?jpg鐨勬枃浠舵墦鎴愪竴涓猼ar鍖咃紝騫朵笖灝嗗叾鐢╣zip鍘嬬緝錛岀敓鎴愪竴涓猤zip鍘嬬緝榪囩殑鍖咃紝鍖呭悕all.tar.gz<br>       # tar -xzf all.tar.gz<br>銆銆榪欐潯鍛戒護鏄皢涓婇潰浜х敓鐨勫寘瑙e紑銆?br>       2) tar璋冪敤bzip2<br>       bzip2鏄竴涓帇緙╄兘鍔涙洿寮虹殑鍘嬬緝紼嬪簭錛?bz2緇撳熬鐨勬枃浠跺氨鏄痓zip2鍘嬬緝鐨勭粨鏋溿備笌bzip2鐩稿鐨勮В鍘嬬▼搴忔槸bunzip2銆倀ar涓嬌鐢?j榪欎釜鍙傛暟鏉ヨ皟鐢╣zip銆備笅闈㈡潵涓句緥璇存槑涓涓嬶細<br>       # tar -cjf all.tar.bz2 *.jpg<br>銆銆榪欐潯鍛戒護鏄皢鎵鏈?jpg鐨勬枃浠舵墦鎴愪竴涓猼ar鍖咃紝騫朵笖灝嗗叾鐢╞zip2鍘嬬緝錛岀敓鎴愪竴涓猙zip2鍘嬬緝榪囩殑鍖咃紝鍖呭悕涓篴ll.tar.bz2<br>       # tar -xjf all.tar.bz2<br>銆銆榪欐潯鍛戒護鏄皢涓婇潰浜х敓鐨勫寘瑙e紑銆?br>       3)tar璋冪敤compress<br>  compress涔熸槸涓涓帇緙╃▼搴忥紝浣嗘槸濂借薄浣跨敤compress鐨勪漢涓嶅gzip鍜宐zip2鐨勪漢澶氥?Z緇撳熬鐨勬枃浠跺氨鏄痓zip2鍘嬬緝鐨勭粨鏋溿備笌 compress鐩稿鐨勮В鍘嬬▼搴忔槸uncompress銆倀ar涓嬌鐢?Z榪欎釜鍙傛暟鏉ヨ皟鐢╟ompress銆備笅闈㈡潵涓句緥璇存槑涓涓嬶細<br>       # tar -cZf all.tar.Z *.jpg<br>銆銆榪欐潯鍛戒護鏄皢鎵鏈?jpg鐨勬枃浠舵墦鎴愪竴涓猼ar鍖咃紝騫朵笖灝嗗叾鐢╟ompress鍘嬬緝錛岀敓鎴愪竴涓猽ncompress鍘嬬緝榪囩殑鍖咃紝鍖呭悕涓篴ll.tar.Z<br>       # tar -xZf all.tar.Z<br>銆銆榪欐潯鍛戒護鏄皢涓婇潰浜х敓鐨勫寘瑙e紑<br>      鏈変簡涓婇潰鐨勭煡璇嗭紝浣犲簲璇ュ彲浠ヨВ寮澶氱鍘嬬緝鏂囦歡浜嗭紝涓嬮潰瀵逛簬tar緋誨垪鐨勫帇緙╂枃<br>浠朵綔涓涓皬緇擄細<br>       #1)瀵逛簬.tar緇撳熬鐨勬枃浠?br>       tar -xf all.tar<br>       2)瀵逛簬.gz緇撳熬鐨勬枃浠?br>       gzip -d all.gz<br>銆銆gunzip all.gz<br>      3)瀵逛簬.tgz鎴?tar.gz緇撳熬鐨勬枃浠?br>       tar -xzf all.tar.gz<br>銆銆tar -xzf all.tgz<br>       4)瀵逛簬.bz2緇撳熬鐨勬枃浠?br>       bzip2 -d all.bz2<br>銆銆bunzip2 all.bz2<br>       5)瀵逛簬tar.bz2緇撳熬鐨勬枃浠?br>       tar -xjf all.tar.bz2<br>       6)瀵逛簬.Z緇撳熬鐨勬枃浠?br>       uncompress all.Z<br>       7)瀵逛簬.tar.Z緇撳熬鐨勬枃浠?br>       tar -xZf all.tar.z<br>       鍙﹀瀵逛簬Window涓嬬殑甯歌鍘嬬緝鏂囦歡.zip鍜?rar錛孡inux涔熸湁鐩稿簲鐨勬柟娉曟潵瑙e帇瀹冧滑錛?br>       1)瀵逛簬.zip<br>        linux涓嬫彁渚涗簡zip鍜寀nzip紼嬪簭錛寊ip鏄帇緙╃▼搴忥紝unzip鏄В鍘嬬▼搴忋傚畠浠殑鍙?br>鏁伴夐」寰堝錛岃繖閲屽彧鍋氱畝鍗曚粙緇嶏紝渚濇棫涓句緥璇存槑涓涓嬪叾鐢ㄦ硶錛?br>       # zip all.zip *.jpg<br>銆銆榪欐潯鍛戒護鏄皢鎵鏈?jpg鐨勬枃浠跺帇緙╂垚涓涓獄ip鍖?br>       # unzip all.zip<br>銆銆榪欐潯鍛戒護鏄皢all.zip涓殑鎵鏈夋枃浠惰В鍘嬪嚭鏉?br>      2)瀵逛簬.rar<br> 瑕佸湪linux涓嬪鐞?rar鏂囦歡錛岄渶瑕佸畨瑁匯AR for Linux錛屽彲浠ヤ粠緗戜笂涓嬭澆錛屼絾瑕佽浣忥紝RAR for Linux涓嶆槸鍏嶈垂鐨勶紱鍙粠<a target=_blank><u><font color=#006600>http://www.rarsoft.com/download.htm</font></u></a>涓嬭澆RARfor Linux 3.2.<br>0錛岀劧鍚庡畨瑁咃細<br>       # tar -xzpvf rarlinux-3.2.0.tar.gz<br>銆銆# cd rar<br>銆銆# make<br>      榪欐牱灝卞畨瑁呭ソ浜嗭紝瀹夎鍚庡氨鏈変簡rar鍜寀nrar榪欎袱涓▼搴忥紝rar鏄帇緙╃▼搴忥紝unrar鏄В鍘嬬▼搴忋傚畠浠殑鍙傛暟閫夐」寰堝錛岃繖閲屽彧鍋氱畝鍗曚粙緇嶏紝渚濇棫涓句緥璇存槑涓涓嬪叾鐢ㄦ硶錛?br>       # rar a all *.jpg<br>銆銆榪欐潯鍛戒護鏄皢鎵鏈?jpg鐨勬枃浠跺帇緙╂垚涓涓猺ar鍖咃紝鍚嶄負all.rar錛岃紼嬪簭浼氬皢.rar鎵╁睍鍚嶅皢鑷姩闄勫姞鍒板寘鍚嶅悗銆?br>       # unrar e all.rar<br>銆銆榪欐潯鍛戒護鏄皢all.rar涓殑鎵鏈夋枃浠惰В鍘嬪嚭鏉?br>       鍒版涓鴻嚦錛屾垜浠凡緇忎粙緇嶈繃linux涓嬬殑tar銆乬zip銆乬unzip銆乥zip2銆乥unzip2銆乧ompress銆乽ncompress銆?zip銆乽nzip銆乺ar銆乽nrar絳夌▼寮忥紝浣犲簲璇ュ凡緇忚兘澶熶嬌鐢ㄥ畠浠.tar銆?gz銆?tar.gz銆?tgz銆?bz2銆?tar.bz2銆?Z銆?tar.Z銆?zip銆?rar榪?0縐嶅帇緙╂枃浠惰繘琛岃В鍘嬩簡錛屼互鍚庡簲璇ヤ笉闇瑕佷負涓嬭澆浜嗕竴涓蔣浠惰屼笉鐭ラ亾濡備綍鍦↙inux涓嬭В寮鑰岀儲鎭間簡銆傝屼笖浠ヤ笂鏂規硶瀵逛簬Unix涔熷熀鏈湁鏁堛?br>      鏈枃浠嬬粛浜唋inux涓嬬殑鍘嬬緝紼嬪紡tar銆乬zip銆乬unzip銆乥zip2銆乥unzip2銆乧ompress銆乽ncompress銆?zip銆?unzip銆乺ar銆乽nrar絳夌▼寮忥紝浠ュ強濡備綍浣跨敤瀹冧滑瀵?tar銆?gz銆?tar.gz銆?tgz銆?bz2銆?tar.bz2銆?Z銆? tar.Z銆?zip銆?rar榪?0縐嶅帇緙╂枃浠惰繘琛屾搷浣溿?</p> <img src ="http://www.shnenglu.com/koson/aggbug/126625.html" width = "1" height = "1" /><br><br><div align=right><a style="text-decoration:none;" href="http://www.shnenglu.com/koson/" target="_blank">koson</a> 2010-09-15 00:56 <a href="http://www.shnenglu.com/koson/archive/2010/09/15/126625.html#Feedback" target="_blank" style="text-decoration:none;">鍙戣〃璇勮</a></div>]]></description></item><item><title>寮濮嬫極闀胯岃壈杈涚殑鎵懼伐璺?/title><link>http://www.shnenglu.com/koson/archive/2010/09/14/126551.html</link><dc:creator>koson</dc:creator><author>koson</author><pubDate>Mon, 13 Sep 2010 20:05:00 GMT</pubDate><guid>http://www.shnenglu.com/koson/archive/2010/09/14/126551.html</guid><wfw:comment>http://www.shnenglu.com/koson/comments/126551.html</wfw:comment><comments>http://www.shnenglu.com/koson/archive/2010/09/14/126551.html#Feedback</comments><slash:comments>0</slash:comments><wfw:commentRss>http://www.shnenglu.com/koson/comments/commentRss/126551.html</wfw:commentRss><trackback:ping>http://www.shnenglu.com/koson/services/trackbacks/126551.html</trackback:ping><description><![CDATA[<p>涓嬫潵涓や笁涓湀錛屽皢鏄壘宸ラ樁孌電殑楂樺嘲鏈燂紝涓嶇浠ュ墠鍐嶅涔堢殑鍚勭涓嶅鎰忥紝閮藉繀欏誨叏韜績鐨勬姇鍏ヨ繘鍘伙紝涓嶅彲鎳堟狅紝甯屾湜鎵懼埌鑷繁婊℃剰鐨勪竴浠藉伐浣滐紒</p> <img src ="http://www.shnenglu.com/koson/aggbug/126551.html" width = "1" height = "1" /><br><br><div align=right><a style="text-decoration:none;" href="http://www.shnenglu.com/koson/" target="_blank">koson</a> 2010-09-14 04:05 <a href="http://www.shnenglu.com/koson/archive/2010/09/14/126551.html#Feedback" target="_blank" style="text-decoration:none;">鍙戣〃璇勮</a></div>]]></description></item><item><title>涓浜涙湁鐢ㄩ摼鎺?/title><link>http://www.shnenglu.com/koson/archive/2010/07/19/120776.html</link><dc:creator>koson</dc:creator><author>koson</author><pubDate>Mon, 19 Jul 2010 06:50:00 GMT</pubDate><guid>http://www.shnenglu.com/koson/archive/2010/07/19/120776.html</guid><wfw:comment>http://www.shnenglu.com/koson/comments/120776.html</wfw:comment><comments>http://www.shnenglu.com/koson/archive/2010/07/19/120776.html#Feedback</comments><slash:comments>2</slash:comments><wfw:commentRss>http://www.shnenglu.com/koson/comments/commentRss/120776.html</wfw:commentRss><trackback:ping>http://www.shnenglu.com/koson/services/trackbacks/120776.html</trackback:ping><description><![CDATA[http://caibinbupt.javaeye.com/ 钄℃枌錛宧adoop婧愮爜鍒嗘瀽<br>http://www.oschina.net/p/hive/recomm HIVE鐨勮緇嗘暀紼?br>http://www.tbdata.org/archives/category/hive 娣樺疂鏁版嵁騫沖彴鍥㈤槦<br>http://www.cnblogs.com/spork/archive/2010/01/11/1644342.html Map/Reduce鏁版嵁嫻?br><br><img src ="http://www.shnenglu.com/koson/aggbug/120776.html" width = "1" height = "1" /><br><br><div align=right><a style="text-decoration:none;" href="http://www.shnenglu.com/koson/" target="_blank">koson</a> 2010-07-19 14:50 <a href="http://www.shnenglu.com/koson/archive/2010/07/19/120776.html#Feedback" target="_blank" style="text-decoration:none;">鍙戣〃璇勮</a></div>]]></description></item><item><title>Hadoop鍒嗗竷寮忔枃浠剁郴緇燂細鏋舵瀯鍜岃璁¤鐐?/title><link>http://www.shnenglu.com/koson/archive/2010/07/19/120775.html</link><dc:creator>koson</dc:creator><author>koson</author><pubDate>Mon, 19 Jul 2010 06:42:00 GMT</pubDate><guid>http://www.shnenglu.com/koson/archive/2010/07/19/120775.html</guid><wfw:comment>http://www.shnenglu.com/koson/comments/120775.html</wfw:comment><comments>http://www.shnenglu.com/koson/archive/2010/07/19/120775.html#Feedback</comments><slash:comments>0</slash:comments><wfw:commentRss>http://www.shnenglu.com/koson/comments/commentRss/120775.html</wfw:commentRss><trackback:ping>http://www.shnenglu.com/koson/services/trackbacks/120775.html</trackback:ping><description><![CDATA[<p style="MARGIN-BOTTOM: 0in" id=nekf0 class=western>Hadoop<font id=nekf1 face="瀹嬩綋, SimSun"><span id=nekf2 lang=zh-CN>鍒嗗竷寮忔枃浠剁郴緇燂細鏋舵瀯鍜岃璁¤鐐?/span></font><br id=nekf3>鍘熸枃錛歨ttp://hadoop.apache.org/core/docs/current/hdfs_design.html<br id=nekf4><font id=nekf5 face="瀹嬩綋, SimSun"><span id=nekf6 lang=zh-CN>涓銆佸墠鎻愬拰璁捐鐩爣</span></font><br id=nekf7>1<font id=nekf8 face="瀹嬩綋, SimSun"><span id=nekf9 lang=zh-CN>銆佺‖浠墮敊璇槸甯告侊紝鑰岄潪寮傚父鎯呭喌錛?/span></font>HDFS<font id=nekf10 face="瀹嬩綋, SimSun"><span id=nekf11 lang=zh-CN>鍙兘鏄湁鎴愮櫨涓婂崈鐨?/span></font>server<font id=nekf12 face="瀹嬩綋, SimSun"><span id=nekf13 lang=zh-CN>緇勬垚錛屼換浣曚竴涓粍浠墮兘鏈夊彲鑳戒竴鐩村け鏁堬紝鍥犳閿欒媯嫻嬪拰蹇熴佽嚜鍔ㄧ殑鎭㈠鏄?/span></font>HDFS<font id=nekf14 face="瀹嬩綋, SimSun"><span id=nekf15 lang=zh-CN>鐨勬牳蹇冩灦鏋勭洰鏍囥?/span></font><br id=nekf16>2<font id=nekf17 face="瀹嬩綋, SimSun"><span id=nekf18 lang=zh-CN>銆佽窇鍦?/span></font>HDFS<font id=nekf19 face="瀹嬩綋, SimSun"><span id=nekf20 lang=zh-CN>涓婄殑搴旂敤涓庝竴鑸殑搴旂敤涓嶅悓錛屽畠浠富瑕佹槸浠ユ祦寮忚涓轟富錛屽仛鎵歸噺澶勭悊錛涙瘮涔嬪叧娉ㄦ暟鎹闂殑浣庡歡榪熼棶棰橈紝鏇村叧閿殑鍦ㄤ簬鏁版嵁璁塊棶鐨勯珮鍚炲悙閲忋?/span></font><br id=nekf21>3<font id=nekf22 face="瀹嬩綋, SimSun"><span id=nekf23 lang=zh-CN>銆?/span></font>HDFS<font id=nekf24 face="瀹嬩綋, SimSun"><span id=nekf25 lang=zh-CN>浠ユ敮鎸佸ぇ鏁版嵁闆嗗悎涓虹洰鏍囷紝涓涓瓨鍌ㄥ湪涓婇潰鐨勫吀鍨嬫枃浠跺ぇ灝忎竴鑸兘鍦ㄥ崈鍏嗚嚦</span></font>T<font id=nekf26 face="瀹嬩綋, SimSun"><span id=nekf27 lang=zh-CN>瀛楄妭錛屼竴涓崟涓</span></font>HDFS<font id=nekf28 face="瀹嬩綋, SimSun"><span id=nekf29 lang=zh-CN>瀹炰緥搴旇鑳芥敮鎾戞暟浠ュ崈涓囪鐨勬枃浠躲?/span></font><br id=nekf30>4<font id=nekf31 face="瀹嬩綋, SimSun"><span id=nekf32 lang=zh-CN>銆?</span></font>HDFS<font id=nekf33 face="瀹嬩綋, SimSun"><span id=nekf34 lang=zh-CN>搴旂敤瀵規枃浠惰姹傜殑鏄?/span></font>write-one-read-many<font id=nekf35 face="瀹嬩綋, SimSun"><span id=nekf36 lang=zh-CN>璁塊棶妯″瀷銆備竴涓枃浠剁粡榪囧垱寤恒佸啓錛屽叧闂箣鍚庡氨涓嶉渶瑕佹敼鍙樸傝繖涓鍋囪綆鍖栦簡鏁版嵁涓鑷存ч棶棰橈紝浣塊珮鍚炲悙閲忕殑鏁版嵁璁塊棶鎴愪負鍙兘銆傚吀鍨嬬殑濡?/span></font>MapReduce<font id=nekf37 face="瀹嬩綋, SimSun"><span id=nekf38 lang=zh-CN>妗嗘灦錛屾垨鑰呬竴涓?/span></font>web crawler<font id=nekf39 face="瀹嬩綋, SimSun"><span id=nekf40 lang=zh-CN>搴旂敤閮藉緢閫傚悎榪欎釜妯″瀷銆?/span></font><br id=nekf41>5<font id=nekf42 face="瀹嬩綋, SimSun"><span id=nekf43 lang=zh-CN>銆佺Щ鍔ㄨ綆楃殑浠d環姣斾箣縐誨姩鏁版嵁鐨勪唬浠蜂綆銆備竴涓簲鐢ㄨ姹傜殑璁$畻錛岀瀹冩搷浣滅殑鏁版嵁瓚婅繎灝辮秺楂樻晥錛岃繖鍦ㄦ暟鎹揪鍒版搗閲忕駭鍒殑鏃跺欐洿鏄姝ゃ傚皢璁$畻縐誨姩鍒版暟鎹檮榪戯紝姣斾箣灝嗘暟鎹Щ鍔ㄥ埌搴旂敤鎵鍦ㄦ樉鐒舵洿濂斤紝</span></font>HDFS<font id=nekf44 face="瀹嬩綋, SimSun"><span id=nekf45 lang=zh-CN>鎻愪緵緇欏簲鐢ㄨ繖鏍風殑鎺ュ彛銆?/span></font><br id=nekf46>6<font id=nekf47 face="瀹嬩綋, SimSun"><span id=nekf48 lang=zh-CN>銆佸湪寮傛瀯鐨勮蔣紜歡騫沖彴闂寸殑鍙Щ妞嶆с?/span></font><br id=nekf49><br id=nekf50><font id=nekf51 face="瀹嬩綋, SimSun"><span id=nekf52 lang=zh-CN>浜屻?/span></font>Namenode<font id=nekf53 face="瀹嬩綋, SimSun"><span id=nekf54 lang=zh-CN>鍜?/span></font>Datanode<br id=nekf55>    HDFS<font id=nekf56 face="瀹嬩綋, SimSun"><span id=nekf57 lang=zh-CN>閲囩敤</span></font>master/slave<font id=nekf58 face="瀹嬩綋, SimSun"><span id=nekf59 lang=zh-CN>鏋舵瀯銆備竴涓?/span></font>HDFS<font id=nekf60 face="瀹嬩綋, SimSun"><span id=nekf61 lang=zh-CN>闆嗙兢鏄湁涓涓?/span></font>Namenode<font id=nekf62 face="瀹嬩綋, SimSun"><span id=nekf63 lang=zh-CN>鍜屼竴瀹氭暟鐩殑</span></font>Datanode<font id=nekf64 face="瀹嬩綋, SimSun"><span id=nekf65 lang=zh-CN>緇勬垚銆?/span></font>Namenode<font id=nekf66 face="瀹嬩綋, SimSun"><span id=nekf67 lang=zh-CN>鏄竴涓腑蹇冩湇鍔″櫒錛岃礋璐g鐞嗘枃浠剁郴緇熺殑</span></font>namespace<font id=nekf68 face="瀹嬩綋, SimSun"><span id=nekf69 lang=zh-CN>鍜屽鎴風瀵規枃浠剁殑璁塊棶銆?/span></font>Datanode<font id=nekf70 face="瀹嬩綋, SimSun"><span id=nekf71 lang=zh-CN>鍦ㄩ泦緹や腑涓鑸槸涓涓妭鐐逛竴涓紝璐熻矗綆$悊鑺傜偣涓婂畠浠檮甯︾殑瀛樺偍銆傚湪鍐呴儴錛屼竴涓枃浠跺叾瀹炲垎鎴愪竴涓垨澶氫釜</span></font>block<font id=nekf72 face="瀹嬩綋, SimSun"><span id=nekf73 lang=zh-CN>錛岃繖浜?/span></font>block<font id=nekf74 face="瀹嬩綋, SimSun"><span id=nekf75 lang=zh-CN>瀛樺偍鍦?/span></font>Datanode<font id=nekf76 face="瀹嬩綋, SimSun"><span id=nekf77 lang=zh-CN>闆嗗悎閲屻?/span></font>Namenode<font id=nekf78 face="瀹嬩綋, SimSun"><span id=nekf79 lang=zh-CN>鎵ц鏂囦歡緋葷粺鐨?/span></font>namespace<font id=nekf80 face="瀹嬩綋, SimSun"><span id=nekf81 lang=zh-CN>鎿嶄綔錛屼緥濡傛墦寮銆佸叧闂侀噸鍛藉悕鏂囦歡鍜岀洰褰曪紝鍚屾椂鍐沖畾</span></font>block<font id=nekf82 face="瀹嬩綋, SimSun"><span id=nekf83 lang=zh-CN>鍒板叿浣?/span></font>Datanode<font id=nekf84 face="瀹嬩綋, SimSun"><span id=nekf85 lang=zh-CN>鑺傜偣鐨勬槧灝勩?/span></font>Datanode<font id=nekf86 face="瀹嬩綋, SimSun"><span id=nekf87 lang=zh-CN>鍦?/span></font>Namenode<font id=nekf88 face="瀹嬩綋, SimSun"><span id=nekf89 lang=zh-CN>鐨勬寚鎸ヤ笅榪涜</span></font>block<font id=nekf90 face="瀹嬩綋, SimSun"><span id=nekf91 lang=zh-CN>鐨勫垱寤恒佸垹闄ゅ拰澶嶅埗銆?/span></font>Namenode<font id=nekf92 face="瀹嬩綋, SimSun"><span id=nekf93 lang=zh-CN>鍜?/span></font>Datanode<font id=nekf94 face="瀹嬩綋, SimSun"><span id=nekf95 lang=zh-CN>閮芥槸璁捐鎴愬彲浠ヨ窇鍦ㄦ櫘閫氱殑寤変環鐨勮繍琛?/span></font>linux<font id=nekf96 face="瀹嬩綋, SimSun"><span id=nekf97 lang=zh-CN>鐨勬満鍣ㄤ笂銆?/span></font>HDFS<font id=nekf98 face="瀹嬩綋, SimSun"><span id=nekf99 lang=zh-CN>閲囩敤</span></font>java<font id=nekf100 face="瀹嬩綋, SimSun"><span id=nekf101 lang=zh-CN>璇█寮鍙戯紝鍥犳鍙互閮ㄧ講鍦ㄥ緢澶ц寖鍥寸殑鏈哄櫒涓娿備竴涓吀鍨嬬殑閮ㄧ講鍦烘櫙鏄竴鍙版満鍣ㄨ窇涓涓崟鐙殑</span></font>Namenode<font id=nekf102 face="瀹嬩綋, SimSun"><span id=nekf103 lang=zh-CN>鑺傜偣錛岄泦緹や腑鐨勫叾浠栨満鍣ㄥ悇璺戜竴涓?/span></font>Datanode<font id=nekf104 face="瀹嬩綋, SimSun"><span id=nekf105 lang=zh-CN>瀹炰緥銆傝繖涓灦鏋勫茍涓嶆帓闄や竴鍙版満鍣ㄤ笂璺戝涓?/span></font>Datanode<font id=nekf106 face="瀹嬩綋, SimSun"><span id=nekf107 lang=zh-CN>錛屼笉榪囪繖姣旇緝灝戣銆?/span></font><br id=nekf108><img id=nekf109 border=0 name=graphics1 alt="" align=bottom src="http://docs.google.com/File?id=ddxkntwd_24fwt83gcp_b" width=577 height=394><br id=nekf110><font id=nekf111 face="瀹嬩綋, SimSun"><span id=nekf112 lang=zh-CN>鍗曚竴鑺傜偣鐨?/span></font>Namenode<font id=nekf113 face="瀹嬩綋, SimSun"><span id=nekf114 lang=zh-CN>澶уぇ綆鍖栦簡緋葷粺鐨勬灦鏋勩?/span></font>Namenode<font id=nekf115 face="瀹嬩綋, SimSun"><span id=nekf116 lang=zh-CN>璐熻矗淇濈鍜岀鐞嗘墍鏈夌殑</span></font>HDFS<font id=nekf117 face="瀹嬩綋, SimSun"><span id=nekf118 lang=zh-CN>鍏冩暟鎹紝鍥犺岀敤鎴鋒暟鎹氨涓嶉渶瑕侀氳繃</span></font>Namenode<font id=nekf119 face="瀹嬩綋, SimSun"><span id=nekf120 lang=zh-CN>錛堜篃灝辨槸璇存枃浠舵暟鎹殑璇誨啓鏄洿鎺ュ湪</span></font>Datanode<font id=nekf121 face="瀹嬩綋, SimSun"><span id=nekf122 lang=zh-CN>涓婏級銆?/span></font><br id=nekf123><br id=nekf124><font id=nekf125 face="瀹嬩綋, SimSun"><span id=nekf126 lang=zh-CN>涓夈佹枃浠剁郴緇熺殑</span></font>namespace<br id=nekf127>   HDFS<font id=nekf128 face="瀹嬩綋, SimSun"><span id=nekf129 lang=zh-CN>鏀寔浼犵粺鐨勫眰嬈″瀷鏂囦歡緇勭粐錛屼笌澶у鏁板叾浠栨枃浠剁郴緇熺被浼鹼紝鐢ㄦ埛鍙互鍒涘緩鐩綍錛屽茍鍦ㄥ叾闂村垱寤恒佸垹闄ゃ佺Щ鍔ㄥ拰閲嶅懡鍚嶆枃浠躲?/span></font>HDFS<font id=nekf130 face="瀹嬩綋, SimSun"><span id=nekf131 lang=zh-CN>涓嶆敮鎸?/span></font>user quotas<font id=nekf132 face="瀹嬩綋, SimSun"><span id=nekf133 lang=zh-CN>鍜岃闂潈闄愶紝涔熶笉鏀寔閾炬帴錛?/span></font>link)<font id=nekf134 face="瀹嬩綋, SimSun"><span id=nekf135 lang=zh-CN>錛屼笉榪囧綋鍓嶇殑鏋舵瀯騫朵笉鎺掗櫎瀹炵幇榪欎簺鐗規с?/span></font>Namenode<font id=nekf136 face="瀹嬩綋, SimSun"><span id=nekf137 lang=zh-CN>緇存姢鏂囦歡緋葷粺鐨?/span></font>namespace<font id=nekf138 face="瀹嬩綋, SimSun"><span id=nekf139 lang=zh-CN>錛屼換浣曞鏂囦歡緋葷粺</span></font>namespace<font id=nekf140 face="瀹嬩綋, SimSun"><span id=nekf141 lang=zh-CN>鍜屾枃浠跺睘鎬х殑淇敼閮藉皢琚?/span></font>Namenode<font id=nekf142 face="瀹嬩綋, SimSun"><span id=nekf143 lang=zh-CN>璁板綍涓嬫潵銆傚簲鐢ㄥ彲浠ヨ緗?/span></font>HDFS<font id=nekf144 face="瀹嬩綋, SimSun"><span id=nekf145 lang=zh-CN>淇濆瓨鐨勬枃浠剁殑鍓湰鏁扮洰錛屾枃浠跺壇鏈殑鏁扮洰縐頒負鏂囦歡鐨?</span></font>replication<font id=nekf146 face="瀹嬩綋, SimSun"><span id=nekf147 lang=zh-CN>鍥犲瓙錛岃繖涓俊鎭篃鏄敱</span></font>Namenode<font id=nekf148 face="瀹嬩綋, SimSun"><span id=nekf149 lang=zh-CN>淇濆瓨銆?/span></font><br id=nekf150><br id=nekf151><font id=nekf152 face="瀹嬩綋, SimSun"><span id=nekf153 lang=zh-CN>鍥涖佹暟鎹鍒?/span></font><br id=nekf154>    HDFS<font id=nekf155 face="瀹嬩綋, SimSun"><span id=nekf156 lang=zh-CN>琚璁℃垚鍦ㄤ竴涓ぇ闆嗙兢涓彲浠ヨ法鏈哄櫒鍦板彲闈犲湴瀛樺偍嫻烽噺鐨勬枃浠躲傚畠灝嗘瘡涓枃浠跺瓨鍌ㄦ垚</span></font>block<font id=nekf157 face="瀹嬩綋, SimSun"><span id=nekf158 lang=zh-CN>搴忓垪錛岄櫎浜嗘渶鍚庝竴涓?/span></font>block<font id=nekf159 face="瀹嬩綋, SimSun"><span id=nekf160 lang=zh-CN>錛屾墍鏈夌殑</span></font>block<font id=nekf161 face="瀹嬩綋, SimSun"><span id=nekf162 lang=zh-CN>閮芥槸鍚屾牱鐨勫ぇ灝忋傛枃浠剁殑鎵鏈?/span></font>block<font id=nekf163 face="瀹嬩綋, SimSun"><span id=nekf164 lang=zh-CN>涓轟簡瀹歸敊閮戒細琚鍒躲傛瘡涓枃浠剁殑</span></font>block<font id=nekf165 face="瀹嬩綋, SimSun"><span id=nekf166 lang=zh-CN>澶у皬鍜?/span></font>replication<font id=nekf167 face="瀹嬩綋, SimSun"><span id=nekf168 lang=zh-CN>鍥犲瓙閮芥槸鍙厤緗殑銆?/span></font>Replication<font id=nekf169 face="瀹嬩綋, SimSun"><span id=nekf170 lang=zh-CN>鍥犲瓙鍙互鍦ㄦ枃浠跺垱寤虹殑鏃跺欓厤緗紝浠ュ悗涔熷彲浠ユ敼鍙樸?/span></font>HDFS<font id=nekf171 face="瀹嬩綋, SimSun"><span id=nekf172 lang=zh-CN>涓殑鏂囦歡鏄?/span></font>write-one<font id=nekf173 face="瀹嬩綋, SimSun"><span id=nekf174 lang=zh-CN>錛屽茍涓斾弗鏍艱姹傚湪浠諱綍鏃跺欏彧鏈変竴涓?/span></font>writer<font id=nekf175 face="瀹嬩綋, SimSun"><span id=nekf176 lang=zh-CN>銆?/span></font>Namenode<font id=nekf177 face="瀹嬩綋, SimSun"><span id=nekf178 lang=zh-CN>鍏ㄦ潈綆$悊</span></font>block<font id=nekf179 face="瀹嬩綋, SimSun"><span id=nekf180 lang=zh-CN>鐨勫鍒訛紝瀹冨懆鏈熸у湴浠庨泦緹や腑鐨勬瘡涓?/span></font>Datanode<font id=nekf181 face="瀹嬩綋, SimSun"><span id=nekf182 lang=zh-CN>鎺ユ敹蹇冭煩鍖呭拰涓涓?/span></font>Blockreport<font id=nekf183 face="瀹嬩綋, SimSun"><span id=nekf184 lang=zh-CN>銆傚績璺沖寘鐨勬帴鏀惰〃紺鴻</span></font>Datanode<font id=nekf185 face="瀹嬩綋, SimSun"><span id=nekf186 lang=zh-CN>鑺傜偣姝e父宸ヤ綔錛岃?/span></font>Blockreport<font id=nekf187 face="瀹嬩綋, SimSun"><span id=nekf188 lang=zh-CN>鍖呮嫭浜嗚</span></font>Datanode<font id=nekf189 face="瀹嬩綋, SimSun"><span id=nekf190 lang=zh-CN>涓婃墍鏈夌殑</span></font>block<font id=nekf191 face="瀹嬩綋, SimSun"><span id=nekf192 lang=zh-CN>緇勬垚鐨勫垪琛ㄣ?/span></font></p> <p style="MARGIN-BOTTOM: 0in" id=nekf193 class=western><img id=nekf194 border=0 name=graphics2 alt="" align=bottom src="http://docs.google.com/File?id=ddxkntwd_25gq43fcg4_b" width=576 height=352><br id=nekf195>1<font id=nekf196 face="瀹嬩綋, SimSun"><span id=nekf197 lang=zh-CN>銆佸壇鏈殑瀛樻斁錛屽壇鏈殑瀛樻斁鏄?/span></font>HDFS<font id=nekf198 face="瀹嬩綋, SimSun"><span id=nekf199 lang=zh-CN>鍙潬鎬у拰鎬ц兘鐨勫叧閿?/span></font>HDFS<font id=nekf200 face="瀹嬩綋, SimSun"><span id=nekf201 lang=zh-CN>閲囩敤涓縐嶇О涓?/span></font>rack-aware<font id=nekf202 face="瀹嬩綋, SimSun"><span id=nekf203 lang=zh-CN>鐨勭瓥鐣ユ潵鏀硅繘鏁版嵁鐨勫彲闈犳с佹湁鏁堟у拰緗戠粶甯﹀鐨勫埄鐢ㄣ傝繖涓瓥鐣ュ疄鐜扮殑鐭湡鐩爣鏄獙璇佸湪鐢熶駭鐜涓嬬殑琛ㄧ幇錛岃瀵熷畠鐨勮涓猴紝鏋勫緩嫻嬭瘯鍜岀爺絀剁殑鍩虹錛屼互渚垮疄鐜版洿鍏堣繘鐨勭瓥鐣ャ傚簽澶х殑</span></font>HDFS<font id=nekf204 face="瀹嬩綋, SimSun"><span id=nekf205 lang=zh-CN>瀹炰緥涓鑸繍琛屽湪澶氫釜鏈烘灦鐨勮綆楁満褰㈡垚鐨勯泦緹や笂錛屼笉鍚屾満鏋墮棿鐨勪袱鍙版満鍣ㄧ殑閫氳闇瑕侀氳繃浜ゆ崲鏈猴紝鏄劇劧閫氬父鎯呭喌涓嬶紝鍚屼竴涓満鏋跺唴鐨勪袱涓妭鐐歸棿鐨勫甫瀹戒細姣斾笉鍚屾満鏋墮棿鐨勪袱鍙版満鍣ㄧ殑甯﹀澶с?/span></font><br id=nekf206>    <font id=nekf207 face="瀹嬩綋, SimSun"><span id=nekf208 lang=zh-CN>閫氳繃涓涓О涓?/span></font>Rack Awareness<font id=nekf209 face="瀹嬩綋, SimSun"><span id=nekf210 lang=zh-CN>鐨勮繃紼嬶紝</span></font>Namenode<font id=nekf211 face="瀹嬩綋, SimSun"><span id=nekf212 lang=zh-CN>鍐沖畾浜嗘瘡涓?/span></font>Datanode<font id=nekf213 face="瀹嬩綋, SimSun"><span id=nekf214 lang=zh-CN>鎵灞炵殑</span></font>rack id<font id=nekf215 face="瀹嬩綋, SimSun"><span id=nekf216 lang=zh-CN>銆備竴涓畝鍗曚絾娌℃湁浼樺寲鐨勭瓥鐣ュ氨鏄皢鍓湰瀛樻斁鍦ㄥ崟鐙殑鏈烘灦涓娿傝繖鏍峰彲浠ラ槻姝㈡暣涓満鏋訛紙闈炲壇鏈瓨鏀撅級澶辨晥鐨勬儏鍐碉紝騫朵笖鍏佽璇繪暟鎹殑鏃跺欏彲浠ヤ粠澶氫釜鏈烘灦璇誨彇銆傝繖涓畝鍗曠瓥鐣ヨ緗彲浠ュ皢鍓湰鍒嗗竷鍦ㄩ泦緹や腑錛屾湁鍒╀簬緇勪歡澶辮觸鎯呭喌涓嬬殑璐熻澆鍧囪 銆備絾鏄紝榪欎釜綆鍗曠瓥鐣ュ姞澶т簡鍐欑殑浠d環錛屽洜涓轟竴涓啓鎿嶄綔闇瑕佷紶杈?/span></font>block<font id=nekf217 face="瀹嬩綋, SimSun"><span id=nekf218 lang=zh-CN>鍒板涓満鏋躲?/span></font><br id=nekf219>    <font id=nekf220 face="瀹嬩綋, SimSun"><span id=nekf221 lang=zh-CN>鍦ㄥぇ澶氭暟鎯呭喌涓嬶紝</span></font>replication<font id=nekf222 face="瀹嬩綋, SimSun"><span id=nekf223 lang=zh-CN>鍥犲瓙鏄?/span></font>3<font id=nekf224 face="瀹嬩綋, SimSun"><span id=nekf225 lang=zh-CN>錛?/span></font>HDFS<font id=nekf226 face="瀹嬩綋, SimSun"><span id=nekf227 lang=zh-CN>鐨勫瓨鏀劇瓥鐣ユ槸灝嗕竴涓壇鏈瓨鏀懼湪鏈湴鏈烘灦涓婄殑鑺傜偣錛屼竴涓壇鏈斁鍦ㄥ悓涓鏈烘灦涓婄殑鍙︿竴涓妭鐐癸紝鏈鍚庝竴涓壇鏈斁鍦ㄤ笉鍚屾満鏋朵笂鐨勪竴涓妭鐐廣傛満鏋剁殑閿欒榪滆繙姣旇妭鐐圭殑閿欒灝戯紝榪欎釜絳栫暐涓嶄細褰卞搷鍒版暟鎹殑鍙潬鎬у拰鏈夋晥鎬с備笁鍒嗕箣涓鐨勫壇鏈湪涓涓妭鐐逛笂錛屼笁鍒嗕箣浜屽湪涓涓満鏋朵笂錛屽叾浠栦繚瀛樺湪鍓╀笅鐨勬満鏋朵腑錛岃繖涓絳栫暐鏀硅繘浜嗗啓鐨勬ц兘銆?/span></font><br id=nekf228><br id=nekf229>2<font id=nekf230 face="瀹嬩綋, SimSun"><span id=nekf231 lang=zh-CN>銆佸壇鏈殑閫夋嫨錛屼負浜嗛檷浣庢暣浣撶殑甯﹀娑堣楀拰璇誨歡鏃訛紝</span></font>HDFS<font id=nekf232 face="瀹嬩綋, SimSun"><span id=nekf233 lang=zh-CN>浼氬敖閲忚</span></font>reader<font id=nekf234 face="瀹嬩綋, SimSun"><span id=nekf235 lang=zh-CN>璇繪渶榪戠殑鍓湰銆傚鏋滃湪</span></font>reader<font id=nekf236 face="瀹嬩綋, SimSun"><span id=nekf237 lang=zh-CN>鐨勫悓涓涓満鏋朵笂鏈変竴涓壇鏈紝閭d箞灝辮璇ュ壇鏈傚鏋滀竴涓?/span></font>HDFS<font id=nekf238 face="瀹嬩綋, SimSun"><span id=nekf239 lang=zh-CN>闆嗙兢璺ㄨ秺澶氫釜鏁版嵁涓績錛岄偅涔?/span></font>reader<font id=nekf240 face="瀹嬩綋, SimSun"><span id=nekf241 lang=zh-CN>涔熷皢棣栧厛灝濊瘯璇繪湰鍦版暟鎹腑蹇冪殑鍓湰銆?/span></font><br id=nekf242><br id=nekf243>3<font id=nekf244 face="瀹嬩綋, SimSun"><span id=nekf245 lang=zh-CN>銆?/span></font>SafeMode<br id=nekf246>    Namenode<font id=nekf247 face="瀹嬩綋, SimSun"><span id=nekf248 lang=zh-CN>鍚姩鍚庝細榪涘叆涓涓О涓?/span></font>SafeMode<font id=nekf249 face="瀹嬩綋, SimSun"><span id=nekf250 lang=zh-CN>鐨勭壒孌婄姸鎬侊紝澶勫湪榪欎釜鐘舵佺殑</span></font>Namenode<font id=nekf251 face="瀹嬩綋, SimSun"><span id=nekf252 lang=zh-CN>鏄笉浼氳繘琛屾暟鎹潡鐨勫鍒剁殑銆?/span></font>Namenode<font id=nekf253 face="瀹嬩綋, SimSun"><span id=nekf254 lang=zh-CN>浠庢墍鏈夌殑 </span></font>Datanode<font id=nekf255 face="瀹嬩綋, SimSun"><span id=nekf256 lang=zh-CN>鎺ユ敹蹇冭煩鍖呭拰</span></font>Blockreport<font id=nekf257 face="瀹嬩綋, SimSun"><span id=nekf258 lang=zh-CN>銆?/span></font>Blockreport<font id=nekf259 face="瀹嬩綋, SimSun"><span id=nekf260 lang=zh-CN>鍖呮嫭浜嗘煇涓?/span></font>Datanode<font id=nekf261 face="瀹嬩綋, SimSun"><span id=nekf262 lang=zh-CN>鎵鏈夌殑鏁版嵁鍧楀垪琛ㄣ傛瘡涓?/span></font>block<font id=nekf263 face="瀹嬩綋, SimSun"><span id=nekf264 lang=zh-CN>閮芥湁鎸囧畾鐨勬渶灝忔暟鐩殑鍓湰銆傚綋</span></font>Namenode<font id=nekf265 face="瀹嬩綋, SimSun"><span id=nekf266 lang=zh-CN>媯嫻嬬‘璁ゆ煇涓?/span></font>Datanode<font id=nekf267 face="瀹嬩綋, SimSun"><span id=nekf268 lang=zh-CN>鐨勬暟鎹潡鍓湰鐨勬渶灝忔暟鐩紝閭d箞璇?/span></font>Datanode<font id=nekf269 face="瀹嬩綋, SimSun"><span id=nekf270 lang=zh-CN>灝變細琚涓烘槸瀹夊叏鐨勶紱濡傛灉涓瀹氱櫨鍒嗘瘮錛堣繖涓弬鏁板彲閰嶇疆錛夌殑鏁版嵁鍧楁嫻嬬‘璁ゆ槸瀹夊叏鐨勶紝閭d箞</span></font>Namenode<font id=nekf271 face="瀹嬩綋, SimSun"><span id=nekf272 lang=zh-CN>灝嗛鍑?/span></font>SafeMode<font id=nekf273 face="瀹嬩綋, SimSun"><span id=nekf274 lang=zh-CN>鐘舵侊紝鎺ヤ笅鏉ュ畠浼氱‘瀹氳繕鏈夊摢浜涙暟鎹潡鐨勫壇鏈病鏈夎揪鍒版寚瀹氭暟鐩紝騫跺皢榪欎簺</span></font>block<font id=nekf275 face="瀹嬩綋, SimSun"><span id=nekf276 lang=zh-CN>澶嶅埗鍒板叾浠?/span></font>Datanode<font id=nekf277 face="瀹嬩綋, SimSun"><span id=nekf278 lang=zh-CN>銆?/span></font><br id=nekf279><br id=nekf280><font id=nekf281 face="瀹嬩綋, SimSun"><span id=nekf282 lang=zh-CN>浜斻佹枃浠剁郴緇熷厓鏁版嵁鐨勬寔涔呭寲</span></font><br id=nekf283>    Namenode<font id=nekf284 face="瀹嬩綋, SimSun"><span id=nekf285 lang=zh-CN>瀛樺偍</span></font>HDFS<font id=nekf286 face="瀹嬩綋, SimSun"><span id=nekf287 lang=zh-CN>鐨勫厓鏁版嵁銆傚浜庝換浣曞鏂囦歡鍏冩暟鎹駭鐢熶慨鏀圭殑鎿嶄綔錛?/span></font>Namenode<font id=nekf288 face="瀹嬩綋, SimSun"><span id=nekf289 lang=zh-CN>閮戒嬌鐢ㄤ竴涓О涓?/span></font>Editlog<font id=nekf290 face="瀹嬩綋, SimSun"><span id=nekf291 lang=zh-CN>鐨勪簨鍔℃棩蹇楄褰曚笅鏉ャ備緥濡傦紝鍦?/span></font>HDFS<font id=nekf292 face="瀹嬩綋, SimSun"><span id=nekf293 lang=zh-CN>涓垱寤轟竴涓枃浠訛紝</span></font>Namenode<font id=nekf294 face="瀹嬩綋, SimSun"><span id=nekf295 lang=zh-CN>灝變細鍦?/span></font>Editlog<font id=nekf296 face="瀹嬩綋, SimSun"><span id=nekf297 lang=zh-CN>涓彃鍏ヤ竴鏉¤褰曟潵琛ㄧず錛涘悓鏍鳳紝淇敼鏂囦歡鐨?/span></font>replication<font id=nekf298 face="瀹嬩綋, SimSun"><span id=nekf299 lang=zh-CN>鍥犲瓙涔熷皢寰 </span></font>Editlog<font id=nekf300 face="瀹嬩綋, SimSun"><span id=nekf301 lang=zh-CN>鎻掑叆涓鏉¤褰曘?/span></font>Namenode<font id=nekf302 face="瀹嬩綋, SimSun"><span id=nekf303 lang=zh-CN>鍦ㄦ湰鍦?/span></font>OS<font id=nekf304 face="瀹嬩綋, SimSun"><span id=nekf305 lang=zh-CN>鐨勬枃浠剁郴緇熶腑瀛樺偍榪欎釜</span></font>Editlog<font id=nekf306 face="瀹嬩綋, SimSun"><span id=nekf307 lang=zh-CN>銆傛暣涓枃浠剁郴緇熺殑</span></font>namespace<font id=nekf308 face="瀹嬩綋, SimSun"><span id=nekf309 lang=zh-CN>錛屽寘鎷?/span></font>block<font id=nekf310 face="瀹嬩綋, SimSun"><span id=nekf311 lang=zh-CN>鍒版枃浠剁殑鏄犲皠銆佹枃浠剁殑灞炴э紝閮藉瓨鍌ㄥ湪縐頒負</span></font>FsImage<font id=nekf312 face="瀹嬩綋, SimSun"><span id=nekf313 lang=zh-CN>鐨勬枃浠朵腑錛岃繖涓枃浠朵篃鏄斁鍦?/span></font>Namenode<font id=nekf314 face="瀹嬩綋, SimSun"><span id=nekf315 lang=zh-CN>鎵鍦ㄧ郴緇熺殑鏂囦歡緋葷粺涓娿?/span></font><br id=nekf316>    Namenode<font id=nekf317 face="瀹嬩綋, SimSun"><span id=nekf318 lang=zh-CN>鍦ㄥ唴瀛樹腑淇濆瓨鐫鏁翠釜鏂囦歡緋葷粺</span></font>namespace<font id=nekf319 face="瀹嬩綋, SimSun"><span id=nekf320 lang=zh-CN>鍜屾枃浠?/span></font>Blockmap<font id=nekf321 face="瀹嬩綋, SimSun"><span id=nekf322 lang=zh-CN>鐨勬槧鍍忋傝繖涓叧閿殑鍏冩暟鎹璁″緱寰堢揣鍑戯紝鍥犺屼竴涓甫鏈?/span></font>4G<font id=nekf323 face="瀹嬩綋, SimSun"><span id=nekf324 lang=zh-CN>鍐呭瓨鐨?</span></font>Namenode<font id=nekf325 face="瀹嬩綋, SimSun"><span id=nekf326 lang=zh-CN>瓚沖鏀拺嫻烽噺鐨勬枃浠跺拰鐩綍銆傚綋</span></font>Namenode<font id=nekf327 face="瀹嬩綋, SimSun"><span id=nekf328 lang=zh-CN>鍚姩鏃訛紝瀹冧粠紜洏涓鍙?/span></font>Editlog<font id=nekf329 face="瀹嬩綋, SimSun"><span id=nekf330 lang=zh-CN>鍜?/span></font>FsImage<font id=nekf331 face="瀹嬩綋, SimSun"><span id=nekf332 lang=zh-CN>錛屽皢鎵鏈?/span></font>Editlog<font id=nekf333 face="瀹嬩綋, SimSun"><span id=nekf334 lang=zh-CN>涓殑浜嬪姟浣滅敤錛?/span></font>apply)<font id=nekf335 face="瀹嬩綋, SimSun"><span id=nekf336 lang=zh-CN>鍦ㄥ唴瀛樹腑鐨?/span></font>FsImage <font id=nekf337 face="瀹嬩綋, SimSun"><span id=nekf338 lang=zh-CN>錛屽茍灝嗚繖涓柊鐗堟湰鐨?/span></font>FsImage<font id=nekf339 face="瀹嬩綋, SimSun"><span id=nekf340 lang=zh-CN>浠庡唴瀛樹腑</span></font>flush<font id=nekf341 face="瀹嬩綋, SimSun"><span id=nekf342 lang=zh-CN>鍒扮‖鐩樹笂</span></font>,<font id=nekf343 face="瀹嬩綋, SimSun"><span id=nekf344 lang=zh-CN>鐒跺悗鍐?/span></font>truncate<font id=nekf345 face="瀹嬩綋, SimSun"><span id=nekf346 lang=zh-CN>榪欎釜鏃х殑</span></font>Editlog<font id=nekf347 face="瀹嬩綋, SimSun"><span id=nekf348 lang=zh-CN>錛屽洜涓鴻繖涓棫鐨?/span></font>Editlog<font id=nekf349 face="瀹嬩綋, SimSun"><span id=nekf350 lang=zh-CN>鐨勪簨鍔¢兘宸茬粡浣滅敤鍦?/span></font>FsImage<font id=nekf351 face="瀹嬩綋, SimSun"><span id=nekf352 lang=zh-CN>涓婁簡銆傝繖涓繃紼嬬О涓?/span></font>checkpoint<font id=nekf353 face="瀹嬩綋, SimSun"><span id=nekf354 lang=zh-CN>銆傚湪褰撳墠瀹炵幇涓紝</span></font>checkpoint<font id=nekf355 face="瀹嬩綋, SimSun"><span id=nekf356 lang=zh-CN>鍙彂鐢熷湪</span></font>Namenode<font id=nekf357 face="瀹嬩綋, SimSun"><span id=nekf358 lang=zh-CN>鍚姩鏃訛紝鍦ㄤ笉涔呯殑灝嗘潵鎴戜滑灝嗗疄鐜版敮鎸佸懆鏈熸х殑</span></font>checkpoint<font id=nekf359 face="瀹嬩綋, SimSun"><span id=nekf360 lang=zh-CN>銆?/span></font><br id=nekf361>    Datanode<font id=nekf362 face="瀹嬩綋, SimSun"><span id=nekf363 lang=zh-CN>騫朵笉鐭ラ亾鍏充簬鏂囦歡鐨勪換浣曚笢瑗匡紝闄や簡灝嗘枃浠朵腑鐨勬暟鎹繚瀛樺湪鏈湴鐨勬枃浠剁郴緇熶笂銆傚畠鎶婃瘡涓?/span></font>HDFS<font id=nekf364 face="瀹嬩綋, SimSun"><span id=nekf365 lang=zh-CN>鏁版嵁鍧楀瓨鍌ㄥ湪鏈湴鏂囦歡緋葷粺涓婇殧紱葷殑鏂囦歡涓?</span></font>Datanode<font id=nekf366 face="瀹嬩綋, SimSun"><span id=nekf367 lang=zh-CN>騫朵笉鍦ㄥ悓涓涓洰褰曞垱寤烘墍鏈夌殑鏂囦歡錛岀浉鍙嶏紝瀹冪敤鍚彂寮忓湴鏂規硶鏉ョ‘瀹氭瘡涓洰褰曠殑鏈浣蟲枃浠舵暟鐩紝騫朵笖鍦ㄩ傚綋鐨勬椂鍊欏垱寤哄瓙鐩綍銆傚湪鍚屼竴涓洰褰曞垱寤烘墍鏈夌殑鏂囦歡涓嶆槸鏈浼樼殑閫夋嫨錛屽洜涓烘湰鍦版枃浠剁郴緇熷彲鑳芥棤娉曢珮鏁堝湴鍦ㄥ崟涓鐩綍涓敮鎸佸ぇ閲忕殑鏂囦歡銆傚綋涓涓?/span></font>Datanode<font id=nekf368 face="瀹嬩綋, SimSun"><span id=nekf369 lang=zh-CN>鍚姩鏃訛紝瀹冩壂鎻忔湰鍦版枃浠剁郴緇燂紝瀵硅繖浜涙湰鍦版枃浠朵駭鐢熺浉搴旂殑涓涓墍鏈?/span></font>HDFS<font id=nekf370 face="瀹嬩綋, SimSun"><span id=nekf371 lang=zh-CN>鏁版嵁鍧楃殑鍒楄〃錛岀劧鍚庡彂閫佹姤鍛婂埌</span></font>Namenode<font id=nekf372 face="瀹嬩綋, SimSun"><span id=nekf373 lang=zh-CN>錛岃繖涓姤鍛婂氨鏄?/span></font>Blockreport<font id=nekf374 face="瀹嬩綋, SimSun"><span id=nekf375 lang=zh-CN>銆?/span></font><br id=nekf376><br id=nekf377><font id=nekf378 face="瀹嬩綋, SimSun"><span id=nekf379 lang=zh-CN>鍏侀氳鍗忚</span></font><br id=nekf380>    <font id=nekf381 face="瀹嬩綋, SimSun"><span id=nekf382 lang=zh-CN>鎵鏈夌殑</span></font>HDFS<font id=nekf383 face="瀹嬩綋, SimSun"><span id=nekf384 lang=zh-CN>閫氳鍗忚閮芥槸鏋勫緩鍦?/span></font>TCP/IP<font id=nekf385 face="瀹嬩綋, SimSun"><span id=nekf386 lang=zh-CN>鍗忚涓娿傚鎴風閫氳繃涓涓彲閰嶇疆鐨勭鍙h繛鎺ュ埌</span></font>Namenode<font id=nekf387 face="瀹嬩綋, SimSun"><span id=nekf388 lang=zh-CN>錛岄氳繃</span></font>ClientProtocol<font id=nekf389 face="瀹嬩綋, SimSun"><span id=nekf390 lang=zh-CN>涓?</span></font>Namenode<font id=nekf391 face="瀹嬩綋, SimSun"><span id=nekf392 lang=zh-CN>浜や簰銆傝?/span></font>Datanode<font id=nekf393 face="瀹嬩綋, SimSun"><span id=nekf394 lang=zh-CN>鏄嬌鐢?/span></font>DatanodeProtocol<font id=nekf395 face="瀹嬩綋, SimSun"><span id=nekf396 lang=zh-CN>涓?/span></font>Namenode<font id=nekf397 face="瀹嬩綋, SimSun"><span id=nekf398 lang=zh-CN>浜や簰銆備粠</span></font>ClientProtocol<font id=nekf399 face="瀹嬩綋, SimSun"><span id=nekf400 lang=zh-CN>鍜?</span></font>Datanodeprotocol<font id=nekf401 face="瀹嬩綋, SimSun"><span id=nekf402 lang=zh-CN>鎶借薄鍑轟竴涓繙紼嬭皟鐢?/span></font>(RPC<font id=nekf403 face="瀹嬩綋, SimSun"><span id=nekf404 lang=zh-CN>錛夛紝鍦ㄨ璁′笂錛?/span></font>Namenode<font id=nekf405 face="瀹嬩綋, SimSun"><span id=nekf406 lang=zh-CN>涓嶄細涓誨姩鍙戣搗</span></font>RPC<font id=nekf407 face="瀹嬩綋, SimSun"><span id=nekf408 lang=zh-CN>錛岃屾槸鏄搷搴旀潵鑷鎴風鍜?</span></font>Datanode <font id=nekf409 face="瀹嬩綋, SimSun"><span id=nekf410 lang=zh-CN>鐨?/span></font>RPC<font id=nekf411 face="瀹嬩綋, SimSun"><span id=nekf412 lang=zh-CN>璇鋒眰銆?/span></font><br id=nekf413><br id=nekf414><font id=nekf415 face="瀹嬩綋, SimSun"><span id=nekf416 lang=zh-CN>涓冦佸仴澹?/span></font><br id=nekf417>    HDFS<font id=nekf418 face="瀹嬩綋, SimSun"><span id=nekf419 lang=zh-CN>鐨勪富瑕佺洰鏍囧氨鏄疄鐜板湪澶辮觸鎯呭喌涓嬬殑鏁版嵁瀛樺偍鍙潬鎬с傚父瑙佺殑涓夌澶辮觸錛?/span></font>Namenode failures, Datanode failures<font id=nekf420 face="瀹嬩綋, SimSun"><span id=nekf421 lang=zh-CN>鍜岀綉緇滃垎鍓詫紙</span></font>network partitions)<font id=nekf422 face="瀹嬩綋, SimSun"><span id=nekf423 lang=zh-CN>銆?/span></font><br id=nekf424>1<font id=nekf425 face="瀹嬩綋, SimSun"><span id=nekf426 lang=zh-CN>銆佺‖鐩樻暟鎹敊璇佸績璺蟲嫻嬪拰閲嶆柊澶嶅埗</span></font><br id=nekf427>    <font id=nekf428 face="瀹嬩綋, SimSun"><span id=nekf429 lang=zh-CN>姣忎釜</span></font>Datanode<font id=nekf430 face="瀹嬩綋, SimSun"><span id=nekf431 lang=zh-CN>鑺傜偣閮藉悜</span></font>Namenode<font id=nekf432 face="瀹嬩綋, SimSun"><span id=nekf433 lang=zh-CN>鍛ㄦ湡鎬у湴鍙戦佸績璺沖寘銆傜綉緇滃垏鍓插彲鑳藉鑷翠竴閮ㄥ垎</span></font>Datanode<font id=nekf434 face="瀹嬩綋, SimSun"><span id=nekf435 lang=zh-CN>璺?/span></font>Namenode<font id=nekf436 face="瀹嬩綋, SimSun"><span id=nekf437 lang=zh-CN>澶卞幓鑱旂郴銆?</span></font>Namenode<font id=nekf438 face="瀹嬩綋, SimSun"><span id=nekf439 lang=zh-CN>閫氳繃蹇冭煩鍖呯殑緙哄け媯嫻嬪埌榪欎竴鎯呭喌錛屽茍灝嗚繖浜?/span></font>Datanode<font id=nekf440 face="瀹嬩綋, SimSun"><span id=nekf441 lang=zh-CN>鏍囪涓?/span></font>dead<font id=nekf442 face="瀹嬩綋, SimSun"><span id=nekf443 lang=zh-CN>錛屼笉浼氬皢鏂扮殑</span></font>IO<font id=nekf444 face="瀹嬩綋, SimSun"><span id=nekf445 lang=zh-CN>璇鋒眰鍙戠粰瀹冧滑銆傚瘎瀛樺湪</span></font>dead Datanode<font id=nekf446 face="瀹嬩綋, SimSun"><span id=nekf447 lang=zh-CN>涓婄殑浠諱綍鏁版嵁灝嗕笉鍐嶆湁鏁堛?/span></font>Datanode<font id=nekf448 face="瀹嬩綋, SimSun"><span id=nekf449 lang=zh-CN>鐨勬浜″彲鑳藉紩璧蜂竴浜?/span></font>block<font id=nekf450 face="瀹嬩綋, SimSun"><span id=nekf451 lang=zh-CN>鐨勫壇鏈暟鐩綆浜庢寚瀹氬鹼紝</span></font>Namenode<font id=nekf452 face="瀹嬩綋, SimSun"><span id=nekf453 lang=zh-CN>涓嶆柇鍦拌窡韙渶瑕佸鍒剁殑 </span></font>block<font id=nekf454 face="瀹嬩綋, SimSun"><span id=nekf455 lang=zh-CN>錛屽湪浠諱綍闇瑕佺殑鎯呭喌涓嬪惎鍔ㄥ鍒躲傚湪涓嬪垪鎯呭喌鍙兘闇瑕侀噸鏂板鍒訛細鏌愪釜</span></font>Datanode<font id=nekf456 face="瀹嬩綋, SimSun"><span id=nekf457 lang=zh-CN>鑺傜偣澶辨晥錛屾煇涓壇鏈伃鍒版崯鍧忥紝</span></font>Datanode<font id=nekf458 face="瀹嬩綋, SimSun"><span id=nekf459 lang=zh-CN>涓婄殑紜洏閿欒錛屾垨鑰呮枃浠剁殑</span></font>replication<font id=nekf460 face="瀹嬩綋, SimSun"><span id=nekf461 lang=zh-CN>鍥犲瓙澧炲ぇ銆?/span></font><br id=nekf462><br id=nekf463>2<font id=nekf464 face="瀹嬩綋, SimSun"><span id=nekf465 lang=zh-CN>銆侀泦緹ゅ潎琛?/span></font><br id=nekf466>   HDFS<font id=nekf467 face="瀹嬩綋, SimSun"><span id=nekf468 lang=zh-CN>鏀寔鏁版嵁鐨勫潎琛¤鍒掞紝濡傛灉鏌愪釜</span></font>Datanode<font id=nekf469 face="瀹嬩綋, SimSun"><span id=nekf470 lang=zh-CN>鑺傜偣涓婄殑絀洪棽絀洪棿浣庝簬鐗瑰畾鐨勪復鐣岀偣錛岄偅涔堝氨浼氬惎鍔ㄤ竴涓鍒掕嚜鍔ㄥ湴灝嗘暟鎹粠涓涓?/span></font>Datanode<font id=nekf471 face="瀹嬩綋, SimSun"><span id=nekf472 lang=zh-CN>鎼Щ鍒扮┖闂茬殑</span></font>Datanode<font id=nekf473 face="瀹嬩綋, SimSun"><span id=nekf474 lang=zh-CN>銆傚綋瀵規煇涓枃浠剁殑璇鋒眰紿佺劧澧炲姞錛岄偅涔堜篃鍙兘鍚姩涓涓鍒掑垱寤鴻鏂囦歡鏂扮殑鍓湰錛屽茍鍒嗗竷鍒伴泦緹や腑浠ユ弧瓚沖簲鐢ㄧ殑瑕佹眰銆傝繖浜涘潎琛¤鍒掔洰鍓嶈繕娌℃湁瀹炵幇銆?/span></font><br id=nekf475><br id=nekf476>3<font id=nekf477 face="瀹嬩綋, SimSun"><span id=nekf478 lang=zh-CN>銆佹暟鎹畬鏁存?/span></font><br id=nekf479>  <font id=nekf480 face="瀹嬩綋, SimSun"><span id=nekf481 lang=zh-CN>浠庢煇涓?/span></font>Datanode<font id=nekf482 face="瀹嬩綋, SimSun"><span id=nekf483 lang=zh-CN>鑾峰彇鐨勬暟鎹潡鏈夊彲鑳芥槸鎹熷潖鐨勶紝榪欎釜鎹熷潖鍙兘鏄敱浜?/span></font>Datanode<font id=nekf484 face="瀹嬩綋, SimSun"><span id=nekf485 lang=zh-CN>鐨勫瓨鍌ㄨ澶囬敊璇佺綉緇滈敊璇垨鑰呰蔣浠?/span></font>bug<font id=nekf486 face="瀹嬩綋, SimSun"><span id=nekf487 lang=zh-CN>閫犳垚鐨勩?/span></font>HDFS<font id=nekf488 face="瀹嬩綋, SimSun"><span id=nekf489 lang=zh-CN>瀹㈡埛绔蔣浠跺疄鐜頒簡</span></font>HDFS<font id=nekf490 face="瀹嬩綋, SimSun"><span id=nekf491 lang=zh-CN>鏂囦歡鍐呭鐨勬牎楠屽拰銆傚綋鏌愪釜瀹㈡埛绔垱寤轟竴涓柊鐨?/span></font>HDFS<font id=nekf492 face="瀹嬩綋, SimSun"><span id=nekf493 lang=zh-CN>鏂囦歡錛屼細璁$畻榪欎釜鏂囦歡姣忎釜</span></font>block<font id=nekf494 face="瀹嬩綋, SimSun"><span id=nekf495 lang=zh-CN>鐨勬牎楠屽拰錛屽茍浣滀負涓涓崟鐙殑闅愯棌鏂囦歡淇濆瓨榪欎簺鏍¢獙鍜屽湪鍚屼竴涓?/span></font>HDFS namespace<font id=nekf496 face="瀹嬩綋, SimSun"><span id=nekf497 lang=zh-CN>涓嬨傚綋瀹㈡埛绔绱㈡枃浠跺唴瀹癸紝瀹冧細紜浠?/span></font>Datanode<font id=nekf498 face="瀹嬩綋, SimSun"><span id=nekf499 lang=zh-CN>鑾峰彇鐨勬暟鎹窡鐩稿簲鐨勬牎楠屽拰鏂囦歡涓殑鏍¢獙鍜屾槸鍚﹀尮閰嶏紝濡傛灉涓嶅尮閰嶏紝瀹㈡埛绔彲浠ラ夋嫨浠庡叾浠?/span></font>Datanode<font id=nekf500 face="瀹嬩綋, SimSun"><span id=nekf501 lang=zh-CN>鑾峰彇璇?/span></font>block<font id=nekf502 face="瀹嬩綋, SimSun"><span id=nekf503 lang=zh-CN>鐨勫壇鏈?/span></font><br id=nekf504><br id=nekf505>4<font id=nekf506 face="瀹嬩綋, SimSun"><span id=nekf507 lang=zh-CN>銆佸厓鏁版嵁紓佺洏閿欒</span></font><br id=nekf508>    FsImage<font id=nekf509 face="瀹嬩綋, SimSun"><span id=nekf510 lang=zh-CN>鍜?/span></font>Editlog<font id=nekf511 face="瀹嬩綋, SimSun"><span id=nekf512 lang=zh-CN>鏄?/span></font>HDFS<font id=nekf513 face="瀹嬩綋, SimSun"><span id=nekf514 lang=zh-CN>鐨勬牳蹇冩暟鎹粨鏋勩傝繖浜涙枃浠跺鏋滄崯鍧忎簡錛屾暣涓?/span></font>HDFS<font id=nekf515 face="瀹嬩綋, SimSun"><span id=nekf516 lang=zh-CN>瀹炰緥閮藉皢澶辨晥銆傚洜鑰岋紝</span></font>Namenode<font id=nekf517 face="瀹嬩綋, SimSun"><span id=nekf518 lang=zh-CN>鍙互閰嶇疆鎴愭敮鎸佺淮鎶ゅ涓?/span></font>FsImage<font id=nekf519 face="瀹嬩綋, SimSun"><span id=nekf520 lang=zh-CN>鍜?/span></font>Editlog<font id=nekf521 face="瀹嬩綋, SimSun"><span id=nekf522 lang=zh-CN>鐨勬嫹璐濄備換浣曞</span></font>FsImage<font id=nekf523 face="瀹嬩綋, SimSun"><span id=nekf524 lang=zh-CN>鎴栬?/span></font>Editlog<font id=nekf525 face="瀹嬩綋, SimSun"><span id=nekf526 lang=zh-CN>鐨勪慨鏀癸紝閮藉皢鍚屾鍒板畠浠殑鍓湰涓娿傝繖涓悓姝ユ搷浣滃彲鑳戒細闄嶄綆 </span></font>Namenode<font id=nekf527 face="瀹嬩綋, SimSun"><span id=nekf528 lang=zh-CN>姣忕鑳芥敮鎸佸鐞嗙殑</span></font>namespace<font id=nekf529 face="瀹嬩綋, SimSun"><span id=nekf530 lang=zh-CN>浜嬪姟銆傝繖涓唬浠鋒槸鍙互鎺ュ彈鐨勶紝鍥犱負</span></font>HDFS<font id=nekf531 face="瀹嬩綋, SimSun"><span id=nekf532 lang=zh-CN>鏄暟鎹瘑闆嗙殑錛岃岄潪鍏冩暟鎹瘑闆嗐傚綋</span></font>Namenode<font id=nekf533 face="瀹嬩綋, SimSun"><span id=nekf534 lang=zh-CN>閲嶅惎鐨勬椂鍊欙紝瀹冩繪槸閫夊彇鏈榪戠殑涓鑷寸殑</span></font>FsImage<font id=nekf535 face="瀹嬩綋, SimSun"><span id=nekf536 lang=zh-CN>鍜?/span></font>Editlog<font id=nekf537 face="瀹嬩綋, SimSun"><span id=nekf538 lang=zh-CN>浣跨敤銆?/span></font><br id=nekf539>   Namenode<font id=nekf540 face="瀹嬩綋, SimSun"><span id=nekf541 lang=zh-CN>鍦?/span></font>HDFS<font id=nekf542 face="瀹嬩綋, SimSun"><span id=nekf543 lang=zh-CN>鏄崟鐐瑰瓨鍦紝濡傛灉</span></font>Namenode<font id=nekf544 face="瀹嬩綋, SimSun"><span id=nekf545 lang=zh-CN>鎵鍦ㄧ殑鏈哄櫒閿欒錛屾墜宸ョ殑騫查鏄繀欏葷殑銆傜洰鍓嶏紝鍦ㄥ彟涓鍙版満鍣ㄤ笂閲嶅惎鍥犳晠闅滆屽仠姝㈡湇鍔$殑</span></font>Namenode<font id=nekf546 face="瀹嬩綋, SimSun"><span id=nekf547 lang=zh-CN>榪欎釜鍔熻兘榪樻病瀹炵幇銆?/span></font><br id=nekf548><br id=nekf549>5<font id=nekf550 face="瀹嬩綋, SimSun"><span id=nekf551 lang=zh-CN>銆佸揩鐓?/span></font><br id=nekf552>   <font id=nekf553 face="瀹嬩綋, SimSun"><span id=nekf554 lang=zh-CN>蹇収鏀寔鏌愪釜鏃墮棿鐨勬暟鎹嫹璐濓紝褰?/span></font>HDFS<font id=nekf555 face="瀹嬩綋, SimSun"><span id=nekf556 lang=zh-CN>鏁版嵁鎹熷潖鐨勬椂鍊欙紝鍙互鎭㈠鍒拌繃鍘諱竴涓凡鐭ユ紜殑鏃墮棿鐐廣?/span></font>HDFS<font id=nekf557 face="瀹嬩綋, SimSun"><span id=nekf558 lang=zh-CN>鐩墠榪樹笉鏀寔蹇収鍔熻兘銆?/span></font><br id=nekf559><br id=nekf560><font id=nekf561 face="瀹嬩綋, SimSun"><span id=nekf562 lang=zh-CN>鍏佹暟鎹粍緇?/span></font><br id=nekf563>1<font id=nekf564 face="瀹嬩綋, SimSun"><span id=nekf565 lang=zh-CN>銆佹暟鎹潡</span></font><br id=nekf566>    <font id=nekf567 face="瀹嬩綋, SimSun"><span id=nekf568 lang=zh-CN>鍏煎</span></font>HDFS<font id=nekf569 face="瀹嬩綋, SimSun"><span id=nekf570 lang=zh-CN>鐨勫簲鐢ㄩ兘鏄鐞嗗ぇ鏁版嵁闆嗗悎鐨勩傝繖浜涘簲鐢ㄩ兘鏄啓鏁版嵁涓嬈★紝璇誨嵈鏄竴嬈″埌澶氭錛屽茍涓旇鐨勯熷害瑕佹弧瓚蟲祦寮忚銆?/span></font>HDFS<font id=nekf571 face="瀹嬩綋, SimSun"><span id=nekf572 lang=zh-CN>鏀寔鏂囦歡鐨?/span></font>write- once-read-many<font id=nekf573 face="瀹嬩綋, SimSun"><span id=nekf574 lang=zh-CN>璇箟銆備竴涓吀鍨嬬殑</span></font>block<font id=nekf575 face="瀹嬩綋, SimSun"><span id=nekf576 lang=zh-CN>澶у皬鏄?/span></font>64MB<font id=nekf577 face="瀹嬩綋, SimSun"><span id=nekf578 lang=zh-CN>錛屽洜鑰岋紝鏂囦歡鎬繪槸鎸夌収</span></font>64M<font id=nekf579 face="瀹嬩綋, SimSun"><span id=nekf580 lang=zh-CN>鍒囧垎鎴?/span></font>chunk<font id=nekf581 face="瀹嬩綋, SimSun"><span id=nekf582 lang=zh-CN>錛屾瘡涓?/span></font>chunk<font id=nekf583 face="瀹嬩綋, SimSun"><span id=nekf584 lang=zh-CN>瀛樺偍浜庝笉鍚岀殑 </span></font>Datanode<br id=nekf585>2<font id=nekf586 face="瀹嬩綋, SimSun"><span id=nekf587 lang=zh-CN>銆佹楠?/span></font><br id=nekf588>    <font id=nekf589 face="瀹嬩綋, SimSun"><span id=nekf590 lang=zh-CN>鏌愪釜瀹㈡埛绔垱寤烘枃浠剁殑璇鋒眰鍏跺疄騫舵病鏈夌珛鍗沖彂緇?/span></font>Namenode<font id=nekf591 face="瀹嬩綋, SimSun"><span id=nekf592 lang=zh-CN>錛屼簨瀹炰笂錛?/span></font>HDFS<font id=nekf593 face="瀹嬩綋, SimSun"><span id=nekf594 lang=zh-CN>瀹㈡埛绔細灝嗘枃浠舵暟鎹紦瀛樺埌鏈湴鐨勪竴涓復鏃舵枃浠躲傚簲鐢ㄧ殑鍐欒閫忔槑鍦伴噸瀹氬悜鍒拌繖涓復鏃舵枃浠躲傚綋榪欎釜涓存椂鏂囦歡绱Н鐨勬暟鎹秴榪囦竴涓?/span></font>block<font id=nekf595 face="瀹嬩綋, SimSun"><span id=nekf596 lang=zh-CN>鐨勫ぇ灝忥紙榛樿</span></font>64M)<font id=nekf597 face="瀹嬩綋, SimSun"><span id=nekf598 lang=zh-CN>錛屽鎴風鎵嶄細鑱旂郴</span></font>Namenode<font id=nekf599 face="瀹嬩綋, SimSun"><span id=nekf600 lang=zh-CN>銆?/span></font>Namenode<font id=nekf601 face="瀹嬩綋, SimSun"><span id=nekf602 lang=zh-CN>灝嗘枃浠跺悕鎻掑叆鏂囦歡緋葷粺鐨勫眰嬈$粨鏋勪腑錛屽茍涓斿垎閰嶄竴涓暟鎹潡緇欏畠錛岀劧鍚庤繑鍥?/span></font>Datanode<font id=nekf603 face="瀹嬩綋, SimSun"><span id=nekf604 lang=zh-CN>鐨勬爣璇嗙鍜岀洰鏍囨暟鎹潡緇欏鎴風銆傚鎴風灝嗘湰鍦頒復鏃舵枃浠?/span></font>flush<font id=nekf605 face="瀹嬩綋, SimSun"><span id=nekf606 lang=zh-CN>鍒版寚瀹氱殑 </span></font>Datanode<font id=nekf607 face="瀹嬩綋, SimSun"><span id=nekf608 lang=zh-CN>涓娿傚綋鏂囦歡鍏抽棴鏃訛紝鍦ㄤ復鏃舵枃浠朵腑鍓╀綑鐨勬病鏈?/span></font>flush<font id=nekf609 face="瀹嬩綋, SimSun"><span id=nekf610 lang=zh-CN>鐨勬暟鎹篃浼氫紶杈撳埌鎸囧畾鐨?/span></font>Datanode<font id=nekf611 face="瀹嬩綋, SimSun"><span id=nekf612 lang=zh-CN>錛岀劧鍚庡鎴風鍛婅瘔</span></font>Namenode<font id=nekf613 face="瀹嬩綋, SimSun"><span id=nekf614 lang=zh-CN>鏂囦歡宸茬粡鍏抽棴銆傛鏃?/span></font>Namenode<font id=nekf615 face="瀹嬩綋, SimSun"><span id=nekf616 lang=zh-CN>鎵嶅皢鏂囦歡鍒涘緩鎿嶄綔鎻愪氦鍒版寔涔呭瓨鍌ㄣ傚鏋?/span></font>Namenode<font id=nekf617 face="瀹嬩綋, SimSun"><span id=nekf618 lang=zh-CN>鍦ㄦ枃浠跺叧闂墠鎸備簡錛岃鏂囦歡灝嗕涪澶便?/span></font><br id=nekf619>   <font id=nekf620 face="瀹嬩綋, SimSun"><span id=nekf621 lang=zh-CN>涓婅堪鏂規硶鏄閫氳繃瀵?/span></font>HDFS<font id=nekf622 face="瀹嬩綋, SimSun"><span id=nekf623 lang=zh-CN>涓婅繍琛岀殑鐩爣搴旂敤璁ょ湡鑰冭檻鐨勭粨鏋溿傚鏋滀笉閲囩敤瀹㈡埛绔紦瀛橈紝鐢變簬緗戠粶閫熷害鍜岀綉緇滃牭濉炰細瀵瑰悶浼伴噺閫犳垚姣旇緝澶х殑褰卞搷銆?/span></font><br id=nekf624><br id=nekf625>3<font id=nekf626 face="瀹嬩綋, SimSun"><span id=nekf627 lang=zh-CN>銆佹祦姘寸嚎澶嶅埗</span></font><br id=nekf628>    <font id=nekf629 face="瀹嬩綋, SimSun"><span id=nekf630 lang=zh-CN>褰撴煇涓鎴風鍚?/span></font>HDFS<font id=nekf631 face="瀹嬩綋, SimSun"><span id=nekf632 lang=zh-CN>鏂囦歡鍐欐暟鎹殑鏃跺欙紝涓寮濮嬫槸鍐欏叆鏈湴涓存椂鏂囦歡錛屽亣璁捐鏂囦歡鐨?/span></font>replication<font id=nekf633 face="瀹嬩綋, SimSun"><span id=nekf634 lang=zh-CN>鍥犲瓙璁劇疆涓?/span></font>3<font id=nekf635 face="瀹嬩綋, SimSun"><span id=nekf636 lang=zh-CN>錛岄偅涔堝鎴風浼氫粠</span></font>Namenode <font id=nekf637 face="瀹嬩綋, SimSun"><span id=nekf638 lang=zh-CN>鑾峰彇涓寮?/span></font>Datanode<font id=nekf639 face="瀹嬩綋, SimSun"><span id=nekf640 lang=zh-CN>鍒楄〃鏉ュ瓨鏀懼壇鏈傜劧鍚庡鎴風寮濮嬪悜絎竴涓?/span></font>Datanode<font id=nekf641 face="瀹嬩綋, SimSun"><span id=nekf642 lang=zh-CN>浼犺緭鏁版嵁錛岀涓涓?/span></font>Datanode<font id=nekf643 face="瀹嬩綋, SimSun"><span id=nekf644 lang=zh-CN>涓灝忛儴鍒嗕竴灝忛儴鍒嗭紙</span></font>4kb)<font id=nekf645 face="瀹嬩綋, SimSun"><span id=nekf646 lang=zh-CN>鍦版帴鏀舵暟鎹紝灝嗘瘡涓儴鍒嗗啓鍏ユ湰鍦頒粨搴擄紝騫朵笖鍚屾椂浼犺緭璇ラ儴鍒嗗埌絎簩涓?/span></font>Datanode<font id=nekf647 face="瀹嬩綋, SimSun"><span id=nekf648 lang=zh-CN>鑺傜偣銆傜浜屼釜</span></font>Datanode<font id=nekf649 face="瀹嬩綋, SimSun"><span id=nekf650 lang=zh-CN>涔熸槸榪欐牱錛岃竟鏀惰竟浼狅紝涓灝忛儴鍒嗕竴灝忛儴鍒嗗湴鏀訛紝瀛樺偍鍦ㄦ湰鍦頒粨搴擄紝鍚屾椂浼犵粰絎笁涓?/span></font>Datanode<font id=nekf651 face="瀹嬩綋, SimSun"><span id=nekf652 lang=zh-CN>錛岀涓変釜</span></font>Datanode<font id=nekf653 face="瀹嬩綋, SimSun"><span id=nekf654 lang=zh-CN>灝變粎浠呮槸鎺ユ敹騫跺瓨鍌ㄤ簡銆傝繖灝辨槸嫻佹按綰垮紡鐨勫鍒躲?/span></font><br id=nekf655><br id=nekf656><font id=nekf657 face="瀹嬩綋, SimSun"><span id=nekf658 lang=zh-CN>涔濄佸彲璁塊棶鎬?/span></font><br id=nekf659>    HDFS<font id=nekf660 face="瀹嬩綋, SimSun"><span id=nekf661 lang=zh-CN>緇欏簲鐢ㄦ彁渚涗簡澶氱璁塊棶鏂瑰紡錛屽彲浠ラ氳繃</span></font>DFSShell<font id=nekf662 face="瀹嬩綋, SimSun"><span id=nekf663 lang=zh-CN>閫氳繃鍛戒護琛屼笌</span></font>HDFS<font id=nekf664 face="瀹嬩綋, SimSun"><span id=nekf665 lang=zh-CN>鏁版嵁榪涜浜や簰錛屽彲浠ラ氳繃</span></font>java API<font id=nekf666 face="瀹嬩綋, SimSun"><span id=nekf667 lang=zh-CN>璋冪敤錛屼篃鍙互閫氳繃</span></font>C<font id=nekf668 face="瀹嬩綋, SimSun"><span id=nekf669 lang=zh-CN>璇█鐨勫皝瑁?/span></font>API<font id=nekf670 face="瀹嬩綋, SimSun"><span id=nekf671 lang=zh-CN>璁塊棶錛屽茍涓旀彁渚涗簡嫻忚鍣ㄨ闂殑鏂瑰紡銆傛鍦ㄥ紑鍙戦氳繃</span></font>WebDav<font id=nekf672 face="瀹嬩綋, SimSun"><span id=nekf673 lang=zh-CN>鍗忚璁塊棶鐨勬柟寮忋傚叿浣撲嬌鐢ㄥ弬鑰冩枃妗c?/span></font><br id=nekf674><font id=nekf675 face="瀹嬩綋, SimSun"><span id=nekf676 lang=zh-CN>鍗併佺┖闂寸殑鍥炴敹</span></font><br id=nekf677>1<font id=nekf678 face="瀹嬩綋, SimSun"><span id=nekf679 lang=zh-CN>銆佹枃浠剁殑鍒犻櫎鍜屾仮澶?/span></font><br id=nekf680>    <font id=nekf681 face="瀹嬩綋, SimSun"><span id=nekf682 lang=zh-CN>鐢ㄦ埛鎴栬呭簲鐢ㄥ垹闄ゆ煇涓枃浠訛紝榪欎釜鏂囦歡騫舵病鏈夌珛鍒諱粠</span></font>HDFS<font id=nekf683 face="瀹嬩綋, SimSun"><span id=nekf684 lang=zh-CN>涓垹闄ゃ傜浉鍙嶏紝</span></font>HDFS<font id=nekf685 face="瀹嬩綋, SimSun"><span id=nekf686 lang=zh-CN>灝嗚繖涓枃浠墮噸鍛藉悕錛屽茍杞Щ鍒?/span></font>/trash<font id=nekf687 face="瀹嬩綋, SimSun"><span id=nekf688 lang=zh-CN>鐩綍銆傚綋鏂囦歡榪樺湪</span></font>/trash<font id=nekf689 face="瀹嬩綋, SimSun"><span id=nekf690 lang=zh-CN>鐩綍鏃訛紝璇ユ枃浠跺彲浠ヨ榪呴熷湴鎭㈠銆傛枃浠跺湪</span></font>/trash<font id=nekf691 face="瀹嬩綋, SimSun"><span id=nekf692 lang=zh-CN>涓繚瀛樼殑鏃墮棿鏄彲閰嶇疆鐨勶紝褰撹秴榪囪繖涓椂闂達紝</span></font>Namenode<font id=nekf693 face="瀹嬩綋, SimSun"><span id=nekf694 lang=zh-CN>灝變細灝嗚鏂囦歡浠?/span></font>namespace<font id=nekf695 face="瀹嬩綋, SimSun"><span id=nekf696 lang=zh-CN>涓垹闄ゃ傛枃浠剁殑鍒犻櫎錛屼篃灝嗛噴鏀懼叧鑱旇鏂囦歡鐨勬暟鎹潡銆傛敞鎰忓埌錛屽湪鏂囦歡琚敤鎴峰垹闄ゅ拰</span></font>HDFS<font id=nekf697 face="瀹嬩綋, SimSun"><span id=nekf698 lang=zh-CN>絀洪棽絀洪棿鐨勫鍔犱箣闂翠細鏈変竴涓瓑寰呮椂闂村歡榪熴?/span></font><br id=nekf699>    <font id=nekf700 face="瀹嬩綋, SimSun"><span id=nekf701 lang=zh-CN>褰撹鍒犻櫎鐨勬枃浠惰繕淇濈暀鍦?/span></font>/trash<font id=nekf702 face="瀹嬩綋, SimSun"><span id=nekf703 lang=zh-CN>鐩綍涓殑鏃跺欙紝濡傛灉鐢ㄦ埛鎯蟲仮澶嶈繖涓枃浠訛紝鍙互媯绱㈡祻瑙?/span></font>/trash<font id=nekf704 face="瀹嬩綋, SimSun"><span id=nekf705 lang=zh-CN>鐩綍騫舵绱㈣鏂囦歡銆?/span></font>/trash<font id=nekf706 face="瀹嬩綋, SimSun"><span id=nekf707 lang=zh-CN>鐩綍浠呬粎淇濆瓨琚垹闄ゆ枃浠剁殑鏈榪戜竴嬈℃嫹璐濄?/span></font>/trash<font id=nekf708 face="瀹嬩綋, SimSun"><span id=nekf709 lang=zh-CN>鐩綍涓庡叾浠栨枃浠剁洰褰曟病鏈変粈涔堜笉鍚岋紝闄や簡涓鐐癸細</span></font>HDFS<font id=nekf710 face="瀹嬩綋, SimSun"><span id=nekf711 lang=zh-CN>鍦ㄨ鐩綍涓婂簲鐢ㄤ簡涓涓壒孌婄殑絳栫暐鏉ヨ嚜鍔ㄥ垹闄ゆ枃浠訛紝鐩墠鐨勯粯璁ょ瓥鐣ユ槸鍒犻櫎淇濈暀瓚呰繃</span></font>6<font id=nekf712 face="瀹嬩綋, SimSun"><span id=nekf713 lang=zh-CN>灝忔椂鐨勬枃浠訛紝榪欎釜絳栫暐浠ュ悗浼氬畾涔夋垚鍙厤緗殑鎺ュ彛銆?/span></font><br id=nekf714><br id=nekf715>2<font id=nekf716 face="瀹嬩綋, SimSun"><span id=nekf717 lang=zh-CN>銆?/span></font>Replication<font id=nekf718 face="瀹嬩綋, SimSun"><span id=nekf719 lang=zh-CN>鍥犲瓙鐨勫噺灝?/span></font><br id=nekf720>    <font id=nekf721 face="瀹嬩綋, SimSun"><span id=nekf722 lang=zh-CN>褰撴煇涓枃浠剁殑</span></font>replication<font id=nekf723 face="瀹嬩綋, SimSun"><span id=nekf724 lang=zh-CN>鍥犲瓙鍑忓皬錛?/span></font>Namenode<font id=nekf725 face="瀹嬩綋, SimSun"><span id=nekf726 lang=zh-CN>浼氶夋嫨瑕佸垹闄ょ殑榪囧墿鐨勫壇鏈備笅嬈″績璺蟲嫻嬪氨灝嗚淇℃伅浼犻掔粰</span></font>Datanode<font id=nekf727 face="瀹嬩綋, SimSun"><span id=nekf728 lang=zh-CN>錛?</span></font>Datanode<font id=nekf729 face="瀹嬩綋, SimSun"><span id=nekf730 lang=zh-CN>灝變細縐婚櫎鐩稿簲鐨?/span></font>block<font id=nekf731 face="瀹嬩綋, SimSun"><span id=nekf732 lang=zh-CN>騫墮噴鏀劇┖闂達紝鍚屾牱錛屽湪璋冪敤</span></font>setReplication<font id=nekf733 face="瀹嬩綋, SimSun"><span id=nekf734 lang=zh-CN>鏂規硶鍜岄泦緹や腑鐨勭┖闂茬┖闂村鍔犱箣闂翠細鏈変竴涓椂闂村歡榪熴?/span></font><br id=nekf735><br id=nekf736><font id=nekf737 face="瀹嬩綋, SimSun"><span id=nekf738 lang=zh-CN>鍙傝冭祫鏂欙細</span></font><br id=nekf739>HDFS Java API: http://hadoop.apache.org/core/docs/current/api/<br id=nekf740>HDFS source code: http://hadoop.apache.org/core/version_control.html<br id=nekf741></p> <img src ="http://www.shnenglu.com/koson/aggbug/120775.html" width = "1" height = "1" /><br><br><div align=right><a style="text-decoration:none;" href="http://www.shnenglu.com/koson/" target="_blank">koson</a> 2010-07-19 14:42 <a href="http://www.shnenglu.com/koson/archive/2010/07/19/120775.html#Feedback" target="_blank" style="text-decoration:none;">鍙戣〃璇勮</a></div>]]></description></item><item><title>Hadoop Map/Reduce鏁欑▼http://www.shnenglu.com/koson/archive/2010/07/19/120774.htmlkosonkosonMon, 19 Jul 2010 06:40:00 GMThttp://www.shnenglu.com/koson/archive/2010/07/19/120774.htmlhttp://www.shnenglu.com/koson/comments/120774.htmlhttp://www.shnenglu.com/koson/archive/2010/07/19/120774.html#Feedback0http://www.shnenglu.com/koson/comments/commentRss/120774.htmlhttp://www.shnenglu.com/koson/services/trackbacks/120774.html鐩殑
榪欑瘒鏁欑▼浠庣敤鎴風殑瑙掑害鍑哄彂錛屽叏闈㈠湴浠嬬粛浜咹adoop Map/Reduce妗嗘灦鐨勫悇涓柟闈€?/p>

鍏堝喅鏉′歡
璇峰厛紜Hadoop琚紜畨瑁呫侀厤緗拰姝e父榪愯涓傛洿澶氫俊鎭錛?/p>

Hadoop蹇熷叆闂ㄥ鍒濇浣跨敤鑰呫?
Hadoop闆嗙兢鎼緩瀵瑰ぇ瑙勬ā鍒嗗竷寮忛泦緹ゃ?
姒傝堪
Hadoop Map/Reduce鏄竴涓嬌鐢ㄧ畝鏄撶殑杞歡妗嗘灦錛屽熀浜庡畠鍐欏嚭鏉ョ殑搴旂敤紼嬪簭鑳藉榪愯鍦ㄧ敱涓婂崈涓晢鐢ㄦ満鍣ㄧ粍鎴愮殑澶у瀷闆嗙兢涓婏紝騫朵互涓縐嶅彲闈犲閿欑殑鏂瑰紡騫惰澶勭悊涓奣綰у埆鐨勬暟鎹泦銆?/p>

涓涓狹ap/Reduce 浣滀笟錛坖ob錛?閫氬父浼氭妸杈撳叆鐨勬暟鎹泦鍒囧垎涓鴻嫢騫茬嫭绔嬬殑鏁版嵁鍧楋紝鐢?map浠誨姟錛坱ask錛変互瀹屽叏騫惰鐨勬柟寮忓鐞嗗畠浠傛鏋朵細瀵筸ap鐨勮緭鍑哄厛榪涜鎺掑簭錛?鐒跺悗鎶婄粨鏋滆緭鍏ョ粰reduce浠誨姟銆傞氬父浣滀笟鐨勮緭鍏ュ拰杈撳嚭閮戒細琚瓨鍌ㄥ湪鏂囦歡緋葷粺涓?鏁翠釜妗嗘灦璐熻矗浠誨姟鐨勮皟搴﹀拰鐩戞帶錛屼互鍙婇噸鏂版墽琛屽凡緇忓け璐ョ殑浠誨姟銆?/p>

閫氬父錛孧ap/Reduce妗嗘灦鍜屽垎甯冨紡鏂囦歡緋葷粺鏄繍琛屽湪涓緇勭浉鍚岀殑鑺傜偣涓婄殑錛屼篃灝辨槸璇達紝璁$畻鑺傜偣鍜屽瓨鍌ㄨ妭鐐歸氬父鍦ㄤ竴璧楓傝繖縐嶉厤緗厑璁告鏋跺湪閭d簺宸茬粡瀛樺ソ鏁版嵁鐨勮妭鐐逛笂楂樻晥鍦拌皟搴︿換鍔★紝榪欏彲浠ヤ嬌鏁翠釜闆嗙兢鐨勭綉緇滃甫瀹借闈炲父楂樻晥鍦板埄鐢ㄣ?/p>

Map/Reduce妗嗘灦鐢變竴涓崟鐙殑master JobTracker 鍜屾瘡涓泦緹よ妭鐐逛竴涓猻lave TaskTracker鍏卞悓緇勬垚銆俶aster璐熻矗璋冨害鏋勬垚涓涓綔涓氱殑鎵鏈変換鍔★紝榪欎簺浠誨姟鍒嗗竷鍦ㄤ笉鍚岀殑slave涓婏紝master鐩戞帶瀹冧滑鐨勬墽琛岋紝閲嶆柊鎵ц宸茬粡澶辮觸鐨勪換鍔°傝宻lave浠呰礋璐f墽琛岀敱master鎸囨淳鐨勪換鍔°?/p>

搴旂敤紼嬪簭鑷沖皯搴旇鎸囨槑杈撳叆/杈撳嚭鐨勪綅緗紙璺緞錛夛紝騫墮氳繃瀹炵幇鍚堥傜殑鎺ュ彛鎴栨娊璞$被鎻愪緵map鍜宺educe鍑芥暟銆傚啀鍔犱笂鍏朵粬浣滀笟鐨勫弬鏁幫紝灝辨瀯鎴愪簡浣滀笟閰嶇疆錛坖ob configuration錛夈傜劧鍚庯紝Hadoop鐨?job client鎻愪氦浣滀笟錛坖ar鍖?鍙墽琛岀▼搴忕瓑錛夊拰閰嶇疆淇℃伅緇橨obTracker錛屽悗鑰呰礋璐e垎鍙戣繖浜涜蔣浠跺拰閰嶇疆淇℃伅緇檚lave銆佽皟搴︿換鍔″茍鐩戞帶瀹冧滑鐨勬墽琛岋紝鍚屾椂鎻愪緵鐘舵佸拰璇婃柇淇℃伅緇檍ob-client銆?/p>

铏界劧Hadoop妗嗘灦鏄敤JavaTM瀹炵幇鐨勶紝浣哅ap/Reduce搴旂敤紼嬪簭鍒欎笉涓瀹氳鐢?Java鏉ュ啓 銆?/p>

Hadoop Streaming鏄竴縐嶈繍琛屼綔涓氱殑瀹炵敤宸ュ叿錛屽畠鍏佽鐢ㄦ埛鍒涘緩鍜岃繍琛屼換浣曞彲鎵ц紼嬪簭 錛堜緥濡傦細Shell宸ュ叿錛夋潵鍋氫負mapper鍜宺educer銆?
Hadoop Pipes鏄竴涓笌SWIG鍏煎鐨凜++ API 錛堟病鏈夊熀浜嶫NITM鎶鏈級錛屽畠涔熷彲鐢ㄤ簬瀹炵幇Map/Reduce搴旂敤紼嬪簭銆?
杈撳叆涓庤緭鍑?br>Map/Reduce妗嗘灦榪愯漿鍦?lt;key, value> 閿煎涓婏紝涔熷氨鏄錛?妗嗘灦鎶婁綔涓氱殑杈撳叆鐪嬩負鏄竴緇?lt;key, value> 閿煎錛屽悓鏍蜂篃浜у嚭涓緇?<key, value> 閿煎鍋氫負浣滀笟鐨勮緭鍑猴紝榪欎袱緇勯敭鍊煎鐨勭被鍨嬪彲鑳戒笉鍚屻?/p>

妗嗘灦闇瑕佸key鍜寁alue鐨勭被(classes)榪涜搴忓垪鍖栨搷浣滐紝 鍥犳錛岃繖浜涚被闇瑕佸疄鐜?Writable鎺ュ彛銆?鍙﹀錛屼負浜嗘柟渚挎鏋舵墽琛屾帓搴忔搷浣滐紝key綾誨繀欏誨疄鐜?WritableComparable鎺ュ彛銆?/p>

涓涓狹ap/Reduce 浣滀笟鐨勮緭鍏ュ拰杈撳嚭綾誨瀷濡備笅鎵紺猴細

(input) <k1, v1> -> map -> <k2, v2> -> combine -> <k2, v2> -> reduce -> <k3, v3> (output)

渚嬪瓙錛歐ordCount v1.0
鍦ㄦ繁鍏ョ粏鑺備箣鍓嶏紝璁╂垜浠厛鐪嬩竴涓狹ap/Reduce鐨勫簲鐢ㄧず渚嬶紝浠ヤ究瀵瑰畠浠殑宸ヤ綔鏂瑰紡鏈変竴涓垵姝ョ殑璁よ瘑銆?/p>

WordCount鏄竴涓畝鍗曠殑搴旂敤錛屽畠鍙互璁$畻鍑烘寚瀹氭暟鎹泦涓瘡涓涓崟璇嶅嚭鐜扮殑嬈℃暟銆?/p>

榪欎釜搴旂敤閫傜敤浜?鍗曟満妯″紡錛?浼垎甯冨紡妯″紡 鎴?瀹屽叏鍒嗗竷寮忔ā寮?涓夌Hadoop瀹夎鏂瑰紡銆?/p>

婧愪唬鐮?br>  WordCount.java
1. package org.myorg;
2.  
3. import java.io.IOException;
4. import java.util.*;
5.  
6. import org.apache.hadoop.fs.Path;
7. import org.apache.hadoop.conf.*;
8. import org.apache.hadoop.io.*;
9. import org.apache.hadoop.mapred.*;
10. import org.apache.hadoop.util.*;
11.  
12. public class WordCount {
13.  
14.    public static class Map extends MapReduceBase implements Mapper<LongWritable, Text, Text, IntWritable> {
15.      private final static IntWritable one = new IntWritable(1);
16.      private Text word = new Text();
17.  
18.      public void map(LongWritable key, Text value, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException {
19.        String line = value.toString();
20.        StringTokenizer tokenizer = new StringTokenizer(line);
21.        while (tokenizer.hasMoreTokens()) {
22.          word.set(tokenizer.nextToken());
23.          output.collect(word, one);
24.        }
25.      }
26.    }
27.  
28.    public static class Reduce extends MapReduceBase implements Reducer<Text, IntWritable, Text, IntWritable> {
29.      public void reduce(Text key, Iterator<IntWritable> values, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException {
30.        int sum = 0;
31.        while (values.hasNext()) {
32.          sum += values.next().get();
33.        }
34.        output.collect(key, new IntWritable(sum));
35.      }
36.    }
37.  
38.    public static void main(String[] args) throws Exception {
39.      JobConf conf = new JobConf(WordCount.class);
40.      conf.setJobName("wordcount");
41.  
42.      conf.setOutputKeyClass(Text.class);
43.      conf.setOutputValueClass(IntWritable.class);
44.  
45.      conf.setMapperClass(Map.class);
46.      conf.setCombinerClass(Reduce.class);
47.      conf.setReducerClass(Reduce.class);
48.  
49.      conf.setInputFormat(TextInputFormat.class);
50.      conf.setOutputFormat(TextOutputFormat.class);
51.  
52.      FileInputFormat.setInputPaths(conf, new Path(args[0]));
53.      FileOutputFormat.setOutputPath(conf, new Path(args[1]));
54.  
55.      JobClient.runJob(conf);
57.    }
58. }
59.  

鐢ㄦ硶
鍋囪鐜鍙橀噺HADOOP_HOME瀵瑰簲瀹夎鏃剁殑鏍圭洰褰曪紝HADOOP_VERSION瀵瑰簲Hadoop鐨勫綋鍓嶅畨瑁呯増鏈紝緙栬瘧WordCount.java鏉ュ垱寤簀ar鍖咃紝鍙涓嬫搷浣滐細

$ mkdir wordcount_classes
$ javac -classpath ${HADOOP_HOME}/hadoop-${HADOOP_VERSION}-core.jar -d wordcount_classes WordCount.java
$ jar -cvf /usr/joe/wordcount.jar -C wordcount_classes/ .

鍋囪錛?/p>

/usr/joe/wordcount/input - 鏄疕DFS涓殑杈撳叆璺緞
/usr/joe/wordcount/output - 鏄疕DFS涓殑杈撳嚭璺緞
鐢ㄧず渚嬫枃鏈枃浠跺仛涓鴻緭鍏ワ細

$ bin/hadoop dfs -ls /usr/joe/wordcount/input/
/usr/joe/wordcount/input/file01
/usr/joe/wordcount/input/file02

$ bin/hadoop dfs -cat /usr/joe/wordcount/input/file01
Hello World Bye World

$ bin/hadoop dfs -cat /usr/joe/wordcount/input/file02
Hello Hadoop Goodbye Hadoop

榪愯搴旂敤紼嬪簭錛?/p>

$ bin/hadoop jar /usr/joe/wordcount.jar org.myorg.WordCount /usr/joe/wordcount/input /usr/joe/wordcount/output

杈撳嚭鏄細

$ bin/hadoop dfs -cat /usr/joe/wordcount/output/part-00000
Bye 1
Goodbye 1
Hadoop 2
Hello 2
World 2

搴旂敤紼嬪簭鑳藉浣跨敤-files閫夐」鏉ユ寚瀹氫竴涓敱閫楀彿鍒嗛殧鐨勮礬寰勫垪琛紝榪欎簺璺緞鏄痶ask鐨勫綋鍓嶅伐浣滅洰褰曘備嬌鐢ㄩ夐」-libjars鍙互鍚憁ap鍜宺educe鐨刢lasspath涓坊鍔爅ar鍖呫備嬌鐢?archives閫夐」紼嬪簭鍙互浼犻掓。妗堟枃浠跺仛涓哄弬鏁幫紝榪欎簺妗f鏂囦歡浼氳瑙e帇騫朵笖鍦╰ask鐨勫綋鍓嶅伐浣滅洰褰曚笅浼氬垱寤轟竴涓寚鍚戣В鍘嬬敓鎴愮殑鐩綍鐨勭鍙烽摼鎺ワ紙浠ュ帇緙╁寘鐨勫悕瀛楀懡鍚嶏級銆?鏈夊叧鍛戒護琛岄夐」鐨勬洿澶氱粏鑺傝鍙傝?Commands manual銆?/p>

浣跨敤-libjars鍜?files榪愯wordcount渚嬪瓙錛?br>hadoop jar hadoop-examples.jar wordcount -files cachefile.txt -libjars mylib.jar input output

瑙i噴
WordCount搴旂敤紼嬪簭闈炲父鐩存埅浜嗗綋銆?/p>

Mapper(14-26琛?涓殑map鏂規硶(18-25琛?閫氳繃鎸囧畾鐨?TextInputFormat(49琛?涓嬈″鐞嗕竴琛屻傜劧鍚庯紝瀹冮氳繃StringTokenizer 浠ョ┖鏍間負鍒嗛殧絎﹀皢涓琛屽垏鍒嗕負鑻ュ共tokens錛屼箣鍚庯紝杈撳嚭< <word>, 1> 褰㈠紡鐨勯敭鍊煎銆?/p>

瀵逛簬紺轟緥涓殑絎竴涓緭鍏ワ紝map杈撳嚭鏄細
< Hello, 1>
< World, 1>
< Bye, 1>
< World, 1>

絎簩涓緭鍏ワ紝map杈撳嚭鏄細
< Hello, 1>
< Hadoop, 1>
< Goodbye, 1>
< Hadoop, 1>

鍏充簬緇勬垚涓涓寚瀹氫綔涓氱殑map鏁扮洰鐨勭‘瀹氾紝浠ュ強濡備綍浠ユ洿綺劇粏鐨勬柟寮忓幓鎺у埗榪欎簺map錛屾垜浠皢鍦ㄦ暀紼嬬殑鍚庣畫閮ㄥ垎瀛︿範鍒版洿澶氱殑鍐呭銆?/p>

WordCount榪樻寚瀹氫簡涓涓猚ombiner (46琛?銆傚洜姝わ紝姣忔map榪愯涔嬪悗錛屼細瀵硅緭鍑烘寜鐓ey榪涜鎺掑簭錛岀劧鍚庢妸杈撳嚭浼犻掔粰鏈湴鐨刢ombiner錛堟寜鐓т綔涓氱殑閰嶇疆涓嶳educer涓鏍鳳級錛岃繘琛屾湰鍦拌仛鍚堛?/p>

絎竴涓猰ap鐨勮緭鍑烘槸錛?br>< Bye, 1>
< Hello, 1>
< World, 2>

絎簩涓猰ap鐨勮緭鍑烘槸錛?br>< Goodbye, 1>
< Hadoop, 2>
< Hello, 1>

Reducer(28-36琛?涓殑reduce鏂規硶(29-35琛? 浠呮槸灝嗘瘡涓猭ey錛堟湰渚嬩腑灝辨槸鍗曡瘝錛夊嚭鐜扮殑嬈℃暟姹傚拰銆?/p>

鍥犳榪欎釜浣滀笟鐨勮緭鍑哄氨鏄細
< Bye, 1>
< Goodbye, 1>
< Hadoop, 2>
< Hello, 2>
< World, 2>

浠g爜涓殑run鏂規硶涓寚瀹氫簡浣滀笟鐨勫嚑涓柟闈紝 渚嬪錛氶氳繃鍛戒護琛屼紶閫掕繃鏉ョ殑杈撳叆/杈撳嚭璺緞銆乲ey/value鐨勭被鍨嬨佽緭鍏?杈撳嚭鐨勬牸寮忕瓑絳塉obConf涓殑閰嶇疆淇℃伅銆傞殢鍚庣▼搴忚皟鐢ㄤ簡JobClient.runJob(55琛?鏉ユ彁浜や綔涓氬茍涓旂洃鎺у畠鐨勬墽琛屻?/p>

鎴戜滑灝嗗湪鏈暀紼嬬殑鍚庣畫閮ㄥ垎瀛︿範鏇村鐨勫叧浜嶫obConf錛?JobClient錛?Tool鍜屽叾浠栨帴鍙e強綾?class)銆?/p>

Map/Reduce - 鐢ㄦ埛鐣岄潰
榪欓儴鍒嗘枃妗d負鐢ㄦ埛灝嗕細闈復鐨凪ap/Reduce妗嗘灦涓殑鍚勪釜鐜妭鎻愪緵浜嗛傚綋鐨勭粏鑺傘傝繖搴旇浼氬府鍔╃敤鎴鋒洿緇嗙矑搴﹀湴鍘誨疄鐜般侀厤緗拰璋冧紭浣滀笟銆傜劧鑰岋紝璇鋒敞鎰忔瘡涓被/鎺ュ彛鐨刯avadoc鏂囨。鎻愪緵鏈鍏ㄩ潰鐨勬枃妗o紱鏈枃鍙槸鎯寵搗鍒版寚鍗楃殑浣滅敤銆?/p>

鎴戜滑浼氬厛鐪嬬湅Mapper鍜孯educer鎺ュ彛銆傚簲鐢ㄧ▼搴忛氬父浼氶氳繃鎻愪緵map鍜宺educe鏂規硶鏉ュ疄鐜板畠浠?/p>

鐒跺悗錛屾垜浠細璁ㄨ鍏朵粬鐨勬牳蹇冩帴鍙o紝鍏朵腑鍖呮嫭錛?JobConf錛孞obClient錛孭artitioner錛?OutputCollector錛孯eporter錛?InputFormat錛孫utputFormat絳夌瓑銆?/p>

鏈鍚庯紝鎴戜滑灝嗛氳繃璁ㄨ妗嗘灦涓竴浜涙湁鐢ㄧ殑鍔熻兘鐐癸紙渚嬪錛欴istributedCache錛?IsolationRunner絳夌瓑錛夋潵鏀跺熬銆?/p>

鏍稿績鍔熻兘鎻忚堪
搴旂敤紼嬪簭閫氬父浼氶氳繃鎻愪緵map鍜宺educe鏉ュ疄鐜?Mapper鍜孯educer鎺ュ彛錛屽畠浠粍鎴愪綔涓氱殑鏍稿績銆?/p>

Mapper
Mapper灝嗚緭鍏ラ敭鍊煎(key/value pair)鏄犲皠鍒頒竴緇勪腑闂存牸寮忕殑閿煎闆嗗悎銆?/p>

Map鏄竴綾誨皢杈撳叆璁板綍闆嗚漿鎹負涓棿鏍煎紡璁板綍闆嗙殑鐙珛浠誨姟銆?榪欑杞崲鐨勪腑闂存牸寮忚褰曢泦涓嶉渶瑕佷笌杈撳叆璁板綍闆嗙殑綾誨瀷涓鑷淬備竴涓粰瀹氱殑杈撳叆閿煎鍙互鏄犲皠鎴?涓垨澶氫釜杈撳嚭閿煎銆?/p>

Hadoop Map/Reduce妗嗘灦涓烘瘡涓涓狪nputSplit浜х敓涓涓猰ap浠誨姟錛岃屾瘡涓狪nputSplit鏄敱璇ヤ綔涓氱殑InputFormat浜х敓鐨勩?/p>

姒傛嫭鍦拌錛屽Mapper鐨勫疄鐜拌呴渶瑕侀噸鍐?JobConfigurable.configure(JobConf)鏂規硶錛岃繖涓柟娉曢渶瑕佷紶閫掍竴涓狫obConf鍙傛暟錛岀洰鐨勬槸瀹屾垚Mapper鐨勫垵濮嬪寲宸ヤ綔銆傜劧鍚庯紝妗嗘灦涓鴻繖涓換鍔$殑InputSplit涓瘡涓敭鍊煎璋冪敤涓嬈?map(WritableComparable, Writable, OutputCollector, Reporter)鎿嶄綔銆傚簲鐢ㄧ▼搴忓彲浠ラ氳繃閲嶅啓Closeable.close()鏂規硶鏉ユ墽琛岀浉搴旂殑娓呯悊宸ヤ綔銆?/p>

杈撳嚭閿煎涓嶉渶瑕佷笌杈撳叆閿煎鐨勭被鍨嬩竴鑷淬備竴涓粰瀹氱殑杈撳叆閿煎鍙互鏄犲皠鎴?涓垨澶氫釜杈撳嚭閿煎銆傞氳繃璋冪敤 OutputCollector.collect(WritableComparable,Writable)鍙互鏀墮泦杈撳嚭鐨勯敭鍊煎銆?/p>

搴旂敤紼嬪簭鍙互浣跨敤Reporter鎶ュ憡榪涘害錛岃瀹氬簲鐢ㄧ駭鍒殑鐘舵佹秷鎭紝鏇存柊Counters錛堣鏁板櫒錛夛紝鎴栬呬粎鏄〃鏄庤嚜宸辮繍琛屾甯搞?/p>

妗嗘灦闅忓悗浼氭妸涓庝竴涓壒瀹歬ey鍏寵仈鐨勬墍鏈変腑闂磋繃紼嬬殑鍊鹼紙value錛夊垎鎴愮粍錛岀劧鍚庢妸瀹冧滑浼犵粰Reducer浠ヤ駭鍑烘渶緇堢殑緇撴灉銆傜敤鎴峰彲浠ラ氳繃 JobConf.setOutputKeyComparatorClass(Class)鏉ユ寚瀹氬叿浣撹礋璐e垎緇勭殑 Comparator銆?/p>

Mapper鐨勮緭鍑鴻鎺掑簭鍚庯紝灝辮鍒掑垎緇欐瘡涓猂educer銆傚垎鍧楃殑鎬繪暟鐩拰涓涓綔涓氱殑reduce浠誨姟鐨勬暟鐩槸涓鏍風殑銆傜敤鎴峰彲浠ラ氳繃瀹炵幇鑷畾涔夌殑 Partitioner鏉ユ帶鍒跺摢涓猭ey琚垎閰嶇粰鍝釜 Reducer銆?/p>

鐢ㄦ埛鍙夋嫨閫氳繃 JobConf.setCombinerClass(Class)鎸囧畾涓涓猚ombiner錛屽畠璐熻矗瀵逛腑闂磋繃紼嬬殑杈撳嚭榪涜鏈湴鐨勮仛闆嗭紝榪欎細鏈夊姪浜庨檷浣庝粠Mapper鍒?Reducer鏁版嵁浼犺緭閲忋?/p>

榪欎簺琚帓濂藉簭鐨勪腑闂磋繃紼嬬殑杈撳嚭緇撴灉淇濆瓨鐨勬牸寮忔槸(key-len, key, value-len, value)錛屽簲鐢ㄧ▼搴忓彲浠ラ氳繃JobConf鎺у埗瀵硅繖浜涗腑闂寸粨鏋滄槸鍚﹁繘琛屽帇緙╀互鍙婃庝箞鍘嬬緝錛屼嬌鐢ㄥ摢縐?CompressionCodec銆?/p>

闇瑕佸灝戜釜Map錛?br>Map鐨勬暟鐩氬父鏄敱杈撳叆鏁版嵁鐨勫ぇ灝忓喅瀹氱殑錛屼竴鑸氨鏄墍鏈夎緭鍏ユ枃浠剁殑鎬誨潡錛坆lock錛夋暟銆?/p>

Map姝e父鐨勫茍琛岃妯″ぇ鑷存槸姣忎釜鑺傜偣錛坣ode錛夊ぇ綰?0鍒?00涓猰ap錛屽浜嶤PU 娑堣楄緝灝忕殑map浠誨姟鍙互璁懼埌300涓乏鍙熾傜敱浜庢瘡涓換鍔″垵濮嬪寲闇瑕佷竴瀹氱殑鏃墮棿錛屽洜姝わ紝姣旇緝鍚堢悊鐨勬儏鍐墊槸map鎵ц鐨勬椂闂磋嚦灝戣秴榪?鍒嗛挓銆?/p>

榪欐牱錛屽鏋滀綘杈撳叆10TB鐨勬暟鎹紝姣忎釜鍧楋紙block錛夌殑澶у皬鏄?28MB錛屼綘灝嗛渶瑕佸ぇ綰?2,000涓猰ap鏉ュ畬鎴愪換鍔★紝闄ら潪浣跨敤 setNumMapTasks(int)錛堟敞鎰忥細榪欓噷浠呬粎鏄妗嗘灦榪涜浜嗕竴涓彁紺?hint)錛屽疄闄呭喅瀹氬洜绱犺榪欓噷錛夊皢榪欎釜鏁板艱緗緱鏇撮珮銆?/p>

Reducer
Reducer灝嗕笌涓涓猭ey鍏寵仈鐨勪竴緇勪腑闂存暟鍊奸泦褰掔害錛坮educe錛変負涓涓洿灝忕殑鏁板奸泦銆?/p>

鐢ㄦ埛鍙互閫氳繃 JobConf.setNumReduceTasks(int)璁懼畾涓涓綔涓氫腑reduce浠誨姟鐨勬暟鐩?/p>

姒傛嫭鍦拌錛屽Reducer鐨勫疄鐜拌呴渶瑕侀噸鍐?JobConfigurable.configure(JobConf)鏂規硶錛岃繖涓柟娉曢渶瑕佷紶閫掍竴涓狫obConf鍙傛暟錛岀洰鐨勬槸瀹屾垚Reducer鐨勫垵濮嬪寲宸ヤ綔銆傜劧鍚庯紝妗嗘灦涓烘垚緇勭殑杈撳叆鏁版嵁涓殑姣忎釜<key, (list of values)>瀵硅皟鐢ㄤ竴嬈?reduce(WritableComparable, Iterator, OutputCollector, Reporter)鏂規硶銆備箣鍚庯紝搴旂敤紼嬪簭鍙互閫氳繃閲嶅啓Closeable.close()鏉ユ墽琛岀浉搴旂殑娓呯悊宸ヤ綔銆?/p>

Reducer鏈?涓富瑕侀樁孌碉細shuffle銆乻ort鍜宺educe銆?/p>

Shuffle
Reducer鐨勮緭鍏ュ氨鏄疢apper宸茬粡鎺掑ソ搴忕殑杈撳嚭銆傚湪榪欎釜闃舵錛屾鏋墮氳繃HTTP涓烘瘡涓猂educer鑾峰緱鎵鏈塎apper杈撳嚭涓笌涔嬬浉鍏崇殑鍒嗗潡銆?/p>

Sort
榪欎釜闃舵錛屾鏋跺皢鎸夌収key鐨勫煎Reducer鐨勮緭鍏ヨ繘琛屽垎緇?錛堝洜涓轟笉鍚宮apper鐨勮緭鍑轟腑鍙兘浼氭湁鐩稿悓鐨刱ey錛夈?/p>

Shuffle鍜孲ort涓や釜闃舵鏄悓鏃惰繘琛岀殑錛沵ap鐨勮緭鍑轟篃鏄竴杈硅鍙栧洖涓杈硅鍚堝茍鐨勩?/p>

Secondary Sort
濡傛灉闇瑕佷腑闂磋繃紼嬪key鐨勫垎緇勮鍒欏拰reduce鍓嶅key鐨勫垎緇勮鍒欎笉鍚岋紝閭d箞鍙互閫氳繃 JobConf.setOutputValueGroupingComparator(Class)鏉ユ寚瀹氫竴涓狢omparator銆傚啀鍔犱笂 JobConf.setOutputKeyComparatorClass(Class)鍙敤浜庢帶鍒朵腑闂磋繃紼嬬殑key濡備綍琚垎緇勶紝鎵浠ョ粨鍚堜袱鑰呭彲浠ュ疄鐜版寜鍊肩殑浜屾鎺掑簭銆?/p>

Reduce
鍦ㄨ繖涓樁孌碉紝妗嗘灦涓哄凡鍒嗙粍鐨勮緭鍏ユ暟鎹腑鐨勬瘡涓?<key, (list of values)>瀵硅皟鐢ㄤ竴嬈?reduce(WritableComparable, Iterator, OutputCollector, Reporter)鏂規硶銆?/p>

Reduce浠誨姟鐨勮緭鍑洪氬父鏄氳繃璋冪敤 OutputCollector.collect(WritableComparable, Writable)鍐欏叆 鏂囦歡緋葷粺鐨勩?/p>

搴旂敤紼嬪簭鍙互浣跨敤Reporter鎶ュ憡榪涘害錛岃瀹氬簲鐢ㄧ▼搴忕駭鍒殑鐘舵佹秷鎭紝鏇存柊Counters錛堣鏁板櫒錛夛紝鎴栬呬粎鏄〃鏄庤嚜宸辮繍琛屾甯搞?/p>

Reducer鐨勮緭鍑烘槸娌℃湁鎺掑簭鐨勩?/p>

闇瑕佸灝戜釜Reduce錛?br>Reduce鐨勬暟鐩緩璁槸0.95鎴?.75涔樹互 (<no. of nodes> * mapred.tasktracker.reduce.tasks.maximum)銆?/p>

鐢?.95錛屾墍鏈塺educe鍙互鍦╩aps涓瀹屾垚鏃跺氨绔嬪埢鍚姩錛屽紑濮嬩紶杈搈ap鐨勮緭鍑虹粨鏋溿傜敤1.75錛岄熷害蹇殑鑺傜偣鍙互鍦ㄥ畬鎴愮涓杞畆educe浠誨姟鍚庯紝鍙互寮濮嬬浜岃疆錛岃繖鏍峰彲浠ュ緱鍒版瘮杈冨ソ鐨勮礋杞藉潎琛$殑鏁堟灉銆?/p>

澧炲姞reduce鐨勬暟鐩細澧炲姞鏁翠釜妗嗘灦鐨勫紑閿錛屼絾鍙互鏀瑰杽璐熻澆鍧囪 錛岄檷浣庣敱浜庢墽琛屽け璐ュ甫鏉ョ殑璐熼潰褰卞搷銆?/p>

涓婅堪姣斾緥鍥犲瓙姣旀暣浣撴暟鐩◢灝忎竴浜涙槸涓轟簡緇欐鏋朵腑鐨勬帹嫻嬫т換鍔★紙speculative-tasks錛?鎴栧け璐ョ殑浠誨姟棰勭暀涓浜況educe鐨勮祫婧愩?/p>

鏃燫educer
濡傛灉娌℃湁褰掔害瑕佽繘琛岋紝閭d箞璁劇疆reduce浠誨姟鐨勬暟鐩負闆舵槸鍚堟硶鐨勩?/p>

榪欑鎯呭喌涓嬶紝map浠誨姟鐨勮緭鍑轟細鐩存帴琚啓鍏ョ敱 setOutputPath(Path)鎸囧畾鐨勮緭鍑鴻礬寰勩傛鏋跺湪鎶婂畠浠啓鍏ileSystem涔嬪墠娌℃湁瀵瑰畠浠繘琛屾帓搴忋?/p>

Partitioner
Partitioner鐢ㄤ簬鍒掑垎閿肩┖闂達紙key space錛夈?/p>

Partitioner璐熻矗鎺у埗map杈撳嚭緇撴灉key鐨勫垎鍓層侹ey錛堟垨鑰呬竴涓猭ey瀛愰泦錛夎鐢ㄤ簬浜х敓鍒嗗尯錛岄氬父浣跨敤鐨勬槸Hash鍑芥暟銆傚垎鍖虹殑鏁扮洰涓庝竴涓綔涓氱殑reduce浠誨姟鐨勬暟鐩槸涓鏍風殑銆傚洜姝わ紝瀹冩帶鍒跺皢涓棿榪囩▼鐨刱ey錛堜篃灝辨槸榪欐潯璁板綍錛夊簲璇ュ彂閫佺粰m涓猺educe浠誨姟涓殑鍝竴涓潵榪涜reduce鎿嶄綔銆?/p>

HashPartitioner鏄粯璁ょ殑 Partitioner銆?/p>

Reporter
Reporter鏄敤浜嶮ap/Reduce搴旂敤紼嬪簭鎶ュ憡榪涘害錛岃瀹氬簲鐢ㄧ駭鍒殑鐘舵佹秷鎭紝 鏇存柊Counters錛堣鏁板櫒錛夌殑鏈哄埗銆?/p>

Mapper鍜孯educer鐨勫疄鐜板彲浠ュ埄鐢≧eporter 鏉ユ姤鍛婅繘搴︼紝鎴栬呬粎鏄〃鏄庤嚜宸辮繍琛屾甯搞傚湪閭g搴旂敤紼嬪簭闇瑕佽姳寰堥暱鏃墮棿澶勭悊涓埆閿煎鐨勫満鏅腑錛岃繖縐嶆満鍒舵槸寰堝叧閿殑錛屽洜涓烘鏋跺彲鑳戒細浠ヤ負榪欎釜浠誨姟瓚呮椂浜嗭紝浠庤屽皢瀹冨己琛屾潃姝匯傚彟涓涓伩鍏嶈繖縐嶆儏鍐靛彂鐢熺殑鏂瑰紡鏄紝灝嗛厤緗弬鏁癿apred.task.timeout璁劇疆涓轟竴涓凍澶熼珮鐨勫鹼紙鎴栬呭共鑴嗚緗負闆訛紝鍒欐病鏈夎秴鏃墮檺鍒朵簡錛夈?/p>

搴旂敤紼嬪簭鍙互鐢≧eporter鏉ユ洿鏂癈ounter錛堣鏁板櫒錛夈?/p>

OutputCollector
OutputCollector鏄竴涓狹ap/Reduce妗嗘灦鎻愪緵鐨勭敤浜庢敹闆?Mapper鎴朢educer杈撳嚭鏁版嵁鐨勯氱敤鏈哄埗 錛堝寘鎷腑闂磋緭鍑虹粨鏋滃拰浣滀笟鐨勮緭鍑虹粨鏋滐級銆?/p>

Hadoop Map/Reduce妗嗘灦闄勫甫浜嗕竴涓寘鍚澶氬疄鐢ㄥ瀷鐨刴apper銆乺educer鍜宲artitioner 鐨勭被搴撱?/p>

浣滀笟閰嶇疆
JobConf浠h〃涓涓狹ap/Reduce浣滀笟鐨勯厤緗?/p>

JobConf鏄敤鎴峰悜Hadoop妗嗘灦鎻忚堪涓涓狹ap/Reduce浣滀笟濡備綍鎵ц鐨勪富瑕佹帴鍙c傛鏋朵細鎸夌収JobConf鎻忚堪鐨勪俊鎭繝瀹炲湴鍘誨皾璇曞畬鎴愯繖涓綔涓氾紝鐒惰岋細

涓浜涘弬鏁板彲鑳戒細琚鐞嗚呮爣璁頒負 final錛岃繖鎰忓懗瀹冧滑涓嶈兘琚洿鏀廣?
涓浜涗綔涓氱殑鍙傛暟鍙互琚洿鎴簡褰撳湴榪涜璁劇疆錛堜緥濡傦細 setNumReduceTasks(int)錛夛紝鑰屽彟涓浜涘弬鏁板垯涓庢鏋舵垨鑰呬綔涓氱殑鍏朵粬鍙傛暟涔嬮棿寰鍦扮浉浜掑獎鍝嶏紝騫朵笖璁劇疆璧鋒潵姣旇緝澶嶆潅錛堜緥濡傦細 setNumMapTasks(int)錛夈?
閫氬父錛孞obConf浼氭寚鏄嶮apper銆丆ombiner(濡傛灉鏈夌殑璇?銆?Partitioner銆丷educer銆両nputFormat鍜?OutputFormat鐨勫叿浣撳疄鐜般侸obConf榪樿兘鎸囧畾涓緇勮緭鍏ユ枃浠?(setInputPaths(JobConf, Path...) /addInputPath(JobConf, Path)) 鍜?setInputPaths(JobConf, String) /addInputPaths(JobConf, String)) 浠ュ強杈撳嚭鏂囦歡搴旇鍐欏湪鍝効 (setOutputPath(Path))銆?/p>

JobConf鍙夋嫨鍦板浣滀笟璁劇疆涓浜涢珮綰ч夐」錛屼緥濡傦細璁劇疆Comparator錛?鏀懼埌DistributedCache涓婄殑鏂囦歡錛涗腑闂寸粨鏋滄垨鑰呬綔涓氳緭鍑虹粨鏋滄槸鍚﹂渶瑕佸帇緙╀互鍙婃庝箞鍘嬬緝錛?鍒╃敤鐢ㄦ埛鎻愪緵鐨勮剼鏈?setMapDebugScript(String)/setReduceDebugScript(String)) 榪涜璋冭瘯錛涗綔涓氭槸鍚﹀厑璁擱闃叉э紙speculative錛変換鍔$殑鎵ц (setMapSpeculativeExecution(boolean))/(setReduceSpeculativeExecution(boolean)) 錛涙瘡涓換鍔℃渶澶х殑灝濊瘯嬈℃暟 (setMaxMapAttempts(int)/setMaxReduceAttempts(int)) 錛涗竴涓綔涓氳兘瀹瑰繊鐨勪換鍔″け璐ョ殑鐧懼垎姣?(setMaxMapTaskFailuresPercent(int)/setMaxReduceTaskFailuresPercent(int)) 錛涚瓑絳夈?/p>

褰撶劧錛岀敤鎴瘋兘浣跨敤 set(String, String)/get(String, String) 鏉ヨ緗垨鑰呭彇寰楀簲鐢ㄧ▼搴忛渶瑕佺殑浠繪剰鍙傛暟銆傜劧鑰岋紝DistributedCache鐨勪嬌鐢ㄦ槸闈㈠悜澶ц妯″彧璇繪暟鎹殑銆?/p>

浠誨姟鐨勬墽琛屽拰鐜
TaskTracker鏄湪涓涓崟鐙殑jvm涓婁互瀛愯繘紼嬬殑褰㈠紡鎵ц Mapper/Reducer浠誨姟錛圱ask錛夌殑銆?/p>

瀛愪換鍔′細緇ф壙鐖禩askTracker鐨勭幆澧冦傜敤鎴峰彲浠ラ氳繃JobConf涓殑 mapred.child.java.opts閰嶇疆鍙傛暟鏉ヨ瀹氬瓙jvm涓婄殑闄勫姞閫夐」錛屼緥濡傦細 閫氳繃-Djava.library.path=<> 灝嗕竴涓潪鏍囧噯璺緞璁句負榪愯鏃剁殑閾炬帴鐢ㄤ互鎼滅儲鍏變韓搴擄紝絳夌瓑銆傚鏋渕apred.child.java.opts鍖呭惈涓涓鍙稝taskid@錛?瀹冧細琚浛鎹㈡垚map/reduce鐨則askid鐨勫箋?/p>

涓嬮潰鏄竴涓寘鍚涓弬鏁板拰鏇挎崲鐨勪緥瀛愶紝鍏朵腑鍖呮嫭錛氳褰昷vm GC鏃ュ織錛?JVM JMX浠g悊紼嬪簭浠ユ棤瀵嗙爜鐨勬柟寮忓惎鍔紝榪欐牱瀹冨氨鑳借繛鎺ュ埌jconsole涓婏紝浠庤屽彲浠ユ煡鐪嬪瓙榪涚▼鐨勫唴瀛樺拰綰跨▼錛屽緱鍒扮嚎紼嬬殑dump錛涜繕鎶婂瓙jvm鐨勬渶澶у爢灝哄璁劇疆涓?12MB錛?騫朵負瀛恓vm鐨刯ava.library.path娣誨姞浜嗕竴涓檮鍔犺礬寰勩?/p>

<property>
  <name>mapred.child.java.opts</name>
  <value>
     -Xmx512M -Djava.library.path=/home/mycompany/lib -verbose:gc -Xloggc:/tmp/@taskid@.gc
     -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false
  </value>
</property>

鐢ㄦ埛鎴栫鐞嗗憳涔熷彲浠ヤ嬌鐢╩apred.child.ulimit璁懼畾榪愯鐨勫瓙浠誨姟鐨勬渶澶ц櫄鎷熷唴瀛樸俶apred.child.ulimit鐨勫間互錛圞B)涓哄崟浣嶏紝騫朵笖蹇呴』澶т簬鎴栫瓑浜?Xmx鍙傛暟浼犵粰JavaVM鐨勫鹼紝鍚﹀垯VM浼氭棤娉曞惎鍔ㄣ?/p>

娉ㄦ剰錛歮apred.child.java.opts鍙敤浜庤緗畉ask tracker鍚姩鐨勫瓙浠誨姟銆備負瀹堟姢榪涚▼璁劇疆鍐呭瓨閫夐」璇鋒煡鐪?cluster_setup.html

${mapred.local.dir}/taskTracker/鏄痶ask tracker鐨勬湰鍦扮洰褰曪紝 鐢ㄤ簬鍒涘緩鏈湴緙撳瓨鍜宩ob銆傚畠鍙互鎸囧畾澶氫釜鐩綍錛堣法瓚婂涓鐩橈級錛屾枃浠朵細鍗婇殢鏈虹殑淇濆瓨鍒版湰鍦拌礬寰勪笅鐨勬煇涓洰褰曘傚綋job鍚姩鏃訛紝task tracker鏍規嵁閰嶇疆鏂囨。鍒涘緩鏈湴job鐩綍錛岀洰褰曠粨鏋勫浠ヤ笅鎵紺猴細

${mapred.local.dir}/taskTracker/archive/ :鍒嗗竷寮忕紦瀛樸傝繖涓洰褰曚繚瀛樻湰鍦扮殑鍒嗗竷寮忕紦瀛樸傚洜姝ゆ湰鍦板垎甯冨紡緙撳瓨鏄湪鎵鏈塼ask鍜宩ob闂村叡浜殑銆?
${mapred.local.dir}/taskTracker/jobcache/$jobid/ : 鏈湴job鐩綍銆?
${mapred.local.dir}/taskTracker/jobcache/$jobid/work/: job鎸囧畾鐨勫叡浜洰褰曘傚悇涓換鍔″彲浠ヤ嬌鐢ㄨ繖涓┖闂村仛涓烘殏瀛樼┖闂達紝鐢ㄤ簬瀹冧滑涔嬮棿鍏變韓鏂囦歡銆傝繖涓洰褰曢氳繃job.local.dir 鍙傛暟鏆撮湶緇欑敤鎴楓傝繖涓礬寰勫彲浠ラ氳繃API JobConf.getJobLocalDir()鏉ヨ闂傚畠涔熷彲浠ヨ鍋氫負緋葷粺灞炴ц幏寰椼傚洜姝わ紝鐢ㄦ埛錛堟瘮濡傝繍琛宻treaming錛夊彲浠ヨ皟鐢⊿ystem.getProperty("job.local.dir")鑾峰緱璇ョ洰褰曘?
${mapred.local.dir}/taskTracker/jobcache/$jobid/jars/: 瀛樻斁jar鍖呯殑璺緞錛岀敤浜庡瓨鏀句綔涓氱殑jar鏂囦歡鍜屽睍寮鐨刯ar銆俲ob.jar鏄簲鐢ㄧ▼搴忕殑jar鏂囦歡錛屽畠浼氳鑷姩鍒嗗彂鍒板悇鍙版満鍣紝鍦╰ask鍚姩鍓嶄細琚嚜鍔ㄥ睍寮銆備嬌鐢╝pi JobConf.getJar() 鍑芥暟鍙互寰楀埌job.jar鐨勪綅緗備嬌鐢↗obConf.getJar().getParent()鍙互璁塊棶瀛樻斁灞曞紑鐨刯ar鍖呯殑鐩綍銆?
${mapred.local.dir}/taskTracker/jobcache/$jobid/job.xml錛?涓涓猨ob.xml鏂囦歡錛屾湰鍦扮殑閫氱敤鐨勪綔涓氶厤緗枃浠躲?
${mapred.local.dir}/taskTracker/jobcache/$jobid/$taskid錛?姣忎釜浠誨姟鏈変竴涓洰褰晅ask-id錛屽畠閲岄潰鏈夊涓嬬殑鐩綍緇撴瀯錛?
${mapred.local.dir}/taskTracker/jobcache/$jobid/$taskid/job.xml錛?涓涓猨ob.xml鏂囦歡錛屾湰鍦板寲鐨勪換鍔′綔涓氶厤緗枃浠躲備換鍔℃湰鍦板寲鏄寚涓鴻task璁懼畾鐗瑰畾鐨勫睘鎬у箋傝繖浜涘間細鍦ㄤ笅闈㈠叿浣撹鏄庛?
${mapred.local.dir}/taskTracker/jobcache/$jobid/$taskid/output 涓涓瓨鏀句腑闂磋繃紼嬬殑杈撳嚭鏂囦歡鐨勭洰褰曘傚畠淇濆瓨浜嗙敱framwork浜х敓鐨勪復鏃秏ap reduce鏁版嵁錛屾瘮濡俶ap鐨勮緭鍑烘枃浠剁瓑銆?
${mapred.local.dir}/taskTracker/jobcache/$jobid/$taskid/work錛?task鐨勫綋鍓嶅伐浣滅洰褰曘?
${mapred.local.dir}/taskTracker/jobcache/$jobid/$taskid/work/tmp錛?task鐨勪復鏃剁洰褰曘傦紙鐢ㄦ埛鍙互璁懼畾灞炴apred.child.tmp 鏉ヤ負map鍜宺educe task璁懼畾涓存椂鐩綍銆傜己鐪佸兼槸./tmp銆傚鏋滆繖涓間笉鏄粷瀵硅礬寰勶紝 瀹冧細鎶妕ask鐨勫伐浣滆礬寰勫姞鍒拌璺緞鍓嶉潰浣滀負task鐨勪復鏃舵枃浠惰礬寰勩傚鏋滆繖涓兼槸緇濆璺緞鍒欑洿鎺ヤ嬌鐢ㄨ繖涓箋?濡傛灉鎸囧畾鐨勭洰褰曚笉瀛樺湪錛屼細鑷姩鍒涘緩璇ョ洰褰曘備箣鍚庯紝鎸夌収閫夐」 -Djava.io.tmpdir='涓存椂鏂囦歡鐨勭粷瀵硅礬寰?鎵цjava瀛愪換鍔°?pipes鍜宻treaming鐨勪復鏃舵枃浠惰礬寰勬槸閫氳繃鐜鍙橀噺TMPDIR='the absolute path of the tmp dir'璁懼畾鐨勶級銆?濡傛灉mapred.child.tmp鏈?/tmp鍊鹼紝榪欎釜鐩綍浼氳鍒涘緩銆?
涓嬮潰鐨勫睘鎬ф槸涓烘瘡涓猼ask鎵ц鏃朵嬌鐢ㄧ殑鏈湴鍙傛暟錛屽畠浠繚瀛樺湪鏈湴鍖栫殑浠誨姟浣滀笟閰嶇疆鏂囦歡閲岋細

鍚嶇О 綾誨瀷 鎻忚堪
mapred.job.id String job id
mapred.jar String job鐩綍涓媕ob.jar鐨勪綅緗?
job.local.dir String job鎸囧畾鐨勫叡浜瓨鍌ㄧ┖闂?
mapred.tip.id String task id
mapred.task.id String task灝濊瘯id
mapred.task.is.map boolean 鏄惁鏄痬ap task
mapred.task.partition int task鍦╦ob涓殑id
map.input.file String map璇誨彇鐨勬枃浠跺悕
map.input.start long map杈撳叆鐨勬暟鎹潡鐨勮搗濮嬩綅緗亸縐?
map.input.length long map杈撳叆鐨勬暟鎹潡鐨勫瓧鑺傛暟
mapred.work.output.dir String task涓存椂杈撳嚭鐩綍

task鐨勬爣鍑嗚緭鍑哄拰閿欒杈撳嚭嫻佷細琚鍒癟askTracker涓紝騫朵笖璁板綍鍒?${HADOOP_LOG_DIR}/userlogs

DistributedCache 鍙敤浜巑ap鎴杛educe task涓垎鍙慾ar鍖呭拰鏈湴搴撱傚瓙jvm鎬繪槸鎶?褰撳墠宸ヤ綔鐩綍 鍔犲埌 java.library.path 鍜?LD_LIBRARY_PATH銆?鍥犳錛屽彲浠ラ氳繃 System.loadLibrary鎴?System.load瑁呰澆緙撳瓨鐨勫簱銆傛湁鍏充嬌鐢ㄥ垎甯冨紡緙撳瓨鍔犺澆鍏變韓搴撶殑緇嗚妭璇峰弬鑰?native_libraries.html

浣滀笟鐨勬彁浜や笌鐩戞帶
JobClient鏄敤鎴鋒彁浜ょ殑浣滀笟涓嶫obTracker浜や簰鐨勪富瑕佹帴鍙c?/p>

JobClient 鎻愪緵鎻愪氦浣滀笟錛岃拷韙繘紼嬶紝璁塊棶瀛愪換鍔$殑鏃ュ織璁板綍錛岃幏寰桵ap/Reduce闆嗙兢鐘舵佷俊鎭瓑鍔熻兘銆?/p>

浣滀笟鎻愪氦榪囩▼鍖呮嫭錛?/p>

媯鏌ヤ綔涓氳緭鍏ヨ緭鍑烘牱寮忕粏鑺?
涓轟綔涓氳綆桰nputSplit鍊箋?
濡傛灉闇瑕佺殑璇濓紝涓轟綔涓氱殑DistributedCache寤虹珛蹇呴』鐨勭粺璁′俊鎭?
鎷瘋礉浣滀笟鐨刯ar鍖呭拰閰嶇疆鏂囦歡鍒癋ileSystem涓婄殑Map/Reduce緋葷粺鐩綍涓嬨?
鎻愪氦浣滀笟鍒癑obTracker騫朵笖鐩戞帶瀹冪殑鐘舵併?
浣滀笟鐨勫巻鍙叉枃浠惰褰曞埌鎸囧畾鐩綍鐨?_logs/history/"瀛愮洰褰曚笅銆傝繖涓寚瀹氱洰褰曠敱hadoop.job.history.user.location璁懼畾錛岄粯璁ゆ槸浣滀笟杈撳嚭鐨勭洰褰曘傚洜姝ら粯璁ゆ儏鍐典笅錛屾枃浠朵細瀛樻斁鍦╩apred.output.dir/_logs/history鐩綍涓嬨傜敤鎴峰彲浠ヨ緗甴adoop.job.history.user.location涓簄one鏉ュ仠姝㈡棩蹇楄褰曘?/p>

鐢ㄦ埛浣跨敤涓嬮潰鐨勫懡浠ゅ彲浠ョ湅鍒板湪鎸囧畾鐩綍涓嬬殑鍘嗗彶鏃ュ織璁板綍鐨勬憳瑕併?
$ bin/hadoop job -history output-dir
榪欎釜鍛戒護浼氭墦鍗板嚭浣滀笟鐨勭粏鑺傦紝浠ュ強澶辮觸鐨勫拰琚潃姝葷殑浠誨姟緇嗚妭銆?br>瑕佹煡鐪嬫湁鍏充綔涓氱殑鏇村緇嗚妭渚嬪鎴愬姛鐨勪換鍔°佹瘡涓換鍔″皾璇曠殑嬈℃暟錛坱ask attempt錛夌瓑錛屽彲浠ヤ嬌鐢ㄤ笅闈㈢殑鍛戒護
$ bin/hadoop job -history all output-dir

鐢ㄦ埛鍙互浣跨敤 OutputLogFilter 浠庤緭鍑虹洰褰曞垪琛ㄤ腑絳涢夋棩蹇楁枃浠躲?/p>

涓鑸儏鍐碉紝鐢ㄦ埛鍒╃敤JobConf鍒涘緩搴旂敤紼嬪簭騫墮厤緗綔涓氬睘鎬э紝 鐒跺悗鐢?JobClient 鎻愪氦浣滀笟騫剁洃瑙嗗畠鐨勮繘紼嬨?/p>

浣滀笟鐨勬帶鍒?br>鏈夋椂鍊欙紝鐢ㄤ竴涓崟鐙殑Map/Reduce浣滀笟騫朵笉鑳藉畬鎴愪竴涓鏉傜殑浠誨姟錛岀敤鎴蜂篃璁歌閾炬帴澶氫釜Map/Reduce浣滀笟鎵嶈銆傝繖鏄鏄撳疄鐜扮殑錛屽洜涓轟綔涓氶氬父杈撳嚭鍒板垎甯冨紡鏂囦歡緋葷粺涓婄殑錛屾墍浠ュ彲浠ユ妸榪欎釜浣滀笟鐨勮緭鍑轟綔涓轟笅涓涓綔涓氱殑杈撳叆瀹炵幇涓茶仈銆?/p>

鐒惰岋紝榪欎篃鎰忓懗鐫錛岀‘淇濇瘡涓浣滀笟瀹屾垚(鎴愬姛鎴栧け璐?鐨勮矗浠誨氨鐩存帴钀藉湪浜嗗鎴瘋韓涓娿傚湪榪欑鎯呭喌涓嬶紝鍙互鐢ㄧ殑鎺у埗浣滀笟鐨勯夐」鏈夛細

runJob(JobConf)錛氭彁浜や綔涓氾紝浠呭綋浣滀笟瀹屾垚鏃惰繑鍥炪?
submitJob(JobConf)錛氬彧鎻愪氦浣滀笟錛屼箣鍚庨渶瑕佷綘杞瀹冭繑鍥炵殑 RunningJob鍙ユ焺鐨勭姸鎬侊紝騫舵牴鎹儏鍐佃皟搴︺?
JobConf.setJobEndNotificationURI(String)錛氳緗竴涓綔涓氬畬鎴愰氱煡錛屽彲閬垮厤杞銆?
浣滀笟鐨勮緭鍏?br>InputFormat 涓篗ap/Reduce浣滀笟鎻忚堪杈撳叆鐨勭粏鑺傝鑼冦?/p>

Map/Reduce妗嗘灦鏍規嵁浣滀笟鐨処nputFormat鏉ワ細

媯鏌ヤ綔涓氳緭鍏ョ殑鏈夋晥鎬с?
鎶婅緭鍏ユ枃浠跺垏鍒嗘垚澶氫釜閫昏緫InputSplit瀹炰緥錛?騫舵妸姣忎竴瀹炰緥鍒嗗埆鍒嗗彂緇欎竴涓?Mapper銆?
鎻愪緵RecordReader鐨勫疄鐜幫紝榪欎釜RecordReader浠庨昏緫InputSplit涓幏寰楄緭鍏ヨ褰曪紝 榪欎簺璁板綍灝嗙敱Mapper澶勭悊銆?
鍩轟簬鏂囦歡鐨処nputFormat瀹炵幇錛堥氬父鏄?FileInputFormat鐨勫瓙綾伙級 榛樿琛屼負鏄寜鐓ц緭鍏ユ枃浠剁殑瀛楄妭澶у皬錛屾妸杈撳叆鏁版嵁鍒囧垎鎴愰昏緫鍒嗗潡錛坙ogical InputSplit 錛夈?鍏朵腑杈撳叆鏂囦歡鎵鍦ㄧ殑FileSystem鐨勬暟鎹潡灝哄鏄垎鍧楀ぇ灝忕殑涓婇檺銆備笅闄愬彲浠ヨ緗甿apred.min.split.size 鐨勫箋?/p>

鑰冭檻鍒拌竟鐣屾儏鍐碉紝瀵逛簬寰堝搴旂敤紼嬪簭鏉ヨ錛屽緢鏄庢樉鎸夌収鏂囦歡澶у皬榪涜閫昏緫鍒嗗壊鏄笉鑳芥弧瓚抽渶姹傜殑銆?鍦ㄨ繖縐嶆儏鍐典笅錛屽簲鐢ㄧ▼搴忛渶瑕佸疄鐜頒竴涓猂ecordReader鏉ュ鐞嗚褰曠殑杈圭晫騫朵負姣忎釜浠誨姟鎻愪緵涓涓昏緫鍒嗗潡鐨勯潰鍚戣褰曠殑瑙嗗浘銆?/p>

TextInputFormat 鏄粯璁ょ殑InputFormat銆?/p>

濡傛灉涓涓綔涓氱殑Inputformat鏄疶extInputFormat錛?騫朵笖妗嗘灦媯嫻嬪埌杈撳叆鏂囦歡鐨勫悗緙鏄?gz鍜?lzo錛屽氨浼氫嬌鐢ㄥ搴旂殑CompressionCodec鑷姩瑙e帇緙╄繖浜涙枃浠躲?浣嗘槸闇瑕佹敞鎰忥紝涓婅堪甯﹀悗緙鐨勫帇緙╂枃浠朵笉浼氳鍒囧垎錛屽茍涓旀暣涓帇緙╂枃浠朵細鍒嗙粰涓涓猰apper鏉ュ鐞嗐?/p>

InputSplit
InputSplit 鏄竴涓崟鐙殑Mapper瑕佸鐞嗙殑鏁版嵁鍧椼?/p>

涓鑸殑InputSplit 鏄瓧鑺傛牱寮忚緭鍏ワ紝鐒跺悗鐢盧ecordReader澶勭悊騫惰漿鍖栨垚璁板綍鏍峰紡銆?/p>

FileSplit 鏄粯璁ょ殑InputSplit銆?瀹冩妸 map.input.file 璁懼畾涓鴻緭鍏ユ枃浠剁殑璺緞錛岃緭鍏ユ枃浠舵槸閫昏緫鍒嗗潡鏂囦歡銆?/p>

RecordReader
RecordReader 浠嶪nputSlit璇誨叆<key, value>瀵廣?/p>

涓鑸殑錛孯ecordReader 鎶婄敱InputSplit 鎻愪緵鐨勫瓧鑺傛牱寮忕殑杈撳叆鏂囦歡錛岃漿鍖栨垚鐢盡apper澶勭悊鐨勮褰曟牱寮忕殑鏂囦歡銆?鍥犳RecordReader璐熻矗澶勭悊璁板綍鐨勮竟鐣屾儏鍐靛拰鎶婃暟鎹〃紺烘垚keys/values瀵瑰艦寮忋?/p>

浣滀笟鐨勮緭鍑?br>OutputFormat 鎻忚堪Map/Reduce浣滀笟鐨勮緭鍑烘牱寮忋?/p>

Map/Reduce妗嗘灦鏍規嵁浣滀笟鐨凮utputFormat鏉ワ細

媯楠屼綔涓氱殑杈撳嚭錛屼緥濡傛鏌ヨ緭鍑鴻礬寰勬槸鍚﹀凡緇忓瓨鍦ㄣ?
鎻愪緵涓涓猂ecordWriter鐨勫疄鐜幫紝鐢ㄦ潵杈撳嚭浣滀笟緇撴灉銆?杈撳嚭鏂囦歡淇濆瓨鍦‵ileSystem涓娿?
TextOutputFormat鏄粯璁ょ殑 OutputFormat銆?/p>

浠誨姟鐨凷ide-Effect File
鍦ㄤ竴浜涘簲鐢ㄧ▼搴忎腑錛屽瓙浠誨姟闇瑕佷駭鐢熶竴浜泂ide-file錛岃繖浜涙枃浠朵笌浣滀笟瀹為檯杈撳嚭緇撴灉鐨勬枃浠朵笉鍚屻?/p>

鍦ㄨ繖縐嶆儏鍐典笅錛屽悓涓涓狹apper鎴栬匯educer鐨勪袱涓疄渚嬶紙姣斿棰勯槻鎬т換鍔★級鍚屾椂鎵撳紑鎴栬呭啓 FileSystem涓婄殑鍚屼竴鏂囦歡灝變細浜х敓鍐茬獊銆傚洜姝ゅ簲鐢ㄧ▼搴忓湪鍐欐枃浠剁殑鏃跺欓渶瑕佷負姣忔浠誨姟灝濊瘯錛堜笉浠呬粎鏄瘡嬈′換鍔★紝姣忎釜浠誨姟鍙互灝濊瘯鎵ц寰堝嬈★級閫夊彇涓涓嫭涓鏃犱簩鐨勬枃浠跺悕(浣跨敤attemptid錛屼緥濡倀ask_200709221812_0001_m_000000_0)銆?/p>

涓轟簡閬垮厤鍐茬獊錛孧ap/Reduce妗嗘灦涓烘瘡嬈″皾璇曟墽琛屼換鍔¢兘寤虹珛鍜岀淮鎶や竴涓壒孌婄殑 ${mapred.output.dir}/_temporary/_${taskid}瀛愮洰褰曪紝榪欎釜鐩綍浣嶄簬鏈灝濊瘯鎵ц浠誨姟杈撳嚭緇撴灉鎵鍦ㄧ殑FileSystem涓婏紝鍙互閫氳繃 ${mapred.work.output.dir}鏉ヨ闂繖涓瓙鐩綍銆?瀵逛簬鎴愬姛瀹屾垚鐨勪換鍔″皾璇曪紝鍙湁${mapred.output.dir}/_temporary/_${taskid}涓嬬殑鏂囦歡浼氱Щ鍔ㄥ埌${mapred.output.dir}銆傚綋鐒訛紝妗嗘灦浼氫涪寮冮偅浜涘け璐ョ殑浠誨姟灝濊瘯鐨勫瓙鐩綍銆傝繖縐嶅鐞嗚繃紼嬪浜庡簲鐢ㄧ▼搴忔潵璇存槸瀹屽叏閫忔槑鐨勩?/p>

鍦ㄤ換鍔℃墽琛屾湡闂達紝搴旂敤紼嬪簭鍦ㄥ啓鏂囦歡鏃跺彲浠ュ埄鐢ㄨ繖涓壒鎬э紝姣斿 閫氳繃 FileOutputFormat.getWorkOutputPath()鑾峰緱${mapred.work.output.dir}鐩綍錛?騫跺湪鍏朵笅鍒涘緩浠繪剰浠誨姟鎵ц鏃舵墍闇鐨剆ide-file錛屾鏋跺湪浠誨姟灝濊瘯鎴愬姛鏃朵細椹笂縐誨姩榪欎簺鏂囦歡錛屽洜姝や笉闇瑕佸湪紼嬪簭鍐呬負姣忔浠誨姟灝濊瘯閫夊彇涓涓嫭涓鏃犱簩鐨勫悕瀛椼?/p>

娉ㄦ剰錛氬湪姣忔浠誨姟灝濊瘯鎵ц鏈熼棿錛?{mapred.work.output.dir} 鐨勫煎疄闄呬笂鏄?${mapred.output.dir}/_temporary/_{$taskid}錛岃繖涓兼槸Map/Reduce妗嗘灦鍒涘緩鐨勩?鎵浠ヤ嬌鐢ㄨ繖涓壒鎬х殑鏂規硶鏄紝鍦?FileOutputFormat.getWorkOutputPath() 璺緞涓嬪垱寤簊ide-file鍗沖彲銆?/p>

瀵逛簬鍙嬌鐢╩ap涓嶄嬌鐢╮educe鐨勪綔涓氾紝榪欎釜緇撹涔熸垚绔嬨傝繖縐嶆儏鍐典笅錛宮ap鐨勮緭鍑虹粨鏋滅洿鎺ョ敓鎴愬埌HDFS涓娿?/p>

RecordWriter
RecordWriter 鐢熸垚<key, value> 瀵瑰埌杈撳嚭鏂囦歡銆?/p>

RecordWriter鐨勫疄鐜版妸浣滀笟鐨勮緭鍑虹粨鏋滃啓鍒?FileSystem銆?/p>

鍏朵粬鏈夌敤鐨勭壒鎬?br>Counters
Counters 鏄涓敱Map/Reduce妗嗘灦鎴栬呭簲鐢ㄧ▼搴忓畾涔夌殑鍏ㄥ眬璁℃暟鍣ㄣ?姣忎竴涓狢ounter鍙互鏄換浣曚竴縐?Enum綾誨瀷銆傚悓涓鐗瑰畾Enum綾誨瀷鐨凜ounter鍙互姹囬泦鍒頒竴涓粍錛屽叾綾誨瀷涓篊ounters.Group銆?/p>

搴旂敤紼嬪簭鍙互瀹氫箟浠繪剰(Enum綾誨瀷)鐨凜ounters騫朵笖鍙互閫氳繃 map 鎴栬?reduce鏂規硶涓殑 Reporter.incrCounter(Enum, long)鎴栬?Reporter.incrCounter(String, String, long) 鏇存柊銆備箣鍚庢鏋朵細姹囨昏繖浜涘叏灞counters銆?/p>

DistributedCache
DistributedCache 鍙皢鍏蜂綋搴旂敤鐩稿叧鐨勩佸ぇ灝哄鐨勩佸彧璇葷殑鏂囦歡鏈夋晥鍦板垎甯冩斁緗?/p>

DistributedCache 鏄疢ap/Reduce妗嗘灦鎻愪緵鐨勫姛鑳斤紝鑳藉緙撳瓨搴旂敤紼嬪簭鎵闇鐨勬枃浠?錛堝寘鎷枃鏈紝妗f鏂囦歡錛宩ar鏂囦歡絳夛級銆?/p>

搴旂敤紼嬪簭鍦↗obConf涓氳繃url(hdfs://)鎸囧畾闇瑕佽緙撳瓨鐨勬枃浠躲?DistributedCache鍋囧畾鐢県dfs://鏍煎紡url鎸囧畾鐨勬枃浠跺凡緇忓湪 FileSystem涓婁簡銆?/p>

Map-Redcue妗嗘灦鍦ㄤ綔涓氭墍鏈変換鍔℃墽琛屼箣鍓嶄細鎶婂繀瑕佺殑鏂囦歡鎷瘋礉鍒皊lave鑺傜偣涓娿?瀹冭繍琛岄珮鏁堟槸鍥犱負姣忎釜浣滀笟鐨勬枃浠跺彧鎷瘋礉涓嬈″茍涓斾負閭d簺娌℃湁鏂囨。鐨剆lave鑺傜偣緙撳瓨鏂囨。銆?/p>

DistributedCache 鏍規嵁緙撳瓨鏂囨。淇敼鐨勬椂闂存埑榪涜榪借釜銆?鍦ㄤ綔涓氭墽琛屾湡闂達紝褰撳墠搴旂敤紼嬪簭鎴栬呭閮ㄧ▼搴忎笉鑳戒慨鏀圭紦瀛樻枃浠躲?/p>

distributedCache鍙互鍒嗗彂綆鍗曠殑鍙鏁版嵁鎴栨枃鏈枃浠訛紝涔熷彲浠ュ垎鍙戝鏉傜被鍨嬬殑鏂囦歡渚嬪褰掓。鏂囦歡鍜宩ar鏂囦歡銆傚綊妗f枃浠?zip,tar,tgz鍜宼ar.gz鏂囦歡)鍦╯lave鑺傜偣涓婁細琚В妗o紙un-archived錛夈?榪欎簺鏂囦歡鍙互璁劇疆鎵ц鏉冮檺銆?/p>

鐢ㄦ埛鍙互閫氳繃璁劇疆mapred.cache.{files|archives}鏉ュ垎鍙戞枃浠躲?濡傛灉瑕佸垎鍙戝涓枃浠訛紝鍙互浣跨敤閫楀彿鍒嗛殧鏂囦歡鎵鍦ㄨ礬寰勩備篃鍙互鍒╃敤API鏉ヨ緗灞炴э細 DistributedCache.addCacheFile(URI,conf)/ DistributedCache.addCacheArchive(URI,conf) and DistributedCache.setCacheFiles(URIs,conf)/ DistributedCache.setCacheArchives(URIs,conf) 鍏朵腑URI鐨勫艦寮忔槸 hdfs://host:port/absolute-path#link-name 鍦⊿treaming紼嬪簭涓紝鍙互閫氳繃鍛戒護琛岄夐」 -cacheFile/-cacheArchive 鍒嗗彂鏂囦歡銆?/p>

鐢ㄦ埛鍙互閫氳繃 DistributedCache.createSymlink(Configuration)鏂規硶璁〥istributedCache 鍦ㄥ綋鍓嶅伐浣滅洰褰曚笅鍒涘緩鍒扮紦瀛樻枃浠剁殑絎﹀彿閾炬帴銆?鎴栬呴氳繃璁劇疆閰嶇疆鏂囦歡灞炴apred.create.symlink涓簓es銆?鍒嗗竷寮忕紦瀛樹細鎴彇URI鐨勭墖孌典綔涓洪摼鎺ョ殑鍚嶅瓧銆?渚嬪錛孶RI鏄?hdfs://namenode:port/lib.so.1#lib.so錛?鍒欏湪task褰撳墠宸ヤ綔鐩綍浼氭湁鍚嶄負lib.so鐨勯摼鎺ワ紝 瀹冧細閾炬帴鍒嗗竷寮忕紦瀛樹腑鐨刲ib.so.1銆?/p>

DistributedCache鍙湪map/reduce浠誨姟涓綔涓?涓縐嶅熀紜杞歡鍒嗗彂鏈哄埗浣跨敤銆傚畠鍙互琚敤浜庡垎鍙慾ar鍖呭拰鏈湴搴擄紙native libraries錛夈?DistributedCache.addArchiveToClassPath(Path, Configuration)鍜?DistributedCache.addFileToClassPath(Path, Configuration) API鑳藉琚敤浜?緙撳瓨鏂囦歡鍜宩ar鍖咃紝騫舵妸瀹冧滑鍔犲叆瀛恓vm鐨刢lasspath銆備篃鍙互閫氳繃璁劇疆閰嶇疆鏂囨。閲岀殑灞炴?mapred.job.classpath.{files|archives}杈懼埌鐩稿悓鐨勬晥鏋溿傜紦瀛樻枃浠跺彲鐢ㄤ簬鍒嗗彂鍜岃杞芥湰鍦板簱銆?/p>

Tool
Tool 鎺ュ彛鏀寔澶勭悊甯哥敤鐨凥adoop鍛戒護琛岄夐」銆?/p>

Tool 鏄疢ap/Reduce宸ュ叿鎴栧簲鐢ㄧ殑鏍囧噯銆傚簲鐢ㄧ▼搴忓簲鍙鐞嗗叾瀹氬埗鍙傛暟錛?瑕佹妸鏍囧噯鍛戒護琛岄夐」閫氳繃 ToolRunner.run(Tool, String[]) 濮旀墭緇?GenericOptionsParser澶勭悊銆?/p>

Hadoop鍛戒護琛岀殑甯哥敤閫夐」鏈夛細
-conf <configuration file>
-D <property=value>
-fs <local|namenode:port>
-jt <local|jobtracker:port>

IsolationRunner
IsolationRunner 鏄府鍔╄皟璇昅ap/Reduce紼嬪簭鐨勫伐鍏楓?/p>

浣跨敤IsolationRunner鐨勬柟娉曟槸錛岄鍏堣緗?keep.failed.tasks.files灞炴т負true 錛堝悓鏃跺弬鑰僰eep.tasks.files.pattern錛夈?/p>

鐒跺悗錛岀櫥褰曞埌浠誨姟榪愯澶辮觸鐨勮妭鐐逛笂錛岃繘鍏?TaskTracker鐨勬湰鍦拌礬寰勮繍琛?IsolationRunner錛?br>$ cd <local path>/taskTracker/${taskid}/work
$ bin/hadoop org.apache.hadoop.mapred.IsolationRunner ../job.xml

IsolationRunner浼氭妸澶辮觸鐨勪換鍔℃斁鍦ㄥ崟鐙殑涓涓兘澶熻皟璇曠殑jvm涓婅繍琛岋紝騫朵笖閲囩敤鍜屼箣鍓嶅畬鍏ㄤ竴鏍風殑杈撳叆鏁版嵁銆?/p>

Profiling
Profiling鏄竴涓伐鍏鳳紝瀹冧嬌鐢ㄥ唴緗殑java profiler宸ュ叿榪涜鍒嗘瀽鑾峰緱(2-3涓?map鎴杛educe鏍蜂緥榪愯鍒嗘瀽鎶ュ憡銆?/p>

鐢ㄦ埛鍙互閫氳繃璁劇疆灞炴apred.task.profile鎸囧畾緋葷粺鏄惁閲囬泦profiler淇℃伅銆?鍒╃敤api JobConf.setProfileEnabled(boolean)鍙互淇敼灞炴у箋傚鏋滆涓簍rue錛?鍒欏紑鍚痯rofiling鍔熻兘銆俻rofiler淇℃伅淇濆瓨鍦ㄧ敤鎴鋒棩蹇楃洰褰曚笅銆傜己鐪佹儏鍐碉紝profiling鍔熻兘鏄叧闂殑銆?/p>

濡傛灉鐢ㄦ埛璁懼畾浣跨敤profiling鍔熻兘錛屽彲浠ヤ嬌鐢ㄩ厤緗枃妗i噷鐨勫睘鎬?mapred.task.profile.{maps|reduces} 璁劇疆瑕乸rofile map/reduce task鐨勮寖鍥淬傝緗灞炴у肩殑api鏄?JobConf.setProfileTaskRange(boolean,String)銆?鑼冨洿鐨勭己鐪佸兼槸0-2銆?/p>

鐢ㄦ埛鍙互閫氳繃璁懼畾閰嶇疆鏂囨。閲岀殑灞炴apred.task.profile.params 鏉ユ寚瀹歱rofiler閰嶇疆鍙傛暟銆備慨鏀瑰睘鎬ц浣跨敤api JobConf.setProfileParams(String)銆傚綋榪愯task鏃訛紝濡傛灉瀛楃涓插寘鍚?s銆?瀹冧細琚浛鎹㈡垚profileing鐨勮緭鍑烘枃浠跺悕銆傝繖浜涘弬鏁頒細鍦ㄥ懡浠よ閲屼紶閫掑埌瀛怞VM涓傜己鐪佺殑profiling 鍙傛暟鏄?-agentlib:hprof=cpu=samples,heap=sites,force=n,thread=y,verbose=n,file=%s銆?/p>

璋冭瘯
Map/Reduce妗嗘灦鑳藉榪愯鐢ㄦ埛鎻愪緵鐨勭敤浜庤皟璇曠殑鑴氭湰紼嬪簭銆?褰搈ap/reduce浠誨姟澶辮觸鏃訛紝鐢ㄦ埛鍙互閫氳繃榪愯鑴氭湰鍦ㄤ換鍔℃棩蹇楋紙渚嬪浠誨姟鐨勬爣鍑嗚緭鍑恒佹爣鍑嗛敊璇佺郴緇熸棩蹇椾互鍙婁綔涓氶厤緗枃浠訛級涓婂仛鍚庣畫澶勭悊宸ヤ綔銆傜敤鎴鋒彁渚涚殑璋冭瘯鑴氭湰紼嬪簭鐨勬爣鍑嗚緭鍑哄拰鏍囧噯閿欒浼氳緭鍑轟負璇婃柇鏂囦歡銆傚鏋滈渶瑕佺殑璇濊繖浜涜緭鍑虹粨鏋滀篃鍙互鎵撳嵃鍦ㄧ敤鎴風晫闈笂銆?/p>

鍦ㄦ帴涓嬫潵鐨勭珷鑺傦紝鎴戜滑璁ㄨ濡備綍涓庝綔涓氫竴璧鋒彁浜よ皟璇曡剼鏈備負浜嗘彁浜よ皟璇曡剼鏈紝 棣栧厛瑕佹妸榪欎釜鑴氭湰鍒嗗彂鍑哄幓錛岃屼笖榪樿鍦ㄩ厤緗枃浠墮噷璁劇疆銆?/p>

濡備綍鍒嗗彂鑴氭湰鏂囦歡錛?br>鐢ㄦ埛瑕佺敤 DistributedCache 鏈哄埗鏉ュ垎鍙戝拰閾炬帴鑴氭湰鏂囦歡

濡備綍鎻愪氦鑴氭湰錛?br>涓涓揩閫熸彁浜よ皟璇曡剼鏈殑鏂規硶鏄垎鍒負闇瑕佽皟璇曠殑map浠誨姟鍜宺educe浠誨姟璁劇疆 "mapred.map.task.debug.script" 鍜?"mapred.reduce.task.debug.script" 灞炴х殑鍊箋傝繖浜涘睘鎬т篃鍙互閫氳繃 JobConf.setMapDebugScript(String) 鍜?JobConf.setReduceDebugScript(String) API鏉ヨ緗傚浜巗treaming錛?鍙互鍒嗗埆涓洪渶瑕佽皟璇曠殑map浠誨姟鍜宺educe浠誨姟浣跨敤鍛戒護琛岄夐」-mapdebug 鍜?-reducedegug鏉ユ彁浜よ皟璇曡剼鏈?/p>

鑴氭湰鐨勫弬鏁版槸浠誨姟鐨勬爣鍑嗚緭鍑恒佹爣鍑嗛敊璇佺郴緇熸棩蹇椾互鍙婁綔涓氶厤緗枃浠躲傚湪榪愯map/reduce澶辮觸鐨勮妭鐐逛笂榪愯璋冭瘯鍛戒護鏄細
$script $stdout $stderr $syslog $jobconf

Pipes 紼嬪簭鏍規嵁絎簲涓弬鏁拌幏寰梒++紼嬪簭鍚嶃?鍥犳璋冭瘯pipes紼嬪簭鐨勫懡浠ゆ槸
$script $stdout $stderr $syslog $jobconf $program

榛樿琛屼負
瀵逛簬pipes錛岄粯璁ょ殑鑴氭湰浼氱敤gdb澶勭悊core dump錛?鎵撳嵃 stack trace騫朵笖緇欏嚭姝e湪榪愯綰跨▼鐨勪俊鎭?/p>

JobControl
JobControl鏄竴涓伐鍏鳳紝瀹冨皝瑁呬簡涓緇凪ap/Reduce浣滀笟浠ュ強浠栦滑涔嬮棿鐨勪緷璧栧叧緋匯?/p>

鏁版嵁鍘嬬緝
Hadoop Map/Reduce妗嗘灦涓哄簲鐢ㄧ▼搴忕殑鍐欏叆鏂囦歡鎿嶄綔鎻愪緵鍘嬬緝宸ュ叿錛岃繖浜涘伐鍏峰彲浠ヤ負map杈撳嚭鐨勪腑闂存暟鎹拰浣滀笟鏈緇堣緭鍑烘暟鎹紙渚嬪reduce鐨勮緭鍑猴級鎻愪緵鏀寔銆傚畠榪橀檮甯︿簡涓浜?CompressionCodec鐨勫疄鐜幫紝姣斿瀹炵幇浜?zlib鍜宭zo鍘嬬緝綆楁硶銆?Hadoop鍚屾牱鏀寔gzip鏂囦歡鏍煎紡銆?/p>

鑰冭檻鍒版ц兘闂錛坺lib錛変互鍙奐ava綾誨簱鐨勭己澶憋紙lzo錛夌瓑鍥犵礌錛孒adoop涔熶負涓婅堪鍘嬬緝瑙e帇綆楁硶鎻愪緵鏈湴搴撶殑瀹炵幇銆傛洿澶氱殑緇嗚妭璇峰弬鑰?榪欓噷銆?/p>

涓棿杈撳嚭
搴旂敤紼嬪簭鍙互閫氳繃 JobConf.setCompressMapOutput(boolean)api鎺у埗map杈撳嚭鐨勪腑闂寸粨鏋滐紝騫朵笖鍙互閫氳繃 JobConf.setMapOutputCompressorClass(Class)api鎸囧畾 CompressionCodec銆?/p>

浣滀笟杈撳嚭
搴旂敤紼嬪簭鍙互閫氳繃 FileOutputFormat.setCompressOutput(JobConf, boolean) api鎺у埗杈撳嚭鏄惁闇瑕佸帇緙╁茍涓斿彲浠ヤ嬌鐢?FileOutputFormat.setOutputCompressorClass(JobConf, Class)api鎸囧畾CompressionCodec銆?/p>

濡傛灉浣滀笟杈撳嚭瑕佷繚瀛樻垚 SequenceFileOutputFormat鏍煎紡錛岄渶瑕佷嬌鐢?SequenceFileOutputFormat.setOutputCompressionType(JobConf, SequenceFile.CompressionType)api錛屾潵璁懼畾 SequenceFile.CompressionType (i.e.RECORD / BLOCK - 榛樿鏄疪ECORD)銆?/p>

渚嬪瓙錛歐ordCount v2.0
榪欓噷鏄竴涓洿鍏ㄩ潰鐨刉ordCount渚嬪瓙錛屽畠浣跨敤浜嗘垜浠凡緇忚璁鴻繃鐨勫緢澶歁ap/Reduce妗嗘灦鎻愪緵鐨勫姛鑳姐?/p>

榪愯榪欎釜渚嬪瓙闇瑕丠DFS鐨勬煇浜涘姛鑳斤紝鐗瑰埆鏄?DistributedCache鐩稿叧鍔熻兘銆傚洜姝よ繖涓緥瀛愬彧鑳借繍琛屽湪 浼垎甯冨紡 鎴栬?瀹屽叏鍒嗗竷寮忔ā寮忕殑 Hadoop涓娿?/p>

婧愪唬鐮?br>  WordCount.java
1. package org.myorg;
2.  
3. import java.io.*;
4. import java.util.*;
5.  
6. import org.apache.hadoop.fs.Path;
7. import org.apache.hadoop.filecache.DistributedCache;
8. import org.apache.hadoop.conf.*;
9. import org.apache.hadoop.io.*;
10. import org.apache.hadoop.mapred.*;
11. import org.apache.hadoop.util.*;
12.  
13. public class WordCount extends Configured implements Tool {
14.  
15.    public static class Map extends MapReduceBase implements Mapper<LongWritable, Text, Text, IntWritable> {
16.  
17.      static enum Counters { INPUT_WORDS }
18.  
19.      private final static IntWritable one = new IntWritable(1);
20.      private Text word = new Text();
21.  
22.      private boolean caseSensitive = true;
23.      private Set<String> patternsToSkip = new HashSet<String>();
24.  
25.      private long numRecords = 0;
26.      private String inputFile;
27.  
28.      public void configure(JobConf job) {
29.        caseSensitive = job.getBoolean("wordcount.case.sensitive", true);
30.        inputFile = job.get("map.input.file");
31.  
32.        if (job.getBoolean("wordcount.skip.patterns", false)) {
33.          Path[] patternsFiles = new Path[0];
34.          try {
35.            patternsFiles = DistributedCache.getLocalCacheFiles(job);
36.          } catch (IOException ioe) {
37.            System.err.println("Caught exception while getting cached files: " + StringUtils.stringifyException(ioe));
38.          }
39.          for (Path patternsFile : patternsFiles) {
40.            parseSkipFile(patternsFile);
41.          }
42.        }
43.      }
44.  
45.      private void parseSkipFile(Path patternsFile) {
46.        try {
47.          BufferedReader fis = new BufferedReader(new FileReader(patternsFile.toString()));
48.          String pattern = null;
49.          while ((pattern = fis.readLine()) != null) {
50.            patternsToSkip.add(pattern);
51.          }
52.        } catch (IOException ioe) {
53.          System.err.println("Caught exception while parsing the cached file '" + patternsFile + "' : " + StringUtils.stringifyException(ioe));
54.        }
55.      }
56.  
57.      public void map(LongWritable key, Text value, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException {
58.        String line = (caseSensitive) ? value.toString() : value.toString().toLowerCase();
59.  
60.        for (String pattern : patternsToSkip) {
61.          line = line.replaceAll(pattern, "");
62.        }
63.  
64.        StringTokenizer tokenizer = new StringTokenizer(line);
65.        while (tokenizer.hasMoreTokens()) {
66.          word.set(tokenizer.nextToken());
67.          output.collect(word, one);
68.          reporter.incrCounter(Counters.INPUT_WORDS, 1);
69.        }
70.  
71.        if ((++numRecords % 100) == 0) {
72.          reporter.setStatus("Finished processing " + numRecords + " records " + "from the input file: " + inputFile);
73.        }
74.      }
75.    }
76.  
77.    public static class Reduce extends MapReduceBase implements Reducer<Text, IntWritable, Text, IntWritable> {
78.      public void reduce(Text key, Iterator<IntWritable> values, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException {
79.        int sum = 0;
80.        while (values.hasNext()) {
81.          sum += values.next().get();
82.        }
83.        output.collect(key, new IntWritable(sum));
84.      }
85.    }
86.  
87.    public int run(String[] args) throws Exception {
88.      JobConf conf = new JobConf(getConf(), WordCount.class);
89.      conf.setJobName("wordcount");
90.  
91.      conf.setOutputKeyClass(Text.class);
92.      conf.setOutputValueClass(IntWritable.class);
93.  
94.      conf.setMapperClass(Map.class);
95.      conf.setCombinerClass(Reduce.class);
96.      conf.setReducerClass(Reduce.class);
97.  
98.      conf.setInputFormat(TextInputFormat.class);
99.      conf.setOutputFormat(TextOutputFormat.class);
100.  
101.      List<String> other_args = new ArrayList<String>();
102.      for (int i=0; i < args.length; ++i) {
103.        if ("-skip".equals(args[i])) {
104.          DistributedCache.addCacheFile(new Path(args[++i]).toUri(), conf);
105.          conf.setBoolean("wordcount.skip.patterns", true);
106.        } else {
107.          other_args.add(args[i]);
108.        }
109.      }
110.  
111.      FileInputFormat.setInputPaths(conf, new Path(other_args.get(0)));
112.      FileOutputFormat.setOutputPath(conf, new Path(other_args.get(1)));
113.  
114.      JobClient.runJob(conf);
115.      return 0;
116.    }
117.  
118.    public static void main(String[] args) throws Exception {
119.      int res = ToolRunner.run(new Configuration(), new WordCount(), args);
120.      System.exit(res);
121.    }
122. }
123.  

榪愯鏍蜂緥
杈撳叆鏍蜂緥錛?/p>

$ bin/hadoop dfs -ls /usr/joe/wordcount/input/
/usr/joe/wordcount/input/file01
/usr/joe/wordcount/input/file02

$ bin/hadoop dfs -cat /usr/joe/wordcount/input/file01
Hello World, Bye World!

$ bin/hadoop dfs -cat /usr/joe/wordcount/input/file02
Hello Hadoop, Goodbye to hadoop.

榪愯紼嬪簭錛?/p>

$ bin/hadoop jar /usr/joe/wordcount.jar org.myorg.WordCount /usr/joe/wordcount/input /usr/joe/wordcount/output

杈撳嚭錛?/p>

$ bin/hadoop dfs -cat /usr/joe/wordcount/output/part-00000
Bye 1
Goodbye 1
Hadoop, 1
Hello 2
World! 1
World, 1
hadoop. 1
to 1

娉ㄦ剰姝ゆ椂鐨勮緭鍏ヤ笌絎竴涓増鏈殑涓嶅悓錛岃緭鍑虹殑緇撴灉涔熸湁涓嶅悓銆?/p>

鐜板湪閫氳繃DistributedCache鎻掑叆涓涓ā寮忔枃浠訛紝鏂囦歡涓繚瀛樹簡瑕佽蹇界暐鐨勫崟璇嶆ā寮忋?/p>

$ hadoop dfs -cat /user/joe/wordcount/patterns.txt
\.
\,
\!
to

鍐嶈繍琛屼竴嬈★紝榪欐浣跨敤鏇村鐨勯夐」錛?/p>

$ bin/hadoop jar /usr/joe/wordcount.jar org.myorg.WordCount -Dwordcount.case.sensitive=true /usr/joe/wordcount/input /usr/joe/wordcount/output -skip /user/joe/wordcount/patterns.txt

搴旇寰楀埌榪欐牱鐨勮緭鍑猴細

$ bin/hadoop dfs -cat /usr/joe/wordcount/output/part-00000
Bye 1
Goodbye 1
Hadoop 1
Hello 2
World 2
hadoop 1

鍐嶈繍琛屼竴嬈★紝榪欎竴嬈″叧闂ぇ灝忓啓鏁忔劅鎬э紙case-sensitivity錛夛細

$ bin/hadoop jar /usr/joe/wordcount.jar org.myorg.WordCount -Dwordcount.case.sensitive=false /usr/joe/wordcount/input /usr/joe/wordcount/output -skip /user/joe/wordcount/patterns.txt

杈撳嚭錛?/p>

$ bin/hadoop dfs -cat /usr/joe/wordcount/output/part-00000
bye 1
goodbye 1
hadoop 2
hello 2
world 2

紼嬪簭瑕佺偣
閫氳繃浣跨敤涓浜汳ap/Reduce妗嗘灦鎻愪緵鐨勫姛鑳斤紝WordCount鐨勭浜屼釜鐗堟湰鍦ㄥ師濮嬬増鏈熀紜涓婃湁浜嗗涓嬬殑鏀硅繘錛?/p>

灞曠ず浜嗗簲鐢ㄧ▼搴忓浣曞湪Mapper (鍜孯educer)涓氳繃configure鏂規硶 淇敼閰嶇疆鍙傛暟(28-43琛?銆?
灞曠ず浜嗕綔涓氬浣曚嬌鐢―istributedCache 鏉ュ垎鍙戝彧璇繪暟鎹?榪欓噷鍏佽鐢ㄦ埛鎸囧畾鍗曡瘝鐨勬ā寮忥紝鍦ㄨ鏁版椂蹇界暐閭d簺絎﹀悎妯″紡鐨勫崟璇?104琛?銆?
灞曠ずTool鎺ュ彛鍜孏enericOptionsParser澶勭悊Hadoop鍛戒護琛岄夐」鐨勫姛鑳?(87-116, 119琛?銆?
灞曠ず浜嗗簲鐢ㄧ▼搴忓浣曚嬌鐢–ounters(68琛?錛屽浣曢氳繃浼犻掔粰map錛堝拰reduce錛?鏂規硶鐨凴eporter瀹炰緥鏉ヨ緗簲鐢ㄧ▼搴忕殑鐘舵佷俊鎭?72琛?銆?
Java鍜孞NI鏄疭un Microsystems, Inc.鍦ㄧ編鍥藉拰鍏跺畠鍥藉鐨勬敞鍐屽晢鏍囥?/p>

 

鏈枃鏉ヨ嚜CSDN鍗氬錛岃漿杞借鏍囨槑鍑哄錛?a >http://blog.csdn.net/superxgl/archive/2010/01/11/5171929.aspx



koson 2010-07-19 14:40 鍙戣〃璇勮
]]>
鍩轟簬hive鐨勬棩蹇楁暟鎹粺璁″疄鎴?/title><link>http://www.shnenglu.com/koson/archive/2010/07/19/120773.html</link><dc:creator>koson</dc:creator><author>koson</author><pubDate>Mon, 19 Jul 2010 06:39:00 GMT</pubDate><guid>http://www.shnenglu.com/koson/archive/2010/07/19/120773.html</guid><wfw:comment>http://www.shnenglu.com/koson/comments/120773.html</wfw:comment><comments>http://www.shnenglu.com/koson/archive/2010/07/19/120773.html#Feedback</comments><slash:comments>4</slash:comments><wfw:commentRss>http://www.shnenglu.com/koson/comments/commentRss/120773.html</wfw:commentRss><trackback:ping>http://www.shnenglu.com/koson/services/trackbacks/120773.html</trackback:ping><description><![CDATA[<span><span><strong><span><span><span><span> <p><strong><span><span>涓銆?span>           </span></span></span></strong><strong><span>hive</span> </strong><strong><span>綆浠?/span></strong><strong></strong></p> <p><span><strong><span>        hive</span> </strong><strong><span>鏄竴涓熀浜?/span> </strong><strong><span>hadoop</span> </strong><strong><span>鐨勫紑婧愭暟鎹粨搴撳伐鍏鳳紝鐢ㄤ簬瀛樺偍鍜屽鐞嗘搗閲忕粨鏋勫寲鏁版嵁銆?/span> </strong><strong><span><span>   </span></span></strong><strong><span>瀹冩妸嫻烽噺鏁版嵁瀛樺偍浜?/span> </strong><strong><span>hadoop</span> </strong><strong><span>鏂囦歡緋葷粺錛岃屼笉鏄暟鎹簱錛屼絾鎻愪緵浜嗕竴濂楃被鏁版嵁搴撶殑鏁版嵁瀛樺偍鍜屽鐞嗘満鍒訛紝騫墮噰鐢?/span> </strong><strong><span>HQL</span> </strong><strong><span>錛堢被</span> </strong><strong><span>SQL</span> </strong><strong><span>錛夎璦瀵硅繖浜涙暟鎹繘琛岃嚜鍔ㄥ寲綆$悊鍜屽鐞嗐傛垜浠彲浠ユ妸</span> </strong><strong><span>hive</span> </strong><strong><span>涓搗閲忕粨鏋勫寲鏁版嵁鐪嬫垚涓涓釜鐨勮〃錛岃屽疄闄呬笂榪欎簺鏁版嵁鏄垎甯冨紡瀛樺偍鍦?/span> </strong><strong><span>HDFS</span> </strong><strong><span>涓殑銆?/span> </strong><strong><span>Hive</span> </strong><strong><span>緇忚繃瀵硅鍙ヨ繘琛岃В鏋愬拰杞崲錛屾渶緇堢敓鎴愪竴緋誨垪鍩轟簬</span> </strong><strong><span>hadoop</span> </strong><strong><span>鐨?/span> </strong><strong><span>map/reduce</span> </strong><span>浠誨姟錛岄氳繃鎵ц榪欎簺浠誨姟瀹屾垚鏁版嵁澶勭悊<strong><span>銆?/span></strong></span><strong></strong></span></p> <p><span><strong><span><span>        </span>Hive</span> </strong><strong><span>璇炵敓浜?/span> </strong><strong><span>facebook</span> </strong><strong><span>鐨勬棩蹇楀垎鏋愰渶姹傦紝闈㈠嫻烽噺鐨勭粨鏋勫寲鏁版嵁錛?/span> </strong><strong><span>hive</span> </strong><strong><span>浠ヨ緝浣庣殑鎴愭湰瀹屾垚浜嗕互寰闇瑕佸ぇ瑙勬ā鏁版嵁搴撴墠鑳藉畬鎴愮殑浠誨姟錛屽茍涓斿涔犻棬妲涚浉瀵硅緝浣庯紝搴旂敤寮鍙戠伒媧昏岄珮鏁堛?/span></strong><strong></strong></span></p> <p><span><strong><span><span>        </span>Hive</span> </strong><strong><span>鑷?/span> </strong><strong><span>2009.4.29</span> </strong><strong><span>鍙戝竷絎竴涓畼鏂圭ǔ瀹氱増</span> </strong><strong><span>0.3.0</span> </strong><strong><span>鑷充粖錛屼笉榪囦竴騫寸殑鏃墮棿錛屾鍦ㄦ參鎱㈠畬鍠勶紝緗戜笂鑳芥壘鍒扮殑鐩稿叧璧勬枡鐩稿綋灝戯紝灝ゅ叾涓枃璧勬枡鏇村皯錛屾湰鏂囩粨鍚堜笟鍔″</span> </strong><strong><span>hive</span> </strong><strong><span>鐨勫簲鐢ㄥ仛浜嗕竴浜涙帰绱紝騫舵妸榪欎簺緇忛獙鍋氫竴涓葷粨錛屾墍璋撳墠杞︿箣閴達紝甯屾湜璇昏呰兘灝戣蛋涓浜涘集璺?/span></strong><strong></strong></span></p> <p><span><strong><span><span>        </span>Hive</span> </strong><strong><span>鐨勫畼鏂?/span> </strong><strong><span>wiki</span> </strong><strong><span>璇峰弬鑰冭繖閲?/span> </strong><strong><span>:</span></strong><strong></strong></span></p> <p><strong><span><span><span>        </span></span><a ><span><span>http://wiki.apache.org/hadoop/Hive</span></span></a></span></strong></p> <p><span><strong><span><span>        </span>瀹樻柟涓婚〉鍦ㄨ繖閲岋細</span></strong><strong></strong></span></p> <p><strong><span><span><span>        </span></span><a ><span><span>http://hadoop.apache.org/hive/</span></span></a></span></strong></p> <p><span><strong><span><span>        </span>hive-0.5.0</span> </strong><strong><span>婧愮爜鍖呭拰浜岃繘鍒跺彂甯冨寘鐨勪笅杞藉湴鍧</span></strong><strong></strong></span></p> <p><strong><span><span><span>        </span></span><a ><span><span>http://labs.renren.com/apache-mirror/hadoop/hive/hive-0.5.0/</span></span></a></span></strong></p> <p><strong><span><span>浜屻?span>           </span></span></span></strong><strong><span>閮ㄧ講</span></strong><strong></strong></p> <p><span><strong><span><span>        </span>鐢變簬</span> </strong><strong><span><span>Hive</span> </span></strong><strong><span>鏄熀浜?/span> </strong><strong><span>hadoop</span> </strong><strong><span>鐨勫伐鍏鳳紝鎵浠?/span> </strong><strong><span>hive</span> </strong><strong><span>鐨勯儴緗查渶瑕佷竴涓甯歌繍琛岀殑</span> </strong><strong><span>hadoop</span> </strong><strong><span>鐜銆備互涓嬩粙緇?/span> </strong><strong><span>hive</span> </strong><strong><span>鐨勭畝鍗曢儴緗插拰搴旂敤銆?/span></strong><strong></strong></span></p> <p><span><strong><span><span>        </span>閮ㄧ講鐜錛?/span></strong><strong></strong></span></p> <p><span><strong><span><span>        </span>鎿嶄綔緋葷粺錛?/span> </strong><strong><span>Red Hat Enterprise Linux AS release 4 (Nahant Update 7)</span></strong></span></p> <p><span><strong><span><span>        </span>Hadoop</span> </strong><strong><span>錛?/span> </strong><strong><span>hadoop-0.20.2</span> </strong><strong><span>錛屾甯歌繍琛?/span></strong><strong></strong></span></p> <p><span><strong><span><span>        </span>閮ㄧ講姝ラ濡備笅錛?/span></strong><strong></strong></span></p> <p><strong><span><span><span>1銆?/span> <span>  </span></span></span></strong><span><strong><span>涓嬭澆鏈鏂扮増鏈彂甯冨寘</span> </strong><strong><span><a ><span>hive-0.5.0-dev.tar.gz</span> </a></span></strong><strong><span>錛屼紶鍒?/span> </strong><strong><span>hadoop</span> </strong><strong><span>鐨?/span> </strong><strong><span>namenode</span> </strong><strong><span>鑺傜偣涓婏紝瑙e帇寰楀埌</span> </strong><strong><span>hive</span> </strong><strong><span>鐩綍銆傚亣璁捐礬寰勪負錛?/span> </strong><strong><span>/opt/hadoop/hive-0.5.0-bin</span></strong></span></p> <p><strong><span><span><span>2銆?/span> <span>  </span></span></span></strong><span><strong><span>璁劇疆鐜鍙橀噺</span> </strong><strong><span>HIVE_HOME</span> </strong><strong><span>錛屾寚鍚?/span> </strong><strong><span>hive</span> </strong><strong><span>鏍圭洰褰?/span> </strong><strong><span>/opt/hadoop/hive-0.5.0-bin</span> </strong><strong><span>銆傜敱浜?/span> </strong><strong><span>hadoop</span> </strong><strong><span>宸茶繍琛岋紝媯鏌ョ幆澧冨彉閲?/span> </strong><strong><span>JAVA_HOME</span> </strong><strong><span>鍜?/span> </strong><strong><span>HADOOP_HOME</span> </strong><strong><span>鏄惁姝g‘鏈夋晥銆?/span></strong><strong></strong></span></p> <p><strong><span><span><span>3銆?/span> <span>  </span></span></span></strong><span><strong><span>鍒囨崲鍒?/span> </strong><strong><span>$HIVE_HOME</span> </strong><strong><span>鐩綍錛?/span> </strong><strong><span>hive</span> </strong><strong><span>閰嶇疆榛樿鍗沖彲錛岃繍琛?/span> </strong><strong><span>bin/hive</span> </strong><strong><span>鍗沖彲鍚姩</span> </strong><strong><span>hive</span> </strong><strong><span>錛屽鏋滄甯稿惎鍔紝灝嗕細鍑虹幇“</span> </strong><strong><span>hive></span> </strong><strong><span>”鎻愮ず絎︺?/span></strong><strong></strong></span></p> <p><strong><span><span><span>4銆?/span> <span>  </span></span></span></strong><span><strong><span>鍦ㄥ懡浠ゆ彁紺虹涓緭鍏?#8220;</span> </strong><strong><span>show tables;</span> </strong><strong><span>”錛屽鏋滄甯歌繍琛岋紝璇存槑宸查儴緗叉垚鍔燂紝鍙緵浣跨敤銆?/span></strong><strong></strong></span></p> <p><span><strong><span>甯歌闂錛?/span></strong><strong></strong></span></p> <p><strong><span><span><span>1銆?/span> <span>       </span></span></span></strong><span><strong><span>鎵ц“</span> </strong><strong><span>show tables;</span> </strong><strong><span>”鍛戒護鎻愮ず“</span> </strong><strong><span>FAILED: Error in metadata: java.lang.IllegalArgumentException: URI:<span>  </span>does not have a scheme</span> </strong><strong><span>”錛岃繖鏄敱浜?/span> </strong><strong><span>hive</span> </strong><strong><span>鎵句笉鍒板瓨鏀懼厓鏁版嵁搴撶殑鏁版嵁搴撹屽鑷寸殑錛屼慨鏀?/span> </strong><strong><span>conf/</span> </strong><span><span><strong><span>hive-default.xml</span> </strong></span></span><strong><span>閰嶇疆鏂囦歡涓殑</span> </strong><strong><span>hive.metastore.local</span> </strong><strong><span>涓?/span> </strong><strong><span>true</span> </strong><strong><span>鍗沖彲銆傜敱浜?/span> </strong><strong><span>hive</span> </strong><strong><span>鎶婄粨鏋勫寲鏁版嵁鐨勫厓鏁版嵁淇℃伅鏀懼湪絎笁鏂規暟鎹簱錛屾澶勮緗負</span> </strong><strong><span>true</span> </strong><strong><span>錛?/span> </strong><strong><span>hive</span> </strong><strong><span>灝嗗湪鏈湴鍒涘緩</span> </strong><strong><span>derby</span> </strong><strong><span>鏁版嵁搴撶敤浜庡瓨鏀懼厓鏁版嵁銆傚綋鐒跺鏋滄湁闇瑕佷篃鍙互閲囩敤</span> </strong><strong><span>mysql</span> </strong><strong><span>絳夌涓夋柟鏁版嵁搴撳瓨鏀懼厓鏁版嵁錛屼笉榪囪繖鏃?/span> </strong><strong><span>hive.metastore.local</span> </strong><strong><span>鐨勯厤緗煎簲涓?/span> </strong><strong><span>false</span> </strong><strong><span>銆?/span></strong><strong></strong></span></p> <p><strong><span><span><span>2銆?/span> <span>       </span></span></span></strong><span><strong><span>濡傛灉浣犲凡鏈変竴濂?/span> </strong><strong><span>nutch1.0</span> </strong><strong><span>緋葷粺姝e湪璺戯紝鑰屼綘涓嶆兂鍗曠嫭鍐嶅幓閮ㄧ講涓濂?/span> </strong><strong><span>hadoop</span> </strong><strong><span>鐜錛屼綘鍙互鐩存帴浣跨敤</span> </strong><strong><span>nutch1.0</span> </strong><strong><span>鑷甫鐨?/span> </strong><strong><span>hadoop</span> </strong><strong><span>鐜錛屼絾榪欐牱鐨勯儴緗蹭細瀵艱嚧</span> </strong><strong><span>hive</span> </strong><strong><span>涓嶈兘姝e父榪愯錛屾彁紺烘壘涓嶅埌鏌愪簺鏂規硶銆傝繖鏄敱浜?/span> </strong><strong><span>nutch1.0</span> </strong><strong><span>浣跨敤浜?/span> </strong><strong><span>commons-lang-2.1.jar</span> </strong><strong><span>榪欎釜鍖咃紝鑰?/span> </strong><strong><span>hive</span> </strong><strong><span>闇瑕佺殑鏄?/span> </strong><strong><span>commons-lang-2.4.jar</span> </strong><strong><span>錛屼笅杞戒竴涓?/span> </strong><strong><span>2.4</span> </strong><strong><span>鐗堟湰鐨勫寘鏇挎崲鎺?/span> </strong><strong><span>2.1</span> </strong><strong><span>鍗沖彲錛?/span> </strong><strong><span>nutch</span> </strong><strong><span>鍜?/span> </strong><strong><span>hive</span> </strong><strong><span>閮借兘姝e父榪愯銆?/span></strong><strong></strong></span></p> <p><strong><span><span>涓夈?span>           </span></span></span></strong><strong><span>搴旂敤鍦烘櫙</span></strong><strong></strong></p> <p><span><strong><span><span>        </span>鏈枃涓昏璁茶堪浣跨敤</span> </strong><strong><span>hive</span> </strong><strong><span>鐨勫疄璺碉紝涓氬姟涓嶆槸鍏抽敭錛岀畝瑕佷粙緇嶄笟鍔″満鏅紝鏈鐨勪換鍔℃槸瀵規悳绱㈡棩蹇楁暟鎹繘琛岀粺璁″垎鏋愩?/span></strong><strong></strong></span></p> <p><span><strong><span><span>        </span>闆嗗洟鎼滅儲鍒氫笂綰夸笉涔咃紝鏃ュ織閲忓茍涓嶅ぇ</span> </strong><strong><span>銆傝繖浜涙棩蹇楀垎甯冨湪</span> </strong><strong><span>5</span> </strong><strong><span>鍙板墠绔満錛屾寜灝忔椂淇濆瓨錛屽茍浠ュ皬鏃朵負鍛ㄦ湡瀹氭椂灝嗕笂涓灝忔椂浜х敓鐨勬暟鎹悓姝ュ埌鏃ュ織鍒嗘瀽鏈猴紝緇熻鏁版嵁瑕佹眰鎸夊皬鏃舵洿鏂般傝繖浜涚粺璁¢」錛屽寘鎷叧閿瘝鎼滅儲閲?/span> </strong><strong><span>pv</span> </strong><strong><span>錛岀被鍒闂噺錛屾瘡縐掕闂噺</span> </strong><strong><span>tps</span> </strong><strong><span>絳夌瓑銆?/span></strong><strong></strong></span></p> <p><span><strong><span>鍩轟簬</span> </strong><strong><span>hive</span> </strong><strong><span>錛屾垜浠皢榪欎簺鏁版嵁鎸夊ぉ涓哄崟浣嶅緩琛紝姣忓ぉ涓涓〃錛屽悗鍙拌剼鏈牴鎹椂闂存埑灝嗘瘡灝忔椂鍚屾榪囨潵鐨?/span> </strong><strong><span>5</span> </strong><strong><span>鍙板墠绔満鐨勬棩蹇楁暟鎹悎騫舵垚涓涓棩蹇楁枃浠訛紝瀵煎叆</span> </strong><strong><span>hive</span> </strong><strong><span>緋葷粺錛屾瘡灝忔椂鍚屾鐨勬棩蹇楁暟鎹榪藉姞鍒板綋澶╂暟鎹〃涓紝瀵煎叆瀹屾垚鍚庯紝褰撳ぉ鍚勯」緇熻欏瑰皢琚噸鏂拌綆楀茍杈撳嚭緇熻緇撴灉銆?/span></strong><strong></strong></span></p> <p><span><strong><span><span>        </span>浠ヤ笂闇姹傝嫢鐩存帴鍩轟簬</span> </strong><strong><span>hadoop</span> </strong><strong><span>寮鍙戯紝闇瑕佽嚜琛岀鐞嗘暟鎹紝閽堝澶氫釜緇熻闇姹傚紑鍙戜笉鍚岀殑</span> </strong><strong><span>map/reduce</span> </strong><strong><span>榪愮畻浠誨姟錛屽鍚堝茍銆佹帓搴忕瓑澶氶」鎿嶄綔榪涜瀹氬埗錛屽茍媯嫻嬩換鍔¤繍琛岀姸鎬侊紝宸ヤ綔閲忓茍涓嶅皬銆備絾浣跨敤</span> </strong><strong><span>hive</span> </strong><strong><span>錛屼粠瀵煎叆鍒板垎鏋愩佹帓搴忋佸幓閲嶃佺粨鏋滆緭鍑猴紝榪欎簺鎿嶄綔閮藉彲浠ヨ繍鐢?/span> </strong><strong><span>hql</span> </strong><strong><span>璇彞鏉ヨВ鍐籌紝涓鏉¤鍙ョ粡榪囧鐞嗚瑙f瀽鎴愬嚑涓換鍔℃潵榪愯錛屽嵆浣挎槸鍏抽敭璇嶈闂噺澧為噺榪欑闇瑕佸悓鏃惰闂澶╂暟鎹殑杈冧負澶嶆潅鐨勯渶姹備篃鑳介氳繃琛ㄥ叧鑱旇繖鏍風殑璇彞鑷姩瀹屾垚錛岃妭鐪佷簡澶ч噺宸ヤ綔閲忋?/span></strong><strong></strong></span></p> <p><strong><span><span>鍥涖?span>           </span></span></span></strong><strong><span>Hive</span> </strong><strong><span>瀹炴垬</span></strong><strong></strong></p> <p><span><strong><span><span>        </span>鍒濇浣跨敤</span> </strong><strong><span>hive</span> </strong><strong><span>錛屽簲璇ヨ涓婃墜榪樻槸鎸哄揩鐨勩?/span> </strong><strong><span>Hive</span> </strong><strong><span>鎻愪緵鐨勭被</span> </strong><strong><span>SQL</span> </strong><strong><span>璇彞涓?/span> </strong><strong><span>mysql</span> </strong><strong><span>璇彞鏋佷負鐩鎬技錛岃娉曚笂鏈夊ぇ閲忕浉鍚岀殑鍦版柟錛岃繖緇欐垜浠笂鎵嬪甫鏉ヤ簡寰堝ぇ鐨勬柟渚匡紝浣嗘槸瑕佸緱蹇冨簲鎵嬪湴鍐欏ソ榪欎簺璇彞錛岃繕闇瑕佸</span> </strong><strong><span>hive</span> </strong><strong><span>鏈夎緝濂界殑浜嗚В錛屾墠鑳界粨鍚?/span> </strong><strong><span>hive</span> </strong><strong><span>鐗硅壊鍐欏嚭綺懼鐨勮鍙ャ?/span></strong><strong></strong></span></p> <p><span><strong><span><span>        </span>鍏充簬</span> </strong><strong><span>hive</span> </strong><strong><span>璇█鐨勮緇嗚娉曞彲鍙傝冨畼鏂?/span> </strong><strong><span>wiki</span> </strong><strong><span>鐨勮璦鎵嬪唽</span> </strong><strong><span>:</span></strong></span></p> <p><strong><span><span><span>        </span></span><a ><span><span>http://wiki.apache.org/hadoop/Hive/LanguageManual</span></span></a></span></strong></p> <p><span><strong><span><span>        </span>铏界劧璇硶椋庢牸涓烘垜浠彁渚涗簡渚垮埄錛屼絾鍒濇浣跨敤閬囧埌鐨勯棶棰樿繕鏄笉灝戠殑錛屼笅闈㈤拡瀵逛笟鍔″満鏅皥璋堟垜浠亣鍒扮殑闂錛屽拰瀵?/span> </strong><strong><span>hive</span> </strong><strong><span>鍔熻兘鐨勫畾鍒躲?/span></strong><strong></strong></span></p> <p><strong><span><span><span>1銆?/span> </span></span></strong><span><strong><span>鍒嗛殧絎﹂棶棰?/span></strong><strong></strong></span></p> <p><span><strong><span><span>       <span><span>        </span> </span></span>棣栧厛閬囧埌鐨勬槸鏃ュ織鏁版嵁鐨勫垎闅旂闂錛屾垜浠殑鏃ュ織鏁版嵁鐨勫ぇ鑷存牸寮忓涓嬶細</span></strong><strong></strong></span></p> <p><strong><span>2010-05-24 00:00:02@$_$@QQ2010@$_$@all@$_$@NOKIA_1681C@$_$@1@$_$@10@$_$@@$_$@-1@$_$@10@$_$@application@$_$@1</span></strong></p> <p><strong></strong><span><strong><span><span><span>        </span></span>浠庢牸寮忓彲瑙佸叾鍒嗛殧絎︽槸“</span> </strong><strong><span>@$_$@</span> </strong><strong><span>”錛岃繖鏄負浜嗗敖鍙兘闃叉鏃ュ織姝f枃鍑虹幇涓庡垎闅旂鐩稿悓鐨勫瓧絎﹁屽鑷存暟鎹販娣嗐傛湰鏉?/span> </strong><strong><span>hive</span></strong><strong><span>鏀寔鍦ㄥ緩琛ㄧ殑鏃跺欐寚瀹氳嚜瀹氫箟鍒嗛殧絎︾殑錛屼絾緇忚繃澶氭嫻嬭瘯鍙戠幇鍙敮鎸佸崟涓瓧絎︾殑鑷畾涔夊垎闅旂錛屽儚“</span> </strong><strong><span>@$_$@</span> </strong><strong><span>”榪欐牱鐨勫垎闅旂鏄笉鑳借鏀寔鐨勶紝浣嗘槸鎴戜滑鍙互閫氳繃瀵瑰垎闅旂鐨勫畾鍒惰В鍐寵繖涓棶棰橈紝</span> </strong><strong><span>hive</span> </strong><strong><span>鐨勫唴閮ㄥ垎闅旂鏄?#8220;</span> </strong><strong><span>\001</span> </strong><strong><span>”錛屽彧瑕佹妸鍒嗛殧絎︽浛鎹㈡垚“</span></strong><strong><span>\001</span> </strong><strong><span>”鍗沖彲銆?/span></strong><strong></strong></span></p> <p><span><strong><span>緇忚繃鎺㈢儲鎴戜滑鍙戠幇鏈変袱鏉¢斿緞瑙e喅榪欎釜闂銆?/span></strong><strong></strong></span></p> <p><strong><span><span><span>a)</span> <span>         </span></span></span></strong><span><strong><span>鑷畾涔?/span> </strong><strong><span>outputformat</span> </strong><strong><span>鍜?/span> </strong><strong><span>inputformat</span> </strong><strong><span>銆?/span></strong><strong></strong></span></p> <p><span><strong><span><span><span>        </span></span>Hive</span> </strong><strong><span>鐨?/span> </strong><strong><span>outputformat/inputformat</span> </strong><strong><span>涓?/span> </strong><strong><span>hadoop</span> </strong><strong><span>鐨?/span> </strong><strong><span>outputformat/inputformat</span> </strong><strong><span>鐩稿綋綾諱技錛?/span> </strong><strong><span>inputformat</span> </strong><strong><span>璐熻矗鎶婅緭鍏ユ暟鎹繘琛屾牸寮忓寲錛岀劧鍚庢彁渚涚粰</span> </strong><strong><span>hive</span> </strong><strong><span>錛?/span> </strong><strong><span>outputformat</span> </strong><strong><span>璐熻矗鎶?/span> </strong><strong><span>hive</span> </strong><strong><span>杈撳嚭鐨勬暟鎹噸鏂版牸寮忓寲鎴愮洰鏍囨牸寮忓啀杈撳嚭鍒版枃浠訛紝榪欑瀵規牸寮忚繘琛屽畾鍒剁殑鏂瑰紡杈冧負搴曞眰錛屽鍏惰繘琛屽畾鍒朵篃鐩稿綆鍗曪紝閲嶅啓</span> </strong><strong><span>InputFormat</span> </strong><strong><span>涓?/span> </strong><strong><span>RecordReader</span> </strong><strong><span>綾諱腑鐨?/span> </strong><strong><span>next</span> </strong><strong><span>鏂規硶鍗沖彲錛岀ず渚嬩唬鐮佸涓嬶細</span></strong><strong></strong></span></p> <p align=left><span><span>    </span></span><strong><span>public</span> </strong><strong><span>boolean</span> </strong><span>next(LongWritable key, BytesWritable value)</span></p> <p align=left><span><span>        </span></span><strong><span>throws</span> </strong><span>IOException {</span></p> <p align=left><span><span>    </span><span>    </span></span><strong><span>while</span> </strong><span>(</span> <span><span>reader</span> </span><span><span>.next(key, </span></span><span><span>text</span> </span><span><span>)</span> </span><span>) {</span></p> <p align=left><span><span>   </span><span>     </span>String <span>strReplace</span> = </span><span>text</span> <span>.toString().toLowerCase().replace(</span> <span>"@$_$@"</span> <span>, </span><span>"\001"</span> <span>);</span></p> <p align=left><span><span>    </span><span>    </span>Text txtReplace = </span><strong><span>new</span> </strong><span>Text();</span></p> <p align=left><span><span>    </span><span>    </span>txtReplace.set(<span>strReplace</span> );</span></p> <p align=left><span><span>        </span>value.set(txtReplace.getBytes(), 0, txtReplace.getLength());</span></p> <p align=left><span><span>        </span></span><strong><span>return</span> </strong><strong><span>true</span> </strong><span>;</span></p> <p align=left><span><span>      </span>}</span></p> <p align=left><span><span>      </span><span>   </span></span><strong><span>return</span> </strong><strong><span>false</span> </strong><span>;</span></p> <p><span>}</span><strong></strong></p> <p><span><strong><span><span><span>        </span></span>閲嶅啓</span> </strong><strong><span>HiveIgnoreKeyTextOutputFormat</span> </strong><strong><span>涓?/span> </strong><strong><span>RecordWriter</span> </strong><strong><span>涓殑</span> </strong><strong><span>write</span> </strong><strong><span>鏂規硶錛岀ず渚嬩唬鐮佸涓嬶細</span></strong><strong></strong></span></p> <p align=left><span><span>    </span></span><strong><span>public</span> </strong><strong><span>void</span> </strong><span><span>write</span> (Writable w) </span><strong><span>throws</span> </strong><span>IOException {</span></p> <p align=left><span><span>      </span>String strReplace = ((Text)w).toString().replace(</span> <span>"\001"</span> <span>, </span><span>"@$_$@"</span> <span>);</span></p> <p align=left><span><span>      </span>Text txtReplace = </span><strong><span>new</span> </strong><span>Text();</span></p> <p align=left><span><span>      </span>txtReplace.set(strReplace);</span></p> <p align=left><span><span>      </span></span><strong><span>byte</span> </strong><span>[] output = txtReplace.getBytes();</span></p> <p align=left><span><span>      </span></span><span>bytesWritable</span> <span>.set(output, 0, output.</span> <span>length</span> <span>);</span></p> <p align=left><span><span>      </span><span>writer</span> .write(</span> <span>bytesWritable</span> <span>);</span></p> <p><span>}</span></p> <p><span><strong><span><span><span>        </span></span>鑷畾涔?/span> </strong><strong><span>outputformat/inputformat</span> </strong><strong><span>鍚庯紝鍦ㄥ緩琛ㄦ椂闇瑕佹寚瀹?/span> </strong><strong><span>outputformat/inputformat</span> </strong><strong><span>錛屽涓嬬ず渚嬶細</span></strong><strong></strong></span></p> <p align=left><span>stored as INPUTFORMAT 'com.aspire.search.loganalysis.hive.SearchLogInputFormat' OUTPUTFORMAT 'com.aspire.search.loganalysis.hive.SearchLogOutputFormat'</span><strong></strong></p> <p><strong><span><span><span>b)</span> <span>         </span></span></span></strong><span><strong><span>閫氳繃</span> </strong><strong><span>SerDe(serialize/deserialize)</span> </strong><strong><span>錛屽湪鏁版嵁搴忓垪鍖栧拰鍙嶅簭鍒楀寲鏃舵牸寮忓寲鏁版嵁銆?/span></strong><strong></strong></span></p> <p><span><strong><span>榪欑鏂瑰紡紼嶅井澶嶆潅涓鐐癸紝瀵規暟鎹殑鎺у埗鑳藉姏涔熻寮變竴浜涳紝瀹冧嬌鐢ㄦ鍒欒〃杈懼紡鏉ュ尮閰嶅拰澶勭悊鏁版嵁錛屾ц兘涔熶細鏈夋墍褰卞搷銆備絾瀹冪殑浼樼偣鏄彲浠ヨ嚜瀹氫箟琛ㄥ睘鎬т俊鎭?/span> </strong><strong><span>SERDEPROPERTIES</span> </strong><strong><span>錛屽湪</span> </strong><strong><span>SerDe</span> </strong><strong><span>涓氳繃榪欎簺灞炴т俊鎭彲浠ユ湁鏇村鐨勫畾鍒惰涓恒?/span></strong><strong></strong></span></p> <p><strong><span><span><span>2銆?/span> </span></span></strong><span><strong><span>鏁版嵁瀵煎叆瀵煎嚭</span></strong><strong></strong></span></p> <p><strong><span><span><span>a)</span> <span>         </span></span></span></strong><span><strong><span>澶氱増鏈棩蹇楁牸寮忕殑鍏煎</span></strong><strong></strong></span></p> <p><span><strong><span><span><span>        </span></span>鐢變簬</span> </strong><strong><span>hive</span> </strong><strong><span>鐨勫簲鐢ㄥ満鏅富瑕佹槸澶勭悊鍐鋒暟鎹紙鍙涓嶅啓錛夛紝鍥犳瀹冨彧鏀寔鎵歸噺瀵煎叆鍜屽鍑烘暟鎹紝騫朵笉鏀寔鍗曟潯鏁版嵁鐨勫啓鍏ユ垨鏇存柊錛屾墍浠ュ鏋滆瀵煎叆鐨勬暟鎹瓨鍦ㄦ煇浜涗笉澶鑼冪殑琛岋紝鍒欓渶瑕佹垜浠畾鍒朵竴浜涙墿灞曞姛鑳藉鍏惰繘琛屽鐞嗐?/span></strong><strong></strong></span></p> <p><span><strong><span><span><span>        </span></span>鎴戜滑闇瑕佸鐞嗙殑鏃ュ織鏁版嵁瀛樺湪澶氫釜鐗堟湰錛屽悇涓増鏈瘡涓瓧孌電殑鏁版嵁鍐呭瀛樺湪涓浜涘樊寮傦紝鍙兘鐗堟湰</span> </strong><strong><span>A</span> </strong><strong><span>鏃ュ織鏁版嵁鐨勭浜屼釜鍒楁槸鎼滅儲鍏抽敭瀛楋紝浣嗙増鏈?/span> </strong><strong><span>B</span> </strong><strong><span>鐨勭浜屽垪鍗存槸鎼滅儲鐨勭粓绔被鍨嬶紝濡傛灉榪欎袱涓増鏈殑鏃ュ織鐩存帴瀵煎叆</span> </strong><strong><span>hive</span> </strong><strong><span>涓紝寰堟槑鏄炬暟鎹皢浼氭販涔憋紝緇熻緇撴灉涔熶笉浼氭紜傛垜浠殑浠誨姟鏄浣垮涓増鏈殑鏃ュ織鏁版嵁鑳藉湪</span> </strong><strong><span>hive</span> </strong><strong><span>鏁版嵁浠撳簱涓叡瀛橈紝涓旇〃鐨?/span> </strong><strong><span>input/output</span> </strong><strong><span>鎿嶄綔鑳藉鏈緇堟槧灝勫埌姝g‘鐨勬棩蹇楃増鏈殑姝g‘瀛楁銆?/span></strong><strong></strong></span></p> <p><span><strong><span><span><span>        </span></span>榪欓噷鎴戜滑涓嶅叧蹇冭繖閮ㄥ垎綣佺悙鐨勫伐浣滐紝鍙叧蹇冩妧鏈疄鐜扮殑鍏抽敭鐐癸紝榪欎釜鍔熻兘璇ュ湪鍝噷瀹炵幇鎵嶈兘璁?/span> </strong><strong><span>hive</span> </strong><strong><span>璁ゅ緱榪欎簺涓嶅悓鏍煎紡鐨勬暟鎹憿錛熺粡榪囧鏂瑰皾璇曪紝鍦ㄤ腑闂翠換浣曠幆鑺傚仛榪欎釜鐗堟湰閫傞厤閮藉皢瀵艱嚧澶嶆潅鍖栵紝鏈緇堣繖涓伐浣滆繕鏄湪</span> </strong><strong><span>inputformat/outputformat</span> </strong><strong><span>涓畬鎴愭渶涓轟紭闆咃紝姣曠珶</span> </strong><strong><span>inputformat</span> </strong><strong><span>鏄簮澶達紝</span> </strong><strong><span>outputformat</span> </strong><strong><span>鏄渶緇堝綊瀹褲傚叿浣撴潵璇達紝鏄湪鍓嶉潰鎻愬埌鐨?/span> </strong><strong><span>inputformat</span> </strong><strong><span>鐨?/span> </strong><strong><span>next</span> </strong><strong><span>鏂規硶涓拰鍦?/span> </strong><strong><span>outputformat</span> </strong><span>鐨?/span> <span><span>write</span> </span><span>鏂規硶涓畬鎴愯繖涓傞厤宸ヤ綔銆?/span></span></p> <p><span><span><span>b)</span> <span>         </span></span></span><span><span><span>Hive</span> </span><span>鎿嶄綔鏈湴鏁版嵁</span></span></p> <p><span><span><span><span>        </span></span>涓寮濮嬶紝鎬繪槸鎶婃湰鍦版暟鎹厛浼犲埌</span> <span><span>HDFS</span> </span><span>錛屽啀鐢?/span> <span><span>hive</span> </span><span>鎿嶄綔</span> <span><span>hdfs</span> </span><span>涓婄殑鏁版嵁錛岀劧鍚庡啀鎶婃暟鎹粠</span> <span><span>HDFS</span> </span><span>涓婁紶鍥炴湰鍦版暟鎹傚悗鏉ュ彂鐜板ぇ鍙笉蹇呭姝わ紝</span> <span><span>hive</span> </span><span>璇彞閮芥彁渚涗簡“</span> <span><span>local</span> </span><span>”鍏抽敭瀛楋紝鏀寔鐩存帴浠庢湰鍦板鍏ユ暟鎹埌</span> <span><span>hive</span> </span><span>錛屼篃鑳戒粠</span> <span><span>hive</span> </span><span>鐩存帴瀵煎嚭鏁版嵁鍒版湰鍦幫紝涓嶈繃鍏跺唴閮ㄨ綆楁椂褰撶劧鏄敤</span> <span><span>HDFS</span> </span><span>涓婄殑鏁版嵁錛屽彧鏄嚜鍔ㄤ負鎴戜滑瀹屾垚瀵煎叆瀵煎嚭鑰屽凡銆?/span></span></p> <p><strong><span><span><span>3銆?/span> </span></span></strong><span><strong><span>鏁版嵁澶勭悊</span></strong><strong></strong></span></p> <p><span><strong><span>鏃ュ織鏁版嵁鐨勭粺璁″鐞嗗湪榪欓噷鍙嶅掓病鏈変粈涔堢壒鍒箣澶勶紝灝辨槸涓浜?/span> </strong><strong><span>SQL</span> </strong><strong><span>璇彞鑰屽凡錛屼篃娌℃湁浠涔堥珮娣辯殑鎶宸э紝涓嶈繃榪樻槸鍒椾婦涓浜涜鍙ョず渚嬶紝浠ョず</span> </strong><strong><span>hive</span> </strong><strong><span>澶勭悊鏁版嵁鐨勬柟渚夸箣澶勶紝騫跺睍紺?/span> </strong><strong><span>hive</span> </strong><strong><span>鐨勪竴浜涚敤娉曘?/span></strong><strong></strong></span></p> <p><strong><span><span><span>a)</span> <span>         </span></span></span></strong><span><strong><span>涓?/span> </strong><strong><span>hive</span> </strong><strong><span>娣誨姞鐢ㄦ埛瀹氬埗鍔熻兘錛岃嚜瀹氫箟鍔熻兘閮戒綅浜?/span> </strong><strong><span>hive_contrib.jar</span> </strong><strong><span>鍖呬腑</span></strong><strong></strong></span></p> <p><strong><span><span>add jar /opt/hadoop/hive-0.5.0-bin/lib/hive_contrib.jar;</span></span></strong></p> <p><strong><span><span><span>b)</span> <span>         </span></span></span></strong><span><strong><span>緇熻姣忎釜鍏抽敭璇嶇殑鎼滅儲閲忥紝騫舵寜鎼滅儲閲忛檷搴忔帓鍒楋紝鐒跺悗鎶婄粨鏋滃瓨鍏ヨ〃</span> </strong><strong><span>keyword_20100603</span> </strong><strong><span>涓?/span></strong><strong></strong></span></p> <p><strong><span><span>create table keyword_20100603 as select keyword,count(keyword) as count from searchlog_20100603 group by keyword order by count desc;</span></span></strong></p> <p><strong><span><span><span>c)</span> <span>         </span></span></span></strong><span><strong><span>緇熻姣忕被鐢ㄦ埛緇堢鐨勬悳绱㈤噺錛屽茍鎸夋悳绱㈤噺闄嶅簭鎺掑垪錛岀劧鍚庢妸緇撴灉瀛樺叆琛?/span> </strong><strong><span>device_20100603</span> </strong><strong><span>涓?/span></strong><strong></strong></span></p> <p><strong><span><span>create table device_20100603 as select device,count(device) as count from searchlog_20100603 group by device order by count desc;</span></span></strong></p> <p><strong><span><span><span>d)</span> <span>         </span></span></span></strong><span><strong><span>鍒涘緩琛?/span> </strong><strong><span>time_20100603</span> </strong><strong><span>錛屼嬌鐢ㄨ嚜瀹氫箟鐨?/span> </strong><strong><span>INPUTFORMAT</span> </strong><strong><span>鍜?/span> </strong><strong><span>OUTPUTFORMAT</span> </strong><strong><span>錛屽茍鎸囧畾琛ㄦ暟鎹殑鐪熷疄瀛樻斁浣嶇疆鍦?/span> </strong><strong><span>'/LogAnalysis/results/time_20100603'</span> </strong><strong><span>錛?/span> </strong><strong><span>HDFS</span> </strong><strong><span>璺緞錛夛紝鑰屼笉鏄斁鍦?/span> </strong><strong><span>hive</span> </strong><strong><span>鑷繁鐨勬暟鎹洰褰曚腑</span></strong><strong></strong></span></p> <p><strong><span><span>create external table if not exists time_20100603(time string, count int) stored as INPUTFORMAT 'com.aspire.search.loganalysis.hive.XmlResultInputFormat' OUTPUTFORMAT 'com.aspire.search.loganalysis.hive.XmlResultOutputFormat' LOCATION '/LogAnalysis/results/time_20100603';</span></span></strong></p> <p><strong><span><span><span>e)</span> <span>         </span></span></span></strong><span><strong><span>緇熻姣忕璁塊棶閲?/span> </strong><strong><span>TPS</span> </strong><strong><span>錛屾寜璁塊棶閲忛檷搴忔帓鍒楋紝騫舵妸緇撴灉杈撳嚭鍒拌〃</span> </strong><strong><span>time_20100603</span> </strong><strong><span>涓紝榪欎釜琛ㄦ垜浠湪涓婇潰鍒氬垰瀹氫箟榪囷紝鍏剁湡瀹炰綅緗湪</span> </strong><strong><span>'/LogAnalysis/results/time_20100603'</span> </strong><strong><span>錛屽茍涓旂敱浜?/span> </strong><strong><span>XmlResultOutputFormat</span> </strong><strong><span>鐨勬牸寮忓寲錛屾枃浠跺唴瀹規槸</span> </strong><strong><span>XML</span> </strong><strong><span>鏍煎紡銆?/span></strong><strong></strong></span></p> <p><strong><span><span>insert overwrite table time_20100603 select time,count(time) as count from searchlog_20100603 group by time order by count desc;</span></span></strong></p> <p><strong><span><span><span>f)</span> <span>          </span></span></span></strong><span><strong><span>璁$畻姣忎釜鎼滅儲璇鋒眰鍝嶅簲鏃墮棿鐨勬渶澶у鹼紝鏈灝忓煎拰騫沖潎鍊?/span></strong><strong></strong></span></p> <p><strong><span><span>insert overwrite table response_20100603 select max(responsetime) as max,min(responsetime) as min,avg(responsetime) as avg from searchlog_20100603;</span></span></strong></p> <p><strong><span><span><span>g)</span> <span>         </span></span></span></strong><span><strong><span>鍒涘緩涓涓〃鐢ㄤ簬瀛樻斁浠婂ぉ涓庢槰澶╃殑鍏抽敭璇嶆悳绱㈤噺鍜屽閲忓強鍏跺閲忔瘮鐜囷紝琛ㄦ暟鎹綅浜?/span> </strong><strong><span>'/LogAnalysis/results/keyword_20100604_20100603'</span> </strong><strong><span>錛屽唴瀹瑰皢鏄?/span> </strong><strong><span>XML</span> </strong><strong><span>鏍煎紡銆?/span></strong><strong></strong></span></p> <p><strong><span><span>create external table if not exists keyword_20100604_20100603(keyword string, count int, increment int, incrementrate double) stored as INPUTFORMAT 'com.aspire.search.loganalysis.hive.XmlResultInputFormat' OUTPUTFORMAT 'com.aspire.search.loganalysis.hive.XmlResultOutputFormat' LOCATION '/LogAnalysis/results/keyword_20100604_20100603';</span></span></strong></p> <p><strong><span><span><span>h)</span> <span>         </span></span></span></strong><span><strong><span>璁劇疆琛ㄧ殑灞炴э紝浠ヤ究</span> </strong><strong><span>XmlResultInputFormat</span> </strong><strong><span>鍜?/span> </strong><strong><span>XmlResultOutputFormat</span> </strong><strong><span>鑳芥牴鎹?/span> </strong><strong><span>output.resulttype</span> </strong><strong><span>鐨勪笉鍚屽唴瀹硅緭鍑轟笉鍚屾牸寮忕殑</span> </strong><strong><span>XML</span> </strong><strong><span>鏂囦歡銆?/span></strong><strong></strong></span></p> <p><strong><span><span>alter table keyword_20100604_20100603 set tblproperties ('output.resulttype'='keyword');</span></span></strong></p> <p><strong><span><span><span>i)</span> <span>           </span></span></span></strong><span><strong><span>鍏寵仈浠婂ぉ鍏抽敭璇嶇粺璁$粨鏋滆〃錛?/span> </strong><strong><span>keyword_20100604</span> </strong><strong><span>錛変笌鏄ㄥぉ鍏抽敭璇嶇粺璁$粨鏋滆〃錛?/span> </strong><strong><span>keyword_20100603</span> </strong><strong><span>錛夛紝緇熻浠婂ぉ涓庢槰澶╁悓鏃跺嚭鐜扮殑鍏抽敭璇嶇殑鎼滅儲嬈℃暟錛屼粖澶╃浉瀵規槰澶╃殑澧為噺鍜屽閲忔瘮鐜囷紝騫舵寜澧為噺姣旂巼闄嶅簭鎺掑垪錛岀粨鏋滆緭鍑哄埌鍒氬垰瀹氫箟鐨?/span> </strong><strong><span>keyword_20100604_20100603</span> </strong><strong><span>琛ㄤ腑錛屽叾鏁版嵁鏂囦歡鍐呭灝嗕負</span> </strong><strong><span>XML</span> </strong><strong><span>鏍煎紡銆?/span></strong><strong></strong></span></p> <p><strong><span><span>insert overwrite table keyword_20100604_20100603 select cur.keyword, cur.count, cur.count-yes.count as increment, (cur.count-yes.count)/yes.count as incrementrate from keyword_20100604 cur join keyword_20100603 yes on (cur.keyword = yes.keyword) order by incrementrate desc;</span></span></strong></p> <p><strong><span><span><span>j)</span> <span>           </span></span></span></strong><strong><span> </span></strong></p> <p><strong><span> </span></strong></p> <p><strong><span><span><span>4銆?/span> </span></span></strong><span><strong><span>鐢ㄦ埛鑷畾涔夊嚱鏁?/span> </strong><strong><span>UDF</span></strong></span></p> <p><span><strong><span>閮ㄥ垎緇熻緇撴灉闇瑕佷互</span> </strong><strong><span>CSV</span> </strong><strong><span>鐨勬牸寮忚緭鍑猴紝瀵逛簬榪欑被鏂囦歡浣撳叏鏄湁鏁堝唴瀹圭殑鏂囦歡錛屼笉闇瑕佸儚</span> </strong><strong><span>XML</span> </strong><strong><span>涓鏍峰寘鍚?/span> </strong><strong><span>version</span> </strong><strong><span>錛?/span> </strong><strong><span>encoding</span> </strong><strong><span>絳変俊鎭殑鏂囦歡澶達紝鏈閫傚悎鐢?/span> </strong><strong><span>UDF(user define function)</span> </strong><strong><span>浜嗐?/span></strong><strong></strong></span></p> <p><span><strong><span>UDF</span> </strong><strong><span>鍑芥暟鍙洿鎺ュ簲鐢ㄤ簬</span> </strong><strong><span>select</span> </strong><strong><span>璇彞錛屽鏌ヨ緇撴瀯鍋氭牸寮忓寲澶勭悊涔嬪悗錛屽啀杈撳嚭鍐呭銆傝嚜瀹氫箟</span> </strong><strong><span>UDF</span> </strong><strong><span>闇瑕佺戶鎵?/span> </strong></span><span>org.apache.hadoop.hive.ql.exec.UDF</span> <strong><span><span>錛屽茍瀹炵幇</span> </span></strong><span>evaluate</span> <strong><span><span>鍑芥暟錛?/span> </span></strong><span>Evaluate</span> <span><strong><span>鍑芥暟鏀寔閲嶈澆錛岃繕鏀寔鍙彉鍙傛暟銆傛垜浠疄鐜頒簡涓涓敮鎸佸彲鍙樺瓧絎︿覆鍙傛暟鐨?/span> </strong><strong><span>UDF</span> </strong><strong><span>錛屾敮鎸佹妸</span> </strong><strong><span>select</span> </strong><strong><span>寰楀嚭鐨勪換鎰忎釜鏁扮殑涓嶅悓綾誨瀷鏁版嵁杞崲涓哄瓧絎︿覆鍚庯紝鎸?/span> </strong><strong><span>CSV</span> </strong><strong><span>鏍煎紡杈撳嚭錛岀敱浜庝唬鐮佽緝綆鍗曪紝榪欓噷緇欏嚭婧愮爜紺轟緥錛?/span></strong><strong></strong></span></p> <p align=left><span><span>    </span></span><strong><span>public</span> </strong><span><span>String</span> evaluate(String... strs) {</span></p> <p align=left><span><span>       </span>StringBuilder sb = </span><strong><span>new</span> </strong><span>StringBuilder();</span></p> <p align=left><span><span>       </span></span><strong><span>for</span> </strong><span>(</span> <strong><span>int</span> </strong><span>i = 0; i < strs.</span> <span>length</span> <span>; i++) {</span></p> <p align=left><span><span>           </span>sb.append(ConvertCSVField(strs[i])).append(</span> <span>','</span> <span>);</span></p> <p align=left><span><span>       </span>}</span></p> <p align=left><span><span>       </span>sb.deleteCharAt(sb.length()-1);</span></p> <p align=left><span><span>       </span></span><strong><span>return</span> </strong><span>sb.toString();</span></p> <p><span>}</span><strong></strong></p> <p><span><strong><span><span>         </span></span></strong><strong><span>闇瑕佹敞鎰忕殑鏄紝瑕佷嬌鐢?/span> </strong><strong><span>UDF</span> </strong><strong><span>鍔熻兘錛岄櫎浜嗗疄鐜拌嚜瀹氫箟</span> </strong><strong><span>UDF</span> </strong><strong><span>澶栵紝榪橀渶瑕佸姞鍏ュ寘鍚?/span> </strong><strong><span>UDF</span> </strong><strong><span>鐨勫寘錛岀ず渚嬶細</span></strong><strong></strong></span></p> <p><strong><span><span>add jar /opt/hadoop/hive-0.5.0-bin/lib/hive_contrib.jar;</span></span></strong></p> <p><span><strong><span>鐒跺悗鍒涘緩涓存椂鏂規硶錛岀ず渚嬶細</span></strong><strong></strong></span></p> <p><strong><span><span>CREATE TEMPORARY FUNCTION Result2CSv AS ‘com.aspire.search.loganalysis.hive. Result2CSv';</span></span></strong></p> <p><span><strong><span><span>         </span></span></strong><strong><span>浣跨敤瀹屾瘯榪樿</span> </strong><strong><span>drop</span> </strong><strong><span>鏂規硶錛岀ず渚嬶細</span></strong><strong></strong></span></p> <p><span><strong><span>DROP TEMPORARY FUNCTION Result2CSv;</span></strong><strong></strong></span></p> <p><strong><span><span><span>5銆?/span> <span>  </span></span></span></strong><span><strong><span>杈撳嚭</span> </strong><strong><span>XML</span> </strong><strong><span>鏍煎紡鐨勭粺璁$粨鏋?/span></strong><strong></strong></span></p> <p><span><strong><span>鍓嶉潰鐪嬪埌閮ㄥ垎鏃ュ織緇熻緇撴灉杈撳嚭鍒頒竴涓〃涓紝鍊熷姪</span> </strong><strong><span>XmlResultInputFormat</span> </strong><strong><span>鍜?/span> </strong><strong><span>XmlResultOutputFormat</span> </strong><strong><span>鏍煎紡鍖栨垚</span> </strong><strong><span>XML</span> </strong><strong><span>鏂囦歡錛岃冭檻鍒板垱寤鴻繖涓〃鍙槸涓轟簡寰楀埌</span> </strong><strong><span>XML</span> </strong><strong><span>鏍煎紡鐨勮緭鍑烘暟鎹紝鎴戜滑鍙渶瀹炵幇</span> </strong><strong><span>XmlResultOutputFormat</span> </strong><strong><span>鍗沖彲錛屽鏋滆繕瑕佹敮鎸?/span> </strong><strong><span>select</span> </strong><strong><span>鏌ヨ錛屽垯鎴戜滑榪橀渶瑕佸疄鐜?/span> </strong><strong><span>XmlResultInputFormat</span> </strong><strong><span>錛岃繖閲屾垜浠彧浠嬬粛</span> </strong><strong><span>XmlResultOutputFormat</span> </strong><strong><span>銆?/span></strong><strong></strong></span></p> <p><span><strong><span>鍓嶉潰浠嬬粛榪囷紝瀹氬埗</span> </strong><strong><span>XmlResultOutputFormat</span> </strong><strong><span>鎴戜滑鍙渶閲嶅啓</span> </strong><strong><span>write</span> </strong><strong><span>鍗沖彲錛岃繖涓柟娉曞皢浼氭妸</span> </strong><strong><span>hive</span> </strong><strong><span>鐨勪互</span> </strong><strong><span>’\001’</span> </strong><strong><span>鍒嗛殧鐨勫瀛楁鏁版嵁鏍煎紡鍖栦負鎴戜滑闇瑕佺殑</span> </strong><strong><span>XML</span> </strong><strong><span>鏍煎紡錛岃綆鍖栫殑紺轟緥浠g爜濡備笅錛?/span></strong><strong></strong></span></p> <p align=left><span><span>    </span></span><strong><span>public</span> </strong><strong><span>void</span> </strong><span>write(Writable w) </span><strong><span>throws</span> </strong><span>IOException {</span></p> <p align=left><span><span>           </span>String[] strFields = ((Text) w).toString().split(</span> <span>"\001"</span> <span>);</span></p> <p align=left><span><span>           </span>StringBuffer sbXml = </span><strong><span>new</span> </strong><span>StringBuffer();</span></p> <p align=left><span><span>           </span></span><strong><span>if</span> </strong><span>(</span> <span>strResultType</span> <span>.equals(</span> <span>"keyword"</span> <span>)) {</span></p> <p align=left><span><span>    </span>sbXml.append(</span> <span>"<record><keyword>"</span> <span>).append(strFields[0]).append(</span></p> <p align=left><span><span>    </span></span><span>"</keyword><count>"</span> <span>).append(strFields[1]).append(<span>       </span><span>    </span></span><span>"</count><increment>"</span> <span>).append(strFields[2]).append(</span></p> <p align=left><span><span>    </span></span><span>"</increment><rate>"</span> <span>).append(strFields[3]).append(</span></p> <p align=left><span>"</rate></result>"</span> <span>);</span></p> <p align=left><span><span>           </span>}</span></p> <p align=left><span><span>           </span>Text txtXml = </span><strong><span>new</span> </strong><span>Text();</span></p> <p align=left><span><span>           </span></span><strong><span>byte</span> </strong><span>[] strBytes = sbXml.toString().getBytes(</span> <span>"utf-8"</span> <span>);</span></p> <p align=left><span><span>           </span>txtXml.set(strBytes, 0, strBytes.</span> <span>length</span> <span>);</span></p> <p align=left><span><span>           </span></span><strong><span>byte</span> </strong><span>[] output = txtXml.getBytes();</span></p> <p align=left><span><span>       </span><span>    </span></span><span>bytesWritable</span> <span>.set(output, 0, output.</span> <span>length</span> <span>);</span></p> <p align=left><span><span>           </span></span><span>writer</span> <span>.write(</span> <span>bytesWritable</span> <span>);</span></p> <p align=left><span><span>    </span>}<strong></strong></span></p> <p><strong><span><span><span><span>        </span></span>鍏朵腑鐨?/span> </span></strong><span>strResultType</span> <span>.equals(</span> <span>"keyword"</span> <span>)</span> <span><strong><span>鎸囧畾鍏抽敭璇嶇粺璁$粨鏋滐紝榪欎釜灞炴ф潵鑷互涓嬭鍙ュ緇撴灉綾誨瀷鐨勬寚瀹氾紝閫氳繃榪欎釜灞炴ф垜浠繕鍙互鐢ㄥ悓涓涓?/span> </strong><strong><span>outputformat</span> </strong><strong><span>杈撳嚭澶氱綾誨瀷鐨勭粨鏋溿?/span></strong><strong></strong></span></p> <p><strong><span><span><span><span>        </span></span>alter table keyword_20100604_20100603 set tblproperties ('output.resulttype'='keyword');</span></span></strong></p> <p><span><strong><span><span><span>        </span></span>浠旂粏鐪嬬湅</span> </strong><strong><span>write</span> </strong><strong><span>鍑芥暟鐨勫疄鐜頒究鍙彂鐜幫紝鍏跺疄榪欓噷鍙緭鍑轟簡</span> </strong><strong><span>XML</span> </strong><strong><span>鏂囦歡鐨勬鏂囷紝鑰?/span> </strong><strong><span>XML</span> </strong><strong><span>鐨勬枃浠跺ご鍜岀粨鏉熸爣絳懼湪鍝噷杈撳嚭鍛紵鎵騫告垜浠噰鐢ㄧ殑鏄熀浜?/span> </strong><strong><span>outputformat</span> </strong><strong><span>鐨勫疄鐜幫紝鎴戜滑鍙互鍦ㄦ瀯閫犲嚱鏁拌緭鍑?/span> </strong><strong><span>version</span> </strong><strong><span>錛?/span> </strong><strong><span>encoding</span> </strong><strong><span>絳夋枃浠跺ご淇℃伅錛屽湪</span> </strong><strong><span>close()</span> </strong><strong><span>鏂規硶涓緭鍑虹粨鏉熸爣絳俱?/span></strong><strong></strong></span></p> <p><span><strong><span><span><span>        </span></span>榪欎篃鏄垜浠負浠涔堜笉浣跨敤</span> </strong><strong><span>UDF</span> </strong><strong><span>鏉ヨ緭鍑虹粨鏋滅殑鍘熷洜錛岃嚜瀹氫箟</span> </strong><strong><span>UDF</span> </strong><strong><span>鍑芥暟涓嶈兘杈撳嚭鏂囦歡澶村拰鏂囦歡灝撅紝瀵逛簬</span> </strong><strong><span>XML</span> </strong><strong><span>鏍煎紡鐨勬暟鎹棤娉曡緭鍑哄畬鏁存牸寮忥紝鍙兘杈撳嚭</span> </strong><strong><span>CSV</span> </strong><strong><span>榪欑被鎵鏈夎閮芥槸鏈夋晥鏁版嵁鐨勬枃浠躲?/span></strong><strong></strong></span></p> <p><strong><span><span>浜斻?span>           </span></span></span></strong><strong><span>鎬葷粨</span></strong><strong></strong></p> <p><span><strong><span><span><span>        </span></span>Hive</span> </strong><strong><span>鏄竴涓彲鎵╁睍鎬ф瀬寮虹殑鏁版嵁浠撳簱宸ュ叿錛屽熷姪浜?/span> </strong><strong><span>hadoop</span> </strong><strong><span>鍒嗗竷寮忓瓨鍌ㄨ綆楀鉤鍙板拰</span> </strong><strong><span>hive</span> </strong><strong><span>瀵?/span> </strong><strong><span>SQL</span> </strong><strong><span>璇彞鐨勭悊瑙h兘鍔涳紝鎴戜滑鎵瑕佸仛鐨勫ぇ閮ㄥ垎宸ヤ綔灝辨槸杈撳叆鍜岃緭鍑烘暟鎹殑閫傞厤錛屾伆鎭拌繖涓ら儴鍒?/span> </strong><strong><span>IO</span> </strong><strong><span>鏍煎紡鏄崈鍙樹竾鍖栫殑錛屾垜浠彧闇瑕佸畾鍒舵垜浠嚜宸辯殑杈撳叆杈撳嚭閫傞厤鍣紝</span> </strong><strong><span>hive</span></strong><strong><span>灝嗕負鎴戜滑閫忔槑鍖栧瓨鍌ㄥ拰澶勭悊榪欎簺鏁版嵁錛屽ぇ澶х畝鍖栨垜浠殑宸ヤ綔銆傛湰鏂囩殑閲嶅績涔熸鍦ㄤ簬姝わ紝榪欓儴鍒嗗伐浣滅浉淇℃瘡涓涓仛鏁版嵁鍒嗘瀽鐨勬湅鍙嬮兘浼氶潰瀵圭殑錛屽笇鏈涘鎮ㄦ湁鐩娿?/span></strong><strong></strong></span></p> <p><span><strong><span><span><span>        </span></span>鏈枃浠嬬粛浜嗕竴嬈$浉褰撶畝鍗曠殑鍩轟簬</span> </strong><strong><span>hive</span> </strong><strong><span>鐨勬棩蹇楃粺璁″疄鎴橈紝瀵?/span> </strong><strong><span>hive</span> </strong><strong><span>鐨勮繍鐢ㄨ繕澶勪簬涓涓浉瀵硅緝嫻呯殑灞傞潰錛岀洰鍓嶅皻鑳芥弧瓚抽渶姹傘傚浜庝竴浜涜緝澶嶆潅鐨勬暟鎹垎鏋愪換鍔★紝浠ヤ笂鎵浠嬬粛鐨勭粡楠屽緢鍙兘鏄笉澶熺敤鐨勶紝鐢氳嚦鏄?/span> </strong><strong><span>hive</span> </strong><strong><span>鍋氫笉鍒扮殑錛?/span> </strong><strong><span>hive</span> </strong><strong><span>榪樻湁寰堝榪涢樁鍔熻兘錛岄檺浜庣瘒騫呮湰鏂囨湭鑳芥秹鍙婏紝寰呮棩鍚庣粨鍚堝叿浣撲換鍔″啀璇︾粏闃愯堪銆?/span></strong><strong></strong></span></p> <p><span><strong><span><span><span>        </span></span>濡傛偍瀵規湰鏂囨湁浠諱綍寤鴻鎴栨寚鏁欙紝璇瘋瘎璁猴紝璋㈣阿銆?/span></strong></span></p> </span></span></span></span></strong></span></span> <img src ="http://www.shnenglu.com/koson/aggbug/120773.html" width = "1" height = "1" /><br><br><div align=right><a style="text-decoration:none;" href="http://www.shnenglu.com/koson/" target="_blank">koson</a> 2010-07-19 14:39 <a href="http://www.shnenglu.com/koson/archive/2010/07/19/120773.html#Feedback" target="_blank" style="text-decoration:none;">鍙戣〃璇勮</a></div>]]></description></item></channel></rss> <a href="http://www.shnenglu.com/">青青草原综合久久大伊人导航_色综合久久天天综合_日日噜噜夜夜狠狠久久丁香五月_热久久这里只有精品</a> <div style="position:fixed;left:-9000px;top:-9000px;"><font id="pjuwb"></font><button id="pjuwb"><pre id="pjuwb"></pre></button><sub id="pjuwb"></sub><tbody id="pjuwb"><var id="pjuwb"><address id="pjuwb"></address></var></tbody><listing id="pjuwb"><label id="pjuwb"><strong id="pjuwb"></strong></label></listing><wbr id="pjuwb"><small id="pjuwb"><tbody id="pjuwb"></tbody></small></wbr><ins id="pjuwb"><xmp id="pjuwb"></xmp></ins><style id="pjuwb"></style><label id="pjuwb"><em id="pjuwb"><li id="pjuwb"></li></em></label><samp id="pjuwb"></samp><menu id="pjuwb"><input id="pjuwb"></input></menu><pre id="pjuwb"><tbody id="pjuwb"><tfoot id="pjuwb"><button id="pjuwb"></button></tfoot></tbody></pre><form id="pjuwb"></form><i id="pjuwb"><style id="pjuwb"><label id="pjuwb"><sup id="pjuwb"></sup></label></style></i><li id="pjuwb"><table id="pjuwb"><abbr id="pjuwb"></abbr></table></li><video id="pjuwb"></video><dfn id="pjuwb"></dfn><progress id="pjuwb"></progress><strong id="pjuwb"></strong><mark id="pjuwb"></mark><em id="pjuwb"></em><tbody id="pjuwb"><p id="pjuwb"><strike id="pjuwb"><acronym id="pjuwb"></acronym></strike></p></tbody><option id="pjuwb"></option><strike id="pjuwb"></strike><u id="pjuwb"></u><td id="pjuwb"><center id="pjuwb"><tr id="pjuwb"></tr></center></td><em id="pjuwb"><mark id="pjuwb"><em id="pjuwb"><tt id="pjuwb"></tt></em></mark></em><strong id="pjuwb"></strong><wbr id="pjuwb"></wbr><s id="pjuwb"></s><strong id="pjuwb"></strong><legend id="pjuwb"></legend><nav id="pjuwb"></nav><dl id="pjuwb"><th id="pjuwb"><dl id="pjuwb"></dl></th></dl><noframes id="pjuwb"><ins id="pjuwb"></ins></noframes><font id="pjuwb"></font><strike id="pjuwb"><i id="pjuwb"><style id="pjuwb"><label id="pjuwb"></label></style></i></strike><output id="pjuwb"></output><thead id="pjuwb"><pre id="pjuwb"></pre></thead><source id="pjuwb"></source><menuitem id="pjuwb"><wbr id="pjuwb"></wbr></menuitem><pre id="pjuwb"><span id="pjuwb"><pre id="pjuwb"><big id="pjuwb"></big></pre></span></pre><cite id="pjuwb"><fieldset id="pjuwb"><s id="pjuwb"><rt id="pjuwb"></rt></s></fieldset></cite><big id="pjuwb"><progress id="pjuwb"><big id="pjuwb"></big></progress></big><samp id="pjuwb"><delect id="pjuwb"></delect></samp><dl id="pjuwb"></dl><strike id="pjuwb"><nav id="pjuwb"><dl id="pjuwb"><strong id="pjuwb"></strong></dl></nav></strike><tbody id="pjuwb"><b id="pjuwb"><optgroup id="pjuwb"><rp id="pjuwb"></rp></optgroup></b></tbody><em id="pjuwb"></em><xmp id="pjuwb"><blockquote id="pjuwb"><pre id="pjuwb"></pre></blockquote></xmp> <i id="pjuwb"><abbr id="pjuwb"><i id="pjuwb"><abbr id="pjuwb"></abbr></i></abbr></i><center id="pjuwb"><acronym id="pjuwb"><center id="pjuwb"></center></acronym></center><pre id="pjuwb"></pre><ul id="pjuwb"><thead id="pjuwb"></thead></ul><blockquote id="pjuwb"><pre id="pjuwb"><sup id="pjuwb"></sup></pre></blockquote><acronym id="pjuwb"></acronym><big id="pjuwb"><s id="pjuwb"></s></big><th id="pjuwb"></th><th id="pjuwb"></th><tbody id="pjuwb"></tbody><thead id="pjuwb"><strike id="pjuwb"></strike></thead><th id="pjuwb"><dl id="pjuwb"><wbr id="pjuwb"></wbr></dl></th><dl id="pjuwb"><strong id="pjuwb"></strong></dl><abbr id="pjuwb"><noframes id="pjuwb"><noscript id="pjuwb"></noscript></noframes></abbr><td id="pjuwb"><ol id="pjuwb"></ol></td><li id="pjuwb"><noscript id="pjuwb"><abbr id="pjuwb"></abbr></noscript></li><small id="pjuwb"><bdo id="pjuwb"><nav id="pjuwb"></nav></bdo></small><style id="pjuwb"></style><optgroup id="pjuwb"><table id="pjuwb"></table></optgroup><center id="pjuwb"><tr id="pjuwb"><dfn id="pjuwb"></dfn></tr></center><th id="pjuwb"></th><u id="pjuwb"></u><tfoot id="pjuwb"><legend id="pjuwb"><i id="pjuwb"></i></legend></tfoot><mark id="pjuwb"></mark><meter id="pjuwb"></meter><nav id="pjuwb"></nav><acronym id="pjuwb"><pre id="pjuwb"><acronym id="pjuwb"><ul id="pjuwb"></ul></acronym></pre></acronym><acronym id="pjuwb"><pre id="pjuwb"><acronym id="pjuwb"><ul id="pjuwb"></ul></acronym></pre></acronym><nobr id="pjuwb"></nobr><sub id="pjuwb"><th id="pjuwb"><menuitem id="pjuwb"><wbr id="pjuwb"></wbr></menuitem></th></sub><thead id="pjuwb"><sub id="pjuwb"></sub></thead><ul id="pjuwb"><address id="pjuwb"><menuitem id="pjuwb"><meter id="pjuwb"></meter></menuitem></address></ul><dfn id="pjuwb"></dfn><pre id="pjuwb"></pre><input id="pjuwb"><cite id="pjuwb"><fieldset id="pjuwb"></fieldset></cite></input><u id="pjuwb"><form id="pjuwb"><u id="pjuwb"></u></form></u><kbd id="pjuwb"><em id="pjuwb"><mark id="pjuwb"></mark></em></kbd><tr id="pjuwb"></tr><del id="pjuwb"><form id="pjuwb"><address id="pjuwb"></address></form></del><tfoot id="pjuwb"><legend id="pjuwb"><ol id="pjuwb"><dl id="pjuwb"></dl></ol></legend></tfoot><menu id="pjuwb"><nobr id="pjuwb"><th id="pjuwb"><nobr id="pjuwb"></nobr></th></nobr></menu><fieldset id="pjuwb"></fieldset><pre id="pjuwb"><blockquote id="pjuwb"><samp id="pjuwb"></samp></blockquote></pre><xmp id="pjuwb"><sup id="pjuwb"><pre id="pjuwb"></pre></sup></xmp><span id="pjuwb"><progress id="pjuwb"></progress></span><font id="pjuwb"></font><var id="pjuwb"><abbr id="pjuwb"></abbr></var><strong id="pjuwb"><label id="pjuwb"><i id="pjuwb"><legend id="pjuwb"></legend></i></label></strong><tr id="pjuwb"><em id="pjuwb"><em id="pjuwb"><output id="pjuwb"></output></em></em></tr><thead id="pjuwb"><strike id="pjuwb"></strike></thead> <acronym id="pjuwb"></acronym><i id="pjuwb"></i><tt id="pjuwb"></tt><rt id="pjuwb"><source id="pjuwb"><rt id="pjuwb"></rt></source></rt><strike id="pjuwb"><acronym id="pjuwb"></acronym></strike><del id="pjuwb"></del><font id="pjuwb"><output id="pjuwb"><ins id="pjuwb"><output id="pjuwb"></output></ins></output></font><kbd id="pjuwb"><tr id="pjuwb"><kbd id="pjuwb"></kbd></tr></kbd><pre id="pjuwb"><sup id="pjuwb"><delect id="pjuwb"><samp id="pjuwb"></samp></delect></sup></pre><samp id="pjuwb"></samp><track id="pjuwb"></track><tr id="pjuwb"></tr><center id="pjuwb"></center><fieldset id="pjuwb"></fieldset><i id="pjuwb"></i><td id="pjuwb"></td><rt id="pjuwb"></rt><object id="pjuwb"></object><pre id="pjuwb"><progress id="pjuwb"><sub id="pjuwb"><thead id="pjuwb"></thead></sub></progress></pre><kbd id="pjuwb"><tr id="pjuwb"><option id="pjuwb"></option></tr></kbd><output id="pjuwb"><ins id="pjuwb"></ins></output><ol id="pjuwb"></ol><source id="pjuwb"></source><strong id="pjuwb"></strong><ruby id="pjuwb"></ruby><sub id="pjuwb"><meter id="pjuwb"><menuitem id="pjuwb"><meter id="pjuwb"></meter></menuitem></meter></sub><pre id="pjuwb"></pre><center id="pjuwb"></center><tr id="pjuwb"><tbody id="pjuwb"><xmp id="pjuwb"><dd id="pjuwb"></dd></xmp></tbody></tr><video id="pjuwb"></video><pre id="pjuwb"></pre><form id="pjuwb"><optgroup id="pjuwb"></optgroup></form><samp id="pjuwb"></samp><kbd id="pjuwb"></kbd><strong id="pjuwb"><option id="pjuwb"></option></strong><object id="pjuwb"></object><abbr id="pjuwb"><noframes id="pjuwb"><abbr id="pjuwb"></abbr></noframes></abbr><ul id="pjuwb"><del id="pjuwb"><button id="pjuwb"><pre id="pjuwb"></pre></button></del></ul><abbr id="pjuwb"></abbr><strong id="pjuwb"><code id="pjuwb"><strong id="pjuwb"></strong></code></strong><option id="pjuwb"></option><optgroup id="pjuwb"><bdo id="pjuwb"><code id="pjuwb"></code></bdo></optgroup><mark id="pjuwb"><em id="pjuwb"><font id="pjuwb"></font></em></mark><acronym id="pjuwb"><code id="pjuwb"></code></acronym><dl id="pjuwb"></dl><em id="pjuwb"></em><object id="pjuwb"><input id="pjuwb"><object id="pjuwb"></object></input></object><output id="pjuwb"><dd id="pjuwb"></dd></output><option id="pjuwb"><button id="pjuwb"><option id="pjuwb"></option></button></option><small id="pjuwb"></small></div> <a href="http://xissy.com" target="_blank">亚洲欧洲一区二区天堂久久 </a>| <a href="http://7485888.com" target="_blank">香港久久久电影</a>| <a href="http://xhtd688.com" target="_blank">欧美电影电视剧在线观看</a>| <a href="http://3848404.com" target="_blank">欧美成在线视频</a>| <a href="http://www5123ri.com" target="_blank">在线亚洲精品</a>| <a href="http://chufengguanye.com" target="_blank">国产精品啊v在线</a>| <a href="http://www72227.com" target="_blank">亚洲欧美综合一区</a>| <a href="http://www-11688.com" target="_blank">亚洲一级片在线观看</a>| <a href="http://caopormo.com" target="_blank">国产精品尤物</a>| <a href="http://987527.com" target="_blank">免费高清在线一区</a>| <a href="http://www633998.com" target="_blank">蜜臀久久99精品久久久画质超高清</a>| <a href="http://www909059.com" target="_blank">亚洲第一福利社区</a>| <a href="http://www993997.com" target="_blank">亚洲国内欧美</a>| <a href="http://wanzhixue.com" target="_blank">国产精品久久久久av免费</a>| <a href="http://99chiji1.com" target="_blank">久久爱www.</a>| <a href="http://china-cvct.com" target="_blank">久久青草欧美一区二区三区</a>| <a href="http://283575.com" target="_blank">亚洲欧洲日本一区二区三区</a>| <a href="http://hzdss.com" target="_blank">亚洲美女网站</a>| <a href="http://pron12.com" target="_blank">国产综合久久久久久鬼色</a>| <a href="http://win-mdk.com" target="_blank">欧美激情精品</a>| <a href="http://1194123.com" target="_blank">国产精自产拍久久久久久</a>| <a href="http://cm168168.com" target="_blank">久久综合九色综合网站</a>| <a href="http://lxpaiju.com" target="_blank">欧美电影免费观看高清完整版</a>| <a href="http://baoxiniao666.com" target="_blank">亚洲视频在线观看</a>| <a href="http://vod3366.com" target="_blank">欧美一区二区三区日韩视频</a>| <a href="http://hengbang50.com" target="_blank">亚洲国产精品久久久久婷婷老年</a>| <a href="http://yw5531.com" target="_blank">日韩视频一区二区</a>| <a href="http://080973.com" target="_blank">好吊视频一区二区三区四区 </a>| <a href="http://7373se.com" target="_blank">一区二区高清</a>| <a href="http://xvideoav99.com" target="_blank">久久国产精品色婷婷</a>| <a href="http://www-8617.com" target="_blank">99国产精品99久久久久久</a>| <a href="http://mathck.com" target="_blank">亚洲欧美综合网</a>| <a href="http://56qn.com" target="_blank">99国产精品久久久久老师</a>| <a href="http://0855dyw.com" target="_blank">久久精品国产精品亚洲</a>| <a href="http://evaporsllc.com" target="_blank">亚洲天堂av图片</a>| <a href="http://com42923.com" target="_blank">久久久久免费视频</a>| <a href="http://318823.com" target="_blank">亚洲免费影视第一页</a>| <a href="http://378682.com" target="_blank">久久亚洲精品一区</a>| <a href="http://668527.com" target="_blank">欧美一级午夜免费电影</a>| <a href="http://psykoptic.com" target="_blank">欧美成人性网</a>| <a href="http://dehuabz.com" target="_blank">久久人人九九</a>| <a href="http://wwwsfsf11.com" target="_blank">国产精品丝袜久久久久久app </a>| <a href="http://223533.com" target="_blank">亚洲天堂久久</a>| <a href="http://www-888593.com" target="_blank">久久午夜羞羞影院免费观看</a>| <a href="http://400206.com" target="_blank">欧美亚洲在线观看</a>| <a href="http://www-68689.com" target="_blank">欧美精品亚洲精品</a>| <a href="http://wy172.com" target="_blank">欧美承认网站</a>| <a href="http://8p3c.com" target="_blank">精品成人一区二区</a>| <a href="http://xiuxiu124.com" target="_blank">午夜精品久久久久久久久久久久久 </a>| <a href="http://www-188445.com" target="_blank">99国产精品99久久久久久粉嫩</a>| <a href="http://sese912.com" target="_blank">韩日在线一区</a>| <a href="http://jsydjxgs.com" target="_blank">欧美在线影院在线视频</a>| <a href="http://timonna.com" target="_blank">亚洲免费视频在线观看</a>| <a href="http://133686.com" target="_blank">欧美精品二区三区四区免费看视频</a>| <a href="http://gffffg.com" target="_blank">久久人人爽人人</a>| <a href="http://xxxyyy168.com" target="_blank">国产欧美一区二区色老头</a>| <a href="http://zztto7.com" target="_blank">99视频有精品</a>| <a href="http://qdpetlove.com" target="_blank">亚洲最新视频在线播放</a>| <a href="http://56718ca.com" target="_blank">欧美成年人视频</a>| <a href="http://91ymz.com" target="_blank">农夫在线精品视频免费观看</a>| <a href="http://teloregen.com" target="_blank">国产一区二区三区免费观看</a>| <a href="http://59jf.com" target="_blank">午夜精品久久久久久久白皮肤</a>| <a href="http://zisxks.com" target="_blank">亚洲欧美成人网</a>| <a href="http://4hup23.com" target="_blank">国产精品卡一卡二</a>| <a href="http://baidubf.com" target="_blank">在线一区观看</a>| <a href="http://029902.com" target="_blank">亚洲欧美色婷婷</a>| <a href="http://pinyuanad.com" target="_blank">亚洲一二三四久久</a>| <a href="http://wwwhaodd116.com" target="_blank">亚洲午夜视频在线</a>| <a href="http://548710.com" target="_blank">欧美日韩一区二区精品</a>| <a href="http://4466o.com" target="_blank">91久久久久久国产精品</a>| <a href="http://3wbb.com" target="_blank">亚洲精品国产无天堂网2021</a>| <a href="http://zz9876.com" target="_blank">毛片一区二区三区</a>| <a href="http://www47777.com" target="_blank">亚洲大片一区二区三区</a>| <a href="http://iotbzw.com" target="_blank">亚洲精品美女免费</a>| <a href="http://www-544778.com" target="_blank">欧美日韩大片</a>| <a href="http://qiezisp2.com" target="_blank">日韩午夜中文字幕</a>| <a href="http://by71222.com" target="_blank">亚洲午夜一级</a>| <a href="http://dsjgqc.com" target="_blank">国产美女精品一区二区三区</a>| <a href="http://xsxcn.com" target="_blank">亚洲在线不卡</a>| <a href="http://www11108b.com" target="_blank">久久精品一区二区三区四区</a>| <a href="http://hhsj31.com" target="_blank">韩国精品在线观看</a>| <a href="http://sese699.com" target="_blank">免费在线看一区</a>| <a href="http://9a5x.com" target="_blank">亚洲靠逼com</a>| <a href="http://mm778899.com" target="_blank">亚洲视频在线一区</a>| <a href="http://373gg.com" target="_blank">国产一区二区三区精品欧美日韩一区二区三区</a>| <a href="http://evaporsllc.com" target="_blank">亚洲一区二区在线看</a>| <a href="http://mm77gg.com" target="_blank">欧美一级片在线播放</a>| <a href="http://xsjj10.com" target="_blank">伊人狠狠色j香婷婷综合</a>| <a href="http://854277.com" target="_blank">久久综合伊人77777</a>| <a href="http://by4672.com" target="_blank">亚洲欧洲精品一区二区</a>| <a href="http://9928k.com" target="_blank">亚洲免费视频观看</a>| <a href="http://www227zh.com" target="_blank">国产在线日韩</a>| <a href="http://y65y.com" target="_blank">欧美另类69精品久久久久9999</a>| <a href="http://2220004.com" target="_blank">在线亚洲欧美</a>| <a href="http://414670.com" target="_blank">久久一区二区三区国产精品</a>| <a href="http://580078.com" target="_blank">亚洲激情欧美</a>| <a href="http://2502255.com" target="_blank">国产精品高潮呻吟久久</a>| <a href="http://qq6699.com" target="_blank">久久精品91</a>| <a href="http://929889.com" target="_blank">亚洲精品欧美在线</a>| <a href="http://taominze.com" target="_blank">久久九九精品99国产精品</a>| <a href="http://961318.com" target="_blank">亚洲国产成人av在线</a>| <a href="http://chenyirong.com" target="_blank">欧美日本乱大交xxxxx</a>| <a href="http://wwwyinyinai149.com" target="_blank">亚洲视频免费在线观看</a>| <a href="http://www-ss777.com" target="_blank">久久亚洲不卡</a>| <a href="http://juegold.com" target="_blank">中国成人亚色综合网站</a>| <a href="http://668527.com" target="_blank">经典三级久久</a>| <a href="http://www297777.com" target="_blank">欧美婷婷在线</a>| <a href="http://www-tk3333.com" target="_blank">久久久精品动漫</a>| <a href="http://csmgxun.com" target="_blank">亚洲精品一区在线</a>| <a href="http://899gh.com" target="_blank">久久最新视频</a>| <a href="http://894322.com" target="_blank">午夜久久美女</a>| <a href="http://99chiji1.com" target="_blank">日韩视频在线免费</a>| <a href="http://aijiashe.com" target="_blank">国内成人精品一区</a>| <a href="http://zxxx3.com" target="_blank">国产精品国产</a>| <a href="http://1323666.com" target="_blank">欧美激情黄色片</a>| <a href="http://www333393.com" target="_blank">午夜久久99</a>| <a href="http://www77577.com" target="_blank">99综合视频</a>| <a href="http://www-68689.com" target="_blank">亚洲福利免费</a>| <a href="http://3838418.com" target="_blank">美国成人毛片</a>| <a href="http://060969.com" target="_blank">久久精品一二三区</a>| <a href="http://lctongda.com" target="_blank">亚洲一区免费看</a>| <a href="http://tavbao.com" target="_blank">亚洲最新色图</a>| <a href="http://278538.com" target="_blank">亚洲精品美女久久7777777</a>| <a href="http://wwwxigua66.com" target="_blank">国内精品福利</a>| <a href="http://989p.com" target="_blank">国产一区二区精品丝袜</a>| <a href="http://spardec.com" target="_blank">欧美小视频在线</a>| <a href="http://ttdy20.com" target="_blank">欧美精品在线一区</a>| <a href="http://17cao8.com" target="_blank">牛牛影视久久网</a>| <a href="http://5418002.com" target="_blank">欧美亚洲自偷自偷</a>| <a href="http://4009935888.com" target="_blank">亚洲专区欧美专区</a>| <a href="http://292q.com" target="_blank">一区二区免费看</a>| <a href="http://2323nn.com" target="_blank">最新成人av在线</a>| <a href="http://ktokt.com" target="_blank">亚洲电影成人</a>| <a href="http://tristooges.com" target="_blank">欧美成人视屏</a>| <a href="http://133868.com" target="_blank">欧美 日韩 国产一区二区在线视频</a>| <a href="http://www387333.com" target="_blank">欧美一区二区三区另类 </a>| <a href="http://0855d.com" target="_blank">国产日韩欧美视频</a>| <a href="http://www-188444.com" target="_blank">亚洲直播在线一区</a>| <a href="http://www-42189.com" target="_blank">欧美成人精品在线观看</a>| <a href="http://www-964664.com" target="_blank">欧美在线资源</a>| <a href="http://2424ddd.com" target="_blank">欧美专区日韩视频</a>| <a href="http://www901aaa.com" target="_blank">欧美一区二区视频在线观看2020</a>| <a href="http://677cn.com" target="_blank">亚洲一区二区三区四区中文</a>| <a href="http://22400com.com" target="_blank">日韩午夜高潮</a>| <a href="http://www44448.com" target="_blank">一本色道久久综合一区</a>| <a href="http://cm1-100.com" target="_blank">99re热精品</a>| <a href="http://9928k.com" target="_blank">亚洲一区999</a>| <a href="http://lexueit.com" target="_blank">亚洲免费伊人电影在线观看av</a>| <a href="http://261262.com" target="_blank">亚洲自拍都市欧美小说</a>| <a href="http://1313js.com" target="_blank">亚洲一区二区3</a>| <a href="http://xiaocao-av.com" target="_blank">亚洲欧美日韩综合一区</a>| <a href="http://lorgou.com" target="_blank">欧美一区二区三区在线</a>| <a href="http://dtwave-ind.com" target="_blank">欧美一区二区三区免费视</a>| <a href="http://www22336.com" target="_blank">亚洲欧美在线观看</a>| <a href="http://2jjzz.com" target="_blank">久久国产欧美日韩精品</a>| <a href="http://5566kan.com" target="_blank">久久网站热最新地址</a>| <a href="http://www497171a.com" target="_blank">浪潮色综合久久天堂</a>| <a href="http://xtgoldsky.com" target="_blank">免费精品视频</a>| <a href="http://bizssr.com" target="_blank">亚洲激情专区</a>| <a href="http://aqd1t.com" target="_blank">一区二区久久久久</a>| <a href="http://xxxx43.com" target="_blank">久久不见久久见免费视频1</a>| <a href="http://nnnn34.com" target="_blank">久久久精品欧美丰满</a>| <a href="http://www55avav.com" target="_blank">久久综合狠狠综合久久综青草 </a>| <a href="http://www333444123.com" target="_blank">亚洲黄色影院</a>| <a href="http://4hup23.com" target="_blank">一区二区欧美在线观看</a>| <a href="http://pron12.com" target="_blank">午夜精品免费在线</a>| <a href="http://788997.com" target="_blank">久久av一区</a>| <a href="http://9113c.com" target="_blank">欧美国内亚洲</a>| <a href="http://csmdjs.com" target="_blank">国产精品亚洲综合</a>| <a href="http://kk8586.com" target="_blank">在线成人h网</a>| <a href="http://sh-hxjdkj.com" target="_blank">夜色激情一区二区</a>| <a href="http://6677ch.com" target="_blank">欧美一区深夜视频</a>| <a href="http://www-5013.com" target="_blank">欧美丰满少妇xxxbbb</a>| <a href="http://www89999.com" target="_blank">夜夜嗨一区二区三区</a>| <a href="http://57sa.com" target="_blank">欧美一区二区三区精品</a>| <a href="http://mm-777.com" target="_blank">欧美成人国产</a>| <a href="http://042225.com" target="_blank">国产精品美女诱惑</a>| <a href="http://juegold.com" target="_blank">亚洲国产成人久久综合一区</a>| <a href="http://yeepey.com" target="_blank">中日韩男男gay无套</a>| <a href="http://playav111.com" target="_blank">久久久久这里只有精品</a>| <a href="http://92y5.com" target="_blank">亚洲日韩欧美一区二区在线</a>| <a href="http://34ak.com" target="_blank">亚洲无线观看</a>| <a href="http://9113c.com" target="_blank">免费观看成人www动漫视频</a>| <a href="http://xpj493.com" target="_blank">欧美午夜a级限制福利片</a>| <a href="http://gzw33.com" target="_blank">欲色影视综合吧</a>| <a href="http://77xyc77.com" target="_blank">亚洲乱码一区二区</a>| <a href="http://340996.com" target="_blank">亚洲精品视频在线观看免费</a>| <a href="http://tareandshare.com" target="_blank">99热精品在线观看</a>| <a href="http://www-444454.com" target="_blank">久久久久九九九九</a>| <a href="http://k37b.com" target="_blank">亚洲美女黄网</a>| <a href="http://qimao360.com" target="_blank">久久综合图片</a>| <a href="http://777777hb.com" target="_blank">国产日韩精品一区二区三区在线</a>| <a href="http://www134rr.com" target="_blank">亚洲国产高清aⅴ视频</a>| <a href="http://am3757.com" target="_blank">欧美亚洲日本一区</a>| <a href="http://www227ee.com" target="_blank">亚洲人体1000</a>| <a href="http://www666se.com" target="_blank">久久亚洲一区</a>| <a href="http://188267.com" target="_blank">国产三级欧美三级</a>| <a href="http://989766.com" target="_blank">亚洲午夜精品在线</a>| <a href="http://jqnyc.com" target="_blank">亚洲国产精品视频一区</a>| <a href="http://456985.com" target="_blank">久久精品免费电影</a>| <script> (function(){ var bp = document.createElement('script'); var curProtocol = window.location.protocol.split(':')[0]; if (curProtocol === 'https') { bp.src = 'https://zz.bdstatic.com/linksubmit/push.js'; } else { bp.src = 'http://push.zhanzhang.baidu.com/push.js'; } var s = document.getElementsByTagName("script")[0]; s.parentNode.insertBefore(bp, s); })(); </script> </body>