锘??xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:trackback="http://madskills.com/public/xml/rss/module/trackback/" xmlns:wfw="http://wellformedweb.org/CommentAPI/" xmlns:slash="http://purl.org/rss/1.0/modules/slash/"><channel><title>&#20037;&#20037;&#20037;&#20037;&#20037;&#20037;&#20037;&#20037;&#20037;&#20037;&#20037;&#20037;&#19968;&#21306;&#44;&#20813;&#36153;&#35270;&#39057;&#19968;&#21306;&#20108;&#21306;&#19977;&#21306;&#22312;&#32447;&#35266;&#30475;&#44;&#22312;&#32447;&#27431;&#32654;&#24433;&#38498;</title><link>http://www.shnenglu.com/beautykingdom/category/9451.html</link><description /><language>zh-cn</language><lastBuildDate>Mon, 02 Feb 2009 16:47:06 GMT</lastBuildDate><pubDate>Mon, 02 Feb 2009 16:47:06 GMT</pubDate><ttl>60</ttl><item><title>NUMA涓庤嫳鐗瑰皵涓嬩竴浠eon澶勭悊鍣ㄥ涔?fàn)蹇?jī)寰?lt;杞?gt;</title><link>http://www.shnenglu.com/beautykingdom/archive/2009/02/02/72826.html</link><dc:creator>chatler</dc:creator><author>chatler</author><pubDate>Mon, 02 Feb 2009 14:18:00 GMT</pubDate><guid>http://www.shnenglu.com/beautykingdom/archive/2009/02/02/72826.html</guid><wfw:comment>http://www.shnenglu.com/beautykingdom/comments/72826.html</wfw:comment><comments>http://www.shnenglu.com/beautykingdom/archive/2009/02/02/72826.html#Feedback</comments><slash:comments>0</slash:comments><wfw:commentRss>http://www.shnenglu.com/beautykingdom/comments/commentRss/72826.html</wfw:commentRss><trackback:ping>http://www.shnenglu.com/beautykingdom/services/trackbacks/72826.html</trackback:ping><description><![CDATA[<p>&nbsp; &nbsp;&nbsp; &nbsp;浣滀負(fù)涓�鍚嶅伐紼嬪笀錛屼竴鍚嶅仛鎶�鏈殑宸ョ▼甯堬紝NUMA涔熸槸鎴戠殑榪戞湡宸ヤ綔閲嶇偣涔嬩竴銆傚湪宸ヤ綔鏃墮棿錛屽湪鑼朵綑楗悗錛屼篃鐪嬩簡(jiǎn)浜汵UMA鐨勮祫鏂欙紝瀛︿範(fàn)浜?jiǎn)鑻辩壒灏斾笅涓�浠eon澶勭悊鍣ㄣ�傝繖閲屽氨鏄垜鐨勪竴鐐瑰皬緇擄紝涓�鐐瑰績(jī)寰楋紝鍜屾劅鍏磋叮鐨勬湅鍙嬪垎浜垎浜�?/p>
<p>鍥犳椂闂存湁闄愶紝姣忔灝辨�葷粨涓�閮ㄥ垎錛屾參鎱㈤亾鏉ワ紝嬈茬煡璇︽儏錛屼笅鍥炲垎鏅擄紒 <img alt=:-) src="http://softwareblogs-zho.intel.com/wordpress/wp-includes/images/smilies/icon_smile.gif"> </p>
<p>&nbsp;<span><span>涓�銆佹榪?/span></span></p>
<p><span><span>浠庣郴緇熸灦鏋勬潵璇達(dá)紝鐩墠鐨勪富嫻佷紒涓氭湇鍔″櫒鍩烘湰鍙互鍒嗕負(fù)涓夌被錛?/span><span><span>SMP (Symmetric Multi Processing</span></span><span>錛屽縐板澶勭悊鏋舵瀯</span><span><span>)</span></span><span>錛?/span><span><span>NUMA (Non-Uniform Memory Access</span></span><span>錛岄潪涓�鑷村瓨鍌ㄨ闂灦鏋?/span><span><span>)</span></span><span>錛屽拰</span><span><span>MPP (Massive Parallel Processing</span></span><span>錛屾搗閲忓茍琛屽鐞嗘灦鏋?/span><span><span>)</span></span><span>銆備笁縐嶆灦鏋勫悇鏈夌壒鐐癸紝鏈枃灝嗛噸鐐硅亰鑱?/span><span><span>NUMA</span></span><span>銆?/span></span></p>
<p><span><span><span><span>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; </span></span></span><span>涓轟簡(jiǎn)浜?jiǎn)瑙?/span><span><span>NUMA</span></span><span>錛屾垜榪欓噷灝變粙緇嶄竴涓?/span><span><span>NUMA</span></span><span>涓庡叾浠栦袱縐?/span><span><span>Non-NUMA</span></span><span>鐨勪富瑕佸尯鍒�?/span></span></p>
<p><span><span><span>1</span></span><span>錛?/span><span><span>SMP</span></span><span>錛?/span><span><span>Symmetric Multi Processing</span></span><span>錛?/span></span></p>
<p><span><span><span><span>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; </span></span><span>SMP</span></span><span>鏄潪甯稿父瑙佺殑涓�縐嶆灦鏋勩�傚湪</span><span><span>SMP</span></span><span>妯″紡涓嬶紝澶氫釜澶勭悊鍣ㄥ潎瀵圭О鐨勮繛鎺ュ湪緋葷粺鍐呭瓨涓婏紝鎵�鏈夊鐞嗗櫒閮戒互騫崇瓑鐨勪唬浠瘋闂郴緇熷唴瀛樸�傚畠鐨勪紭鐐規(guī)槸瀵瑰唴瀛樼殑璁塊棶鏄鉤絳夈�佷竴鑷寸殑錛涚己鐐規(guī)槸鍥犱負(fù)澶у閮芥槸涓�鑷寸殑錛屽湪浼犵粺鐨?/span><span><span> SMP </span></span><span>緋葷粺涓紝鎵�鏈夊鐞嗗櫒閮藉叡浜郴緇熸�葷嚎錛屽洜姝ゅ綋澶勭悊鍣ㄧ殑鏁扮洰澧炲鏃訛紝緋葷粺鎬葷嚎鐨勭珵浜夊啿紿佽繀閫熷姞澶э紝緋葷粺鎬葷嚎鎴愪負(fù)浜?jiǎn)鎬ц兘鐡墮錛屾墍浠ョ洰鍓?/span><span><span> SMP </span></span><span>緋葷粺鐨勫鐞嗗櫒鏁扮洰涓�鑸彧鏈夋暟鍗佷釜錛屽彲鎵╁睍鎬у彈鍒板緢澶ч檺鍒躲�?/span></span></p>
<p><span><span><span>2</span></span><span>錛?/span><span><span>MPP (Massive Parallel Processing)</span></span></span></p>
<p><span><span><span><span>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; </span>MPP</span></span><span>鍒欐槸閫昏緫涓婂皢鏁翠釜緋葷粺鍒掑垎涓哄涓妭鐐癸紝姣忎釜鑺傜偣鐨勫鐞嗗櫒鍙彲浠ヨ闂湰韜殑鏈湴璧勬簮錛屾槸瀹屽叏鏃犲叡浜殑鏋舵瀯銆傝妭鐐逛箣闂寸殑鏁版嵁浜ゆ崲闇�瑕佽蔣浠跺疄鏂姐�傚畠鐨勪紭鐐規(guī)槸鍙墿灞曟�ч潪甯稿ソ錛涚己鐐規(guī)槸褰兼鏁版嵁浜ゆ崲鍥伴毦錛岄渶瑕佹帶鍒惰蔣浠剁殑澶ч噺宸ヤ綔鏉ュ疄鐜伴�氳浠ュ強(qiáng)浠誨姟鐨勫垎閰嶃�佽皟搴︼紝瀵逛簬涓�鑸殑浼佷笟搴旂敤鑰岃█榪囦簬澶嶆潅錛屾晥鐜囦笉楂樸�?/span></span></p>
<p><span><span><span>3</span></span><span>錛?/span><span><span>NUMA</span></span><span>錛?/span><span><span>Non-Uniform Memory Access</span></span><span>錛?/span></span></p>
<p><span><span><span><span>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; </span>NUMA</span></span><span>鏋舵瀯鍒欏湪鏌愮鎰忎箟涓婃槸緇煎悎浜?/span><span><span>SMP</span></span><span>鍜?/span><span><span>MPP</span></span><span>鐨勭壒鐐癸細(xì)閫昏緫涓婃暣涓郴緇熶篃鏄垎涓哄涓妭鐐癸紝姣忎釜鑺傜偣鍙互璁塊棶鏈湴鍐呭瓨璧勬簮錛屼篃鍙互璁塊棶榪滅▼鍐呭瓨璧勬簮錛屼絾璁塊棶鏈湴鍐呭瓨璧勬簮榪滆繙蹇簬榪滅▼鍐呭瓨璧勬簮銆傚畠鐨勪紭鐐規(guī)槸鍏奸【浜?/span><span><span>SMP</span></span><span>鍜?/span><span><span>MPP</span></span><span>鐨勭壒鐐?/span><span><span>, </span></span><span>鏄撲簬綆＄悊錛屽彲鎵╁厖鎬уソ錛涚己鐐規(guī)槸璁塊棶榪滅▼鍐呭瓨璧勬簮鐨勬墍闇�鏃墮棿闈炲父鐨勫ぇ銆?/span></span></p>
<p><span><span><span><span>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; </span></span></span><span>鍦ㄥ疄闄呯郴緇熶腑浣跨敤姣旇緝騫跨殑鏄?/span><span><span>SMP</span></span><span>鍜?/span><span><span>NUMA</span></span><span>鏋舵瀯銆傚儚浼犵粺鐨勮嫳鐗瑰皵</span><span><span>IA</span></span><span>鏋舵瀯灝辨槸</span><span><span>SMP</span></span><span>錛岃�屽緢澶氬ぇ鍨嬫満閲囩敤浜?/span><span><span>NUMA</span></span><span>鏋舵瀯銆?/span></span></p>
<p><span><span><span><span>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; </span></span></span><span>鐜板湪宸茬粡榪涘叆浜?jiǎn)澶氭牳鏃朵唬锛岄殢鐫�鏍告暟鐨勮秺鏉ヨ秺澶氾紝瀵逛簬鍐呭瓨鍚炲悙閲忓拰寤惰繜鏈変簡(jiǎn)鏇撮珮鐨勮姹傘�傛鏄�冭檻鍒拌繖縐嶉渶姹傦紝</span><span><span>NUMA</span></span><span>鏋舵瀯鍑虹幇鍦ㄤ簡(jiǎn)鏈�鏂扮殑鑻辯壒灝?/span><span><span>涓嬩竴浠eon澶勭悊鍣?/span></span><span>涓�?br><br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &nbsp; 鍋氫負(fù)鑻辯壒灝?dāng)涓嬩竴浠ｇ殑45nm Xeon澶勭悊鍣紝 瀹冧細(xì)鎴愪負(fù)鏈潵鑻辯壒灝?dāng)浠庡彴寮忔満銆佺瑪璁版湰鍒版湇鍔″櫒鍏ㄧ嚎浜у搧鐨勪富嫻佸鐞嗗櫒銆?姣旇緝鍓嶄竴浠ｉ叿鐫垮鐞嗗櫒騫沖彴錛屽畠鐨勫鉤鍙板湪瀵逛互鍓嶇殑緋葷粺鏋舵瀯鍜屽唴瀛樺眰嬈′綋緋昏繘琛屼簡(jiǎn)閲嶅ぇ鏀瑰彉鐨勫悓鏃訛紝瀵瑰井鏋舵瀯涔熻繘琛屼簡(jiǎn)鍏ㄦ柟浣嶇殑緇嗗寲錛?涓昏鏀硅繘琛ㄧ幇鍦ㄤ互涓嬬殑鐗規(guī)�э細(xì)<br>飩?&gt; 鏂扮殑鏍稿績(jī)鏋舵瀯錛屾渶澶у彲鎵╁睍鍒版瘡涓帴鍙?涓牳蹇?br>飩?&gt; 鍚屾澶氱嚎紼?SMT) 鎶�鏈渶澶у厑璁告瘡涓鐞嗗櫒鍙互榪愯8涓嚎紼?br>飩?&gt; 鏈�鏂扮殑鐐瑰埌鐐圭洿榪炴灦鏋勶細(xì)Intel&#174; QuickPath Interconnect (Intel&#174; QPI)鎶�鏈?br>飩?&gt; Intel&#174; QuickPath 闆嗘垚鍐呭瓨鎺у埗鍣紙IMC錛夛紝DDR3鎺ュ彛<br>飩?&gt; 寰灦鏋勫姛鑳界殑鏀硅繘錛屽寘鎷寮虹殑SSE4.2鎸囦護(hù)闆嗭紝鏀硅繘鐨勯攣瀹氭敮鎸侊紝寰幆嫻佸拰鍒嗘敮棰勬祴絳夌壒鎬?br>飩?&gt; 鏇村ソ鐨勮妭鑳界壒鎬?/p>
<p>涓嬮潰璇︾粏浠嬬粛涓�涓嬩笅涓�浠eon澶勭悊鍣ㄥ洓澶т富瑕佹妧鏈?</p>
<p>飩?&gt; Intel&#174; QuickPath Interconnect (Intel&#174; QPI)鎶�鏈?br>浣跨敤QPI鏋舵瀯浠ｆ浛浜?jiǎn)鍘熸潵鐨凢SB鏋舵瀯錛孮PI鏄熀浜庢暟鎹寘浼犺緭錛岄珮甯﹀浣庡歡榪熺殑鐐瑰埌鐐逛紶杈撴妧鏈紝閫熷害鍙互杈懼埌6.4GT/s錛屽鍙屽悜浼犺緭鐨凲PI鎬葷嚎榪炴帴鏉ヨ鐞嗚鏈�澶у�煎彲浠ヨ揪鍒?5.6GB/s鐨勬暟鎹紶杈擄紝榪滆繙楂樹簬鍘熸潵鍩轟簬FSB鏋舵瀯鐨勬暟鎹甫瀹姐�?/p>
<p>飩?&gt; Intel&#174; QuickPath 闆嗘垚鍐呭瓨鎺у埗鍣紙IMC錛?br>鍦ㄦ瘡涓�涓猻ocket涓婇泦鎴愪簡(jiǎn)鐙珛鐨凞DR3鍐呭瓨鎺у埗鍣紙IMC錛変緵鎺ュ彛璁塊棶鍐呭瓨錛岃緝涔嬮潪IMC鐨勫鉤鍙幫紝澶уぇ鎻愰珮浜?jiǎn)甯﹀锛堜娇鐢―DR3錛?333鍙互杈懼埌32GB/s鐨勫嘲鍊煎甫瀹斤紝杈冧箣浠ュ墠鐨勫鉤鍙板叿鏈夊洓鍒板叚鍊嶇殑甯﹀鎻愬崌錛夛紝鏄捐憲鍦伴檷浣庝簡(jiǎn)鍐呭瓨寤惰繜錛屼粠鑰屾彁鍗囦簡(jiǎn)鎬ц兘錛屼負(fù)姣忎釜CPU鎻愪緵浜?jiǎn)璁块棶鏈湴鍐呭瓨璧勬簮鐨勫揩閫熼�氶亾銆備笌鍓嶄竴浠ｅ鉤鍙頒笉鍚岀殑鏄紝鍐呭瓨璁塊棶閲囩敤NUMA鏋舵瀯錛屽浜嶯UMA-aware鐨勫簲鐢ㄦ潵璇村彲浠ュ緱鍒版洿澶х殑鎬ц兘鎻愬崌銆侱DR3鐨処MC鏈�澶ф敮鎸佸埌姣忎釜CPU鎺ュ彛96GB鐨凞DR3鍐呭瓨瀹歸噺錛屽皢鏉ユ渶澶у閲忓彲浠ヨ揪鍒?44GB錛屼負(fù)楂樼鐨勪紒涓氳繍綆楁彁渚涗簡(jiǎn)寮烘湁鍔涚殑鍐呭瓨鏀寔銆?/p>
<p><strong>鍚屽織浠紝NUMA鍦ㄨ繖灝遍棯浜櫥鍦轟簡(jiǎn)錛?</strong></p>
<p>飩?&gt; 鏀硅繘鐨勭數(shù)婧愮鐞?br>闆嗘垚鍦ㄨ姱鐗囦笂鐨勭數(shù)婧愮鐞嗕嬌寰楄兘鑰楃殑鎺у埗鏇村姞楂樻晥銆?/p>
<p>飩?&gt; 鍚屾澶氱嚎紼嬫妧鏈紙SMT錛?br>鍚屾澶氱嚎紼嬫妧鏈嬌寰楁瘡涓牳蹇?jī)鍙互鍚屾椂鎵ц?涓嚎紼嬶紝鎵�浠ュ浜?鏍哥殑CPU鏉ヨ錛屽氨鍙互鍦ㄦ瘡涓鐞嗗櫒鑺墖涓婅揪鍒版渶澶?涓�昏緫澶勭悊鍣ㄣ�?/p>
<p><span><span>鍓嶉潰浠嬬粛浜?jiǎn)NUMA鐨勫緢鐗涚殑鏋舵瀯錛岄偅鐩墠緋葷粺灞傞潰涓婏紝杞歡瀵筃UMA鐨勬敮鎸佹�庝箞鏍峰憿錛熻瑙佹湰鏂囷細(xì)NUMA鏋舵瀯杞歡鏀寔鏍?/span></span></p>
<p><span><span>瀵逛簬</span><span>NUMA</span><span>鏋舵瀯鑰岃█錛岀粡榪囦簡(jiǎn)鍑犲崄騫寸殑鍙戝睍錛岀洰鍓嶇殑杞歡鏀寔鏍堝凡緇忛潪甯稿畬澶囷紝浠庡簳灞傜殑鎿嶄綔緋葷粺錛屽埌涔嬩笂鐨勬暟鎹簱銆佸簲鐢ㄦ湇鍔″櫒錛屽熀鏈墍鏈変富嫻佺殑浜у搧鍧囦互涓?/span><span>NUMA</span><span>鎻愪緵浜?jiǎn)鍏呭垎鐨勬敮鎸併�?/span></span></p>
<p><span><strong><em><span>鎿嶄綔緋葷粺錛?/span></em></strong><strong><em><span>Operating System</span></em></strong><strong><em><span>錛?/span></em></strong><strong><em></em></strong></span></p>
<p><span><span>鐩墠錛?/span><span>Windows Server 2003 </span><span>鍜?/span><span>Windows XP 64-bit Edition, Windows XP</span><span>絳夐兘鏄?/span><span>NUMA aware</span><span>鐨勶紝鑰?/span><span>Windows Vista</span><span>鍒欐湁浜?jiǎn)瀵?/span><span>Numa</span><span>璋冨害鐨勬敮鎸併�傛墍鏈変嬌鐢?/span><span>2.6</span><span>鐗堟湰浠ヤ笂</span><span>kernel</span><span>鐨?/span><span>Linux</span><span>鎿嶄綔緋葷粺閮借兘澶熸敮鎸?/span><span>NUMA</span><span>銆傝�?/span><span>Solaris</span><span>錛?/span><span>HP-Unix</span><span>絳?/span><span>UNIX</span><span>鎿嶄綔緋葷粺涔熸槸鍏呭垎鏀寔</span><span>NUMA</span><span>鏋舵瀯鐨勩�?/span></span></p>
<p><span><strong><em><span>鏁版嵁搴擄紙</span></em></strong><strong><em><span>Database</span></em></strong><strong><em><span>錛?/span></em></strong></span></p>
<p><span><span>瀵逛簬鏁版嵁搴撲駭鍝佹潵璇達(dá)紝</span><span>Oracle</span><span>浠?/span><span>8i</span><span>寮�濮嬫敮鎸?/span><span>NUMA,</span><span>鑰屼箣鍚庣殑</span><span>Oracle9i</span><span>錛?/span><span>Oracle10g</span><span>錛?/span><span>Oracle11g</span><span>閮借兘澶熸敮鎸?/span><span>NUMA</span><span>銆?/span><span>SQL Server 2005 </span><span>鍜?/span><span>SQL Server 2008</span><span>鍧囨湁鏁堢殑鎻愪緵浜?jiǎn)瀵?/span><span>NUMA</span><span>鐨勬敮鎸併�?/span></span></p>
<p><span><strong><em><span>涓棿浠舵湇鍔″櫒錛?/span></em></strong><strong><em><span>Middleware</span></em></strong><strong><em><span>錛?/span></em></strong><strong><em></em></strong></span></p>
<p><span><span>鐩墠涓氱晫鍏稿瀷鐨勫彈鎺х▼搴忎富瑕佹槸</span><span>Java</span><span>搴旂敤鍜?/span><span>.Net</span><span>搴旂敤銆傜敱浜庡唴瀛樺垎閰嶏紝綰跨▼璋冨害瀵逛簬搴旂敤鑰岃█鏄�忔槑鐨勶紝瀹屽叏鏄敱铏氭嫙鏈烘潵澶勭悊銆傚洜姝ゅ畠浠湪</span><span>NUMA</span><span>鐜涓嬬殑鎬ц兘琛ㄧ幇涓昏鍙栧喅浜庤櫄鎷熸満鐨勫疄鐜版槸鍚﹁兘鍏呭垎鍒╃敤鍒板簳灞傛搷浣滅郴緇熷</span><span>NUMA</span><span>鐨勬敮鎸併�?/span></span></p>
<p><span><span><span>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; </span></span><span>緇間笂鎵�榪幫紝鐩墠鐨勮蔣浠舵爤瀵?/span><span>NUMA</span><span>鏋舵瀯鍧囧凡緇忎綔浜?jiǎn)鍏呭垎鐨勬敮鎸併�傞偅涔堝簲鐢ㄨ蔣浠跺浣曟敮鎸?/span><span>NUMA</span><span>鏋舵瀯鍛紵璇瘋涓嬮潰绔犺妭鐨勮榪般�?/span></span></p>
<p><span><span>鍦ㄤ紶緇?/span><span>SMP</span><span>緋葷粺涓婏紝鎵�鏈?/span><span>CPU</span><span>閮戒互鍚屾牱鐨勬柟寮忛�氳繃涓�涓叡浜唴瀛樻帶鍒跺櫒鏉ヨ闂唴瀛橈紝鍚?/span><span>CPU</span><span>涔嬮棿涔熸槸閫氳繃瀹冩潵榪涜浜ゆ祦錛屾墍浠ュ緢瀹規(guī)槗閫犳垚鎷ュ牭銆傝�屼竴涓唴瀛樻帶鍒跺櫒鎵�鑳藉綆＄悊鐨勫唴瀛樻暟閲忎篃鏄潪甯告湁闄愮殑銆傛澶栵紝閫氳繃鍞竴鐨?/span><span>hub</span><span>璁塊棶鍐呭瓨閫犳垚鐨勫歡榪熶篃鏄潪甯擱珮鐨勩�?/span></span></p>
<p><span>&nbsp;</span><span><span>鍦?/span><span>NUMA</span><span>緇撴瀯涓嬶紝姣忎釜璁＄畻鏈轟笉鍐嶅彧鏈夊敮涓�鐨勫唴瀛樻帶鍒跺櫒錛岃�屾槸鎶婃暣涓郴緇熷垎鎴愬涓妭鐐廣�傛瘡涓妭鐐瑰垎鍒湁鑷繁鐨勫鐞嗗櫒鍜屽唴瀛樸�傜郴緇熶腑鎵�鏈夌殑鑺傜偣閮介�氳繃鍏ㄤ簰鑱旂殑鏂瑰紡榪炴帴銆傛墍浠ワ紝姣忓綋鍦ㄧ郴緇熶腑澧炲姞鏂扮殑鑺傜偣錛岀郴緇熸墍鑳藉鏀寔鐨勫唴瀛樺拰甯﹀閮戒細(xì)澧炲姞錛屽叿鏈夐潪甯稿ソ鐨勬墿灞曟�с�?/span></span></p>
<p><strong><em><span>&nbsp;</span></em></strong><span><span>涓嬮潰灝辮璁?strong>NUMA鐨勫唴瀛樼粍緇?/strong></span></span></p>
<p><strong><em><span>&nbsp;</span></em></strong><span><span>鍦?/span><span>NUMA</span><span>緋葷粺涓紝姣忎釜</span><span>CPU</span><span>鍙互璁塊棶涓ょ鍐呭瓨錛氭湰鍦板唴瀛橈紙</span><span>Local Memory</span><span>錛夊拰榪滅鍐呭瓨錛?/span><span>Remote Memory</span><span>錛夈�傚拰</span><span>CPU</span><span>鍦ㄥ悓涓�涓妭鐐圭殑鍐呭瓨縐頒負(fù)鏈湴鍐呭瓨錛岃闂歡榪熼潪甯鎬綆銆傚拰</span><span>CPU</span><span>鍦ㄤ笉鍚岃妭鐐逛笂鐨勫唴瀛樺彨鍋氳繙绔唴瀛橈紝</span><span>CPU</span><span>闇�瑕侀�氳繃鑺傜偣浜掕仈鏂瑰紡璁塊棶錛屾墍浠ヨ闂歡榪熻姣旇闂湰鍦板唴瀛橀暱銆?/span></span></p>
<p><span>&nbsp;</span><span><span>浠庤蔣浠剁殑瑙掑害鏉ョ湅錛岃繙绔唴瀛樺拰鏈湴鍐呭瓨鏄互鍚屾牱鐨勬柟寮忚闂殑銆傜悊璁轟笂璁詫紝</span><span>NUMA</span><span>緋葷粺鍙互琚蔣浠惰涓轟笌</span><span>SMP</span><span>鍚屾牱鐨勭郴緇燂紝涓嶅尯鍒嗘湰鍦板拰榪滅鍐呭瓨銆備絾鏄鏋滆拷姹傛洿濂界殑鎬ц兘錛岃繖涓尯鍒繕鏄渶瑕佽鑰冭檻鐨勩�?/span></span></p>
<p><span>&nbsp;</span><span><span>緇忓疄楠岋紝瀵逛簬甯歌鐨勫唴瀛樻搷浣滐紝濡傛竻絀猴紙</span><span>Memset</span><span>錛夛紝鍧楀鍒訛紙</span><span>Memcpy</span><span>錛夛紝嫻佽鍐欙紙</span><span>Stream</span><span>錛夛紝鎸囬拡榪芥函錛?/span><span>Pointer Chase</span><span>錛夌瓑鎿嶄綔鏉ヨ錛屾湰鍦板唴瀛樼殑璁塊棶閫熷害瑕佽繙榪滀紭浜庤繙绔唴瀛樸�?/span></span></p>
<p><span>&nbsp;</span><span><span>鐢變簬</span><span> NUMA </span><span>鍚屾椂浣跨敤鏈湴鍐呭瓨鍜岃繙绔唴瀛橈紝鍥犳錛岃闂煇浜涘唴瀛樺尯鍩熺殑鏃墮棿浼?xì)姣旇畨K棶鍏朵粬鍐呭瓨鍖哄煙鐨勮闀褲�傛湰鍦板唴瀛樺拰榪滅鍐呭瓨閫氬父鐢ㄤ簬寮曠敤褰撳墠姝ｅ湪榪愯鐨勭嚎紼嬨�傛湰鍦板唴瀛樻槸鎸囦笌褰撳墠姝ｅ湪榪愯綰跨▼鐨?/span><span> CPU </span><span>浣嶄簬鍚屼竴鑺傜偣涓婄殑鍐呭瓨銆備換浣曚笉灞炰簬褰撳墠姝ｅ湪榪愯鐨勭嚎紼嬫墍鍦ㄧ殑鑺傜偣鐨勫唴瀛樺潎涓鴻繙绔唴瀛樸�傝闂繙绔唴瀛樼殑寮�閿�涓庤闂湰鍦板唴瀛樼殑寮�閿�姣旂巼縐頒負(fù)</span><span> NUMA </span><span>姣旂巼銆傚鏋?/span><span> NUMA </span><span>姣旂巼涓?/span><span> 1</span><span>錛屽垯瀹冩槸瀵圭О澶氬鐞?/span><span> (SMP)</span><span>銆傛瘮鐜囪秺楂橈紝璁塊棶鍏朵粬鑺傜偣鍐呭瓨鐨勫紑閿�灝辮秺澶с�備笉鏀寔</span><span> NUMA </span><span>鐨?/span><span> </span><span>搴旂敤紼嬪簭鏈夋椂鍦?/span><span> NUMA </span><span>紜歡涓婄殑鎵ц鏁堟灉闈炲父宸�?/span></span></p>
<p><span>&nbsp;</span><span><span>鐢變簬璁塊棶鏈湴鍐呭瓨鍜岃繙绔唴瀛樼殑寮�閿�鏄湁鍖哄埆鐨勶紝鎵�浠ュ湪</span><span>NUMA</span><span>妯″紡涓嬶紝濡傛灉姣忎釜綰跨▼鏇村鐨勬槸璁塊棶鏈湴鍐呭瓨錛岄偅涔堟�ц兘鐩告瘮鑰岃█浼?xì)鏈変竴瀹氭彁鍗囥�?/span></span></p>
<p>澶氳阿鍚勪綅鐨勫弬涓庡拰鏀寔錛岃鎴戞洿鏈夊姩鍔涘幓鎶婅繖涓郴鍒楀啓濂姐�傚墠闈㈡湁鍚屽闂搗浜?jiǎn)QPI錛屾垜榪欓噷灝辮緇嗚В閲婁竴涓嬶紝鑰孮PI涔熸槸涓嬩竴浠eon澶勭悊鍣ㄧ殑鐗規(guī)�т箣涓�銆?/p>
<p>QPI鍏ㄧО<span>Intel&#174; QuickPath Interconnect錛屾槸鐩存帴榪炴帴鍚屼竴鍙版満鍣ㄧ殑涓嶅悓CPU涔嬮棿鐨勪紶杈撻�氶亾錛屼嬌寰楀悇涓牳錛圕ORE錛変箣闂寸殑鏁版嵁浼犺緭鏇村揩錛氬鏋滄暟鎹湪cache閲岋紝灝卞彲浠ョ洿鎺ョ敤QPI鏉ヤ紶杈擄紝鑰屼笉鐢ㄥ啀璁塊棶鍐呭瓨浜?jiǎn)銆?/span></p>
<p><span><a ><img class="alignnone size-medium wp-image-866" height=260 alt="" src="http://softwareblogs-zho.intel.com/wordpress/wp-content/uploads/2008/12/untitled-300x260.jpg" width=300></a></span></p>
<p>涓嬩竴浠eon澶勭悊鍣ㄤ嬌鐢≦PI鏋舵瀯浠ｆ浛浜?jiǎn)鍘熸潵鐨凢SB鏋舵瀯錛孮PI鏄熀浜庢暟鎹寘浼犺緭錛岄珮甯﹀浣庡歡榪熺殑鐐瑰埌鐐逛紶杈撴妧鏈紝閫熷害鍙互杈懼埌6.4GT/s,榪滆繙楂樹簬鍘熸潵鍩轟簬FSB鏋舵瀯鐨勬暟鎹甫瀹姐�傚綋鐒?鍏蜂綋騫沖彴鐨勫疄鐜頒腑QPI榪炴帴鏁扮洰鍙互鏍規(guī)嵁鐩爣甯?jìng)鍦哄拰绯痪l熷鏉傛�ц�屾湁鎵�涓嶅悓錛岃〃鐜板嚭鏋佸ぇ鐨勭伒媧繪�у拰鎵╁睍鎬с�?nbsp;</p>
<p><span>鍙堟湁鍚屽鍙兘瑕侀棶錛岄偅鍚屼竴涓狢PU鍐呯殑涓嶅悓鐨勬牳鎬庝箞浜ゆ崲鏁版嵁鍛紵榪欏氨鏇寸畝鍗曚簡(jiǎn)銆備笅涓�浠eon澶勭悊鍣ㄧ殑涓嶅悓鏍告槸瀛樺湪cache鍏變韓鐨勶紝榪欐牱濡傛灉鏁版嵁鍦╟ache閲岋紝閭ｅ氨鐩存帴鍏變韓浜?jiǎn)锛屼笉鐢ㄥ啀鍒板唴瀛橀噷鎵惧Q岀畝鍗曞惂錛屽懙鍛?/span></p>
<p>鎺ヤ笅鏉ヨ璁?span><em><strong>NUMA</strong></em></span><span><em><strong>絳栫暐</strong></em>錛屼篃灝辨槸涓轟簡(jiǎn)鏇村ソ鐨勫埄鐢∟UMA鏉ョ粰鍜變滑騫叉椿錛?/span><span></p>
<p><span>涓烘弿榪板湪</span><span>NUMA</span><span>鏋舵瀯涓嬮拡瀵瑰唴瀛樿闂殑浼樺寲錛屾垜浠彲浠ュ紩鍏?/span><span>NUMA</span><span>絳栫暐鐨勬蹇點(diǎn)�?/span><span>NUMA</span><span>絳栫暐</span><span>(NUMA Policy)</span><span>鍗蟲槸鎸囧湪澶氫釜鑺傜偣涓婂悎鐞嗙殑榪涜鍐呭瓨鍒嗛厤鐨勬満鍒躲�傚浜庝笉鍚岃蔣浠惰璁¤姹傦紝絳栫暐鐨勭洰鏍囧彲鑳戒細(xì)涓嶅悓錛氭湁涓�浜涜璁″彲鑳藉己璋冧綆寤惰繜璁塊棶錛屽彟涓�浜涘垯鍙兘鏇村姞鐪嬮噸鍐呭瓨鐨勮闂甫瀹姐�?/span></p>
<p>&nbsp;<span>瀵逛簬寮鴻皟浣庡歡榪熻闂殑璁捐錛屽熀鏈殑鍒嗛厤鏂瑰紡灝辨槸灝介噺鍦ㄧ嚎紼嬬殑鏈湴鍐呭瓨涓婁負(fù)鍏惰繘琛屽垎閰嶏紝</span><span> </span><span>騫跺敖閲忚綰跨▼淇濇寔鍦ㄨ鑺傜偣涓娿�傝繖琚О涓虹嚎紼嬬殑鑺傜偣浜插拰鎬?/span><span>(Node affinity)</span><span>銆傝繖鏍鋒棦鍏呭垎鍒╃敤浜?jiǎn)鏈湴鍐呭瓨鐨勪綆鍤g榪燂紝</span><span> </span><span>鍚屾椂涔熻兘鏈夋晥闄嶄綆鑺傜偣闂寸殑閫氫俊璐熸媴銆?/span></p>
<p>&nbsp;<span>NUMA</span><span>鏋舵瀯鐨勪竴涓紭鍔挎槸錛屽嵆渚挎槸鍦ㄦ嫢鏈夊ぇ閲?/span><span>CPU</span><span>鐨勫ぇ瑙勬ā緋葷粺涓紝鎴戜滑涔熷彲浠ヤ繚璇佸眬閮ㄥ唴瀛樿闂殑浣庡歡榪熴�傞�氬父鏉ヨ錛?/span><span>CPU</span><span>鐨勫鐞嗛�熷害鏄繙澶т簬鍐呭瓨鐨勫瓨鍙栭�熷害鐨勩�傚湪璇誨啓鍐呭瓨鏃訛紝</span><span>CPU</span><span>甯稿父闇�瑕佽姳澶ч噺鐨勬椂閽熷懆鏈熸潵絳夊緟銆傞檷浣庡唴瀛樿闂殑寤惰繜鍥犺�岃兘澶熸湁鏁堢殑鎻愬崌杞歡鎬ц兘銆?/span></p>
<p>&nbsp;<span>鍙﹀錛屼負(fù)</span><span>SMP</span><span>璁捐鐨勬搷浣滅郴緇熼�氬父浼?xì)鏈壘~撳瓨浜插拰鎬?/span><span>(Cache Affinity) </span><span>鐨勪紭鍖栨帾鏂姐�傜紦瀛樹翰鍜屾�ф満鍒跺彲浠ヨ鏁版嵁灝介噺闀挎椂闂寸殑淇濈暀鍦ㄦ煇涓�涓?/span><span>CPU</span><span>鐨勭紦瀛樹腑錛岃�屼笉鏄潵鍥炲湪澶氫釜</span><span>CPU</span><span>鐨勭紦瀛橀噷鎹㈡潵鎹㈠幓銆傛搷浣滅郴緇熼�氬父鏄�氳繃浼樺寲榪涜綰跨▼</span><span>/</span><span>榪涚▼璋冨害鏉ヤ繚璇佽繖涓�鐐癸細(xì)鍦ㄧ嚎紼嬭閲嶆柊璋冨叆鏃訛紝璋冨害鍣ㄤ細(xì)灝介噺璁╃嚎紼嬪湪涔嬪墠榪愯鐨勫悓涓�涓?/span><span>CPU</span><span>涓婅繍琛岋紝浠庤�屼繚璇佺紦瀛樺埄鐢ㄧ巼銆傝繖涓�鏈哄埗鏄劇劧鏄拰</span><span>NUMA</span><span>緋葷粺灝介噺鍒╃敤鏈湴鍐呭瓨鐨勭瓥鐣ユ槸涓�鑷寸殑錛屾湁鍒╀簬闈㈠悜</span><span>SMP</span><span>緋葷粺鐨勭▼搴忓悜</span><span>NUMA</span><span>鏋舵瀯縐繪銆?/span></p>
<p>&nbsp;<span>浣嗙紦瀛樹翰鍜屾�ф満鍒跺悓</span><span>NUMA</span><span>緋葷粺鐨勮妭鐐逛翰鍜屾�у張鏄湁鍖哄埆鐨勶細(xì)棣栧厛錛屽悓涓�涓妭鐐歸棿澶氫釜</span><span>CPU</span><span>鎴栬�呮牳鐨勭嚎紼嬭縼縐誨茍涓嶅獎(jiǎng)鍝嶈綰跨▼鐨勮妭鐐逛翰鍜屾�э紱鍏舵錛屽綋綰跨▼琚揩榪佺Щ鍒板叾浠栬妭鐐規(guī)椂錛屼粬鎵�鎷ユ湁鐨勫唴瀛樻槸涓嶄細(xì)璺熺潃榪佺Щ鐨勶紝</span><span> </span><span>浠嶇劧淇濈暀鍦ㄥ師鏉ヤ綅緗�傝繖涓椂鍊欙紝鏈湴鍐呭瓨灝卞彉鎴愪簡(jiǎn)榪滅鍐呭瓨錛屽瀹冪殑璁塊棶鏃㈡參鍙堝崰鐢ㄨ妭鐐歸�氫俊甯﹀銆傜浉瀵圭殑錛岀嚎紼嬪湪榪佺Щ涔嬪悗鑳藉浠ヨ緝?yōu)畯鐨勪唬洵h(huán)榪呴�熷緩绔嬭搗鏂扮殑緙撳瓨錛屽茍緇х畫鍦ㄦ柊</span><span>CPU</span><span>涓婁綋鐜扮紦瀛樼殑浜插拰浼樺娍銆?/span><span> </span><span>鍥犳錛?/span><span>NUMA</span><span>緋葷粺瀵逛簬鑺傜偣浜插拰鎬х殑渚濊禆鏇村ぇ銆?/span></p>
<p>&nbsp;<span>鎿嶄綔緋葷粺鐨勮皟搴﹀櫒鍚屾椂涔熶笉鑳戒粎浠呬負(fù)淇濊瘉鑺傜偣浜插拰鎬у仛浼樺寲銆傚洜涓洪�氬父鐩稿浜庨綣佽闂繙绔唴瀛樻潵璇達(dá)紝璁?/span><span>CPU</span><span>絀洪棽甯︽潵鐨勬�ц兘鎹熷け鏇村ぇ銆傚鏋滅壒瀹氬簲鐢ㄧ郴緇熺殑鎬ц兘鍙楀唴瀛樿闂殑褰卞搷榪滃ぇ浜?/span><span>CPU</span><span>鐨勫埄鐢ㄧ巼錛岃繖涓椂鍊欑▼搴忓憳鎴栬�呯鐞嗗憳鍒欏彲閲囩敤鐗瑰埆鐨?/span><span>NUMA</span><span>絳栫暐鏉ュ己璋冭妭鐐圭殑浜插拰鎬э紝浠庤�屾彁鍗囨�ц兘銆?/span></p>
<p>&nbsp;<span>鍙﹀</span><span>, </span><span>灝界澶ч儴鍒嗗簲鐢ㄤ細(xì)鍥犱負(fù)浼樺寲鍝嶅簲鏃墮棿鑰屾敹鐩婏紝榪樻湁涓�閮ㄥ垎搴旂敤鍒欏鍐呭瓨甯﹀姣旇緝鏁忔劅銆備負(fù)浜?jiǎn)鎻愬崌鍐呭瓨甯﹀锛?/span><span>NUMA</span><span>鏋舵瀯涓嬬殑澶氫釜鍐呭瓨鎺у埗鍣ㄥ彲浠ュ茍琛屼嬌鐢ㄣ�傝繖綾諱技浜?/span><span>RAID</span><span>闃靛垪閫氳繃騫惰澶勭悊紓佺洏</span><span>IO</span><span>鏉ユ彁鍗囪鍐欐�ц兘銆傞�氳繃閫傚綋鐨勮蔣浠舵垨鑰呯‖浠舵満鍒訛紝</span><span>NUMA</span><span>鏋舵瀯鍙互浣垮唴瀛樻帶鍒跺崟鍏冨湪鍚勪釜鍐呭瓨鎺у埗鍣ㄤ笂浜ゆ浛鐨勫垎閰嶅唴瀛樸�傝繖鎰忓懗鐫�鍒嗛厤寰楀埌鐨勮繛緇唴瀛橀〉闈細(xì)姘村鉤鍦板垎甯冨埌鍚勪釜鑺傜偣涓娿�傚綋搴旂敤紼嬪簭瀵瑰唴瀛樿繘琛屾祦寮忚鍐欐椂錛屽悇涓唴瀛樻帶鍒跺櫒鐨勫甫瀹藉氨鐩稿綋浜庣瘡鍔犱簡(jiǎn)銆傛鏈哄埗鑾峰緱鎬ц兘鎻愬崌鍐沖畾浜?/span><span>NUMA</span><span>鏋舵瀯鐨勫疄鐜般�傚浜庤繙绔唴瀛樿闂歡榪熶弗閲嶇殑鏋舵瀯錛岃鎻愬崌寰�寰�浼?xì)姣旇緝鏄庢樉銆傚湪涓�浜?/span><span>NUMA</span><span>緋葷粺涓紝緋葷粺紜歡鏈韓鎻愪緵浜?jiǎn)鑺傜偣浜ぞl囧垎閰嶆満鍒訛紱鑰屽湪娌℃湁紜歡鎻愪緵鑺傜偣浜ょ粐鐨勭郴緇熶腑錛屽彲鐢辨搷浣滅郴緇熸潵瀹炵幇璇ユ満鍒躲�?/span></p>
<br>
<p><span><span>涓嬮潰鏄竴浜?strong>NUMA</strong><strong>絳栫暐鎺у埗宸ュ叿</strong></span></span></p>
<p><strong><em></em></strong></p>
<p><strong><em><span>NUMACTL </span></em></strong><span><span>鏄瀹氳繘紼?/span><span>NUMA</span><span>絳栫暐鐨勫懡浠よ宸ュ叿銆傚浜庨偅浜涙棤娉曚慨鏀瑰拰閲嶆柊緙栬瘧鐨勭▼搴忥紝瀹冨彲浠ヨ繘琛岄潪甯告湁鏁堢殑絳栫暐璁懼畾銆?/span><span>Numactl</span><span>浣跨鐞嗗憳鍙互閫氳繃綆�鍗曠殑鍛戒護(hù)琛岃皟鐢ㄦ潵璁懼畾榪涚▼鐨勭瓥鐣ワ紝</span><span> </span><span>騫跺彲浠ラ泦鎴愬埌綆＄悊鑴氭湰涓�?/span><span> </span></span></p>
<p>&nbsp;<span><span>Numactl</span><span>鐨勪富瑕佸姛鑳藉寘鎷細(xì)</span><span> </span></span></p>
<p><span><span><span>1.</span>&nbsp;</span></span><span><span>璁懼畾榪涚▼鐨勫唴瀛樺垎閰嶅熀鏈瓥鐣?/span></span></p>
<p><span><span><span>2.&nbsp;</span></span></span><span><span>闄愬畾鍐呭瓨鍒嗛厤鑼冨洿錛屽鏌愪竴鐗瑰畾鑺傜偣鎴栭儴鍒嗚妭鐐歸泦鍚?/span></span></p>
<p><span><span><span>3.&nbsp;</span></span></span><span><span>瀵硅繘紼嬭繘琛岃妭鐐規(guī)垨鑺傜偣闆嗗悎鐨勭粦瀹?/span></span></p>
<p><span><span><span>4.&nbsp;</span></span></span><span><span>淇敼鍛藉悕鍏變韓鍐呭瓨錛?/span><span>tmpfs</span><span>鎴?/span><span>hugetblfs</span><span>絳夌殑鍐呭瓨絳栫暐</span></span></p>
<p><span><span><span>5.&nbsp;</span></span></span><span><span>鑾峰彇褰撳墠絳栫暐淇℃伅鍙?qiáng)鐘舵�?/span></span></p>
<p><span><span><span>6.&nbsp;</span></span></span><span><span>鑾峰彇</span><span>NUMA</span><span>紜歡鎷撴墤</span></span></p>
<p><span>&nbsp;</span><span><span>涓嬮潰鏄嬌鐢?/span><span>numactl</span><span>璁懼畾榪涚▼絳栫暐鐨勫疄渚嬶細(xì)</span></span></p>
<p><span>&nbsp;</span><span>numactl --cpubind=0 --membind=0,1 program</span></p>
<p>&nbsp;<span><span>鍏舵剰涔変負(fù)錛氬湪鑺傜偣</span><span>0</span><span>涓婄殑</span><span>CPU</span><span>榪愯鍚嶄負(fù)</span><span>program</span><span>鐨勭▼搴忥紝騫朵笖鍙湪鑺傜偣</span><span>0</span><span>錛?/span><span>1</span><span>涓婂垎閰嶅唴瀛樸�?/span><span>Cpubind</span><span>鐨勫弬鏁版槸鑺傜偣緙栧彿錛岃�屼笉鏄?/span><span>cpu</span><span>緙栧彿銆傚湪姣忎釜鑺傜偣涓婃湁澶氫釜</span><span>CPU</span><span>鐨勭郴緇熶笂錛岀紪鍙風(fēng)殑瀹氫箟欏哄簭鍙兘浼?xì)涓嶅悓銆?/span></span></p>
<p><span>&nbsp;</span><span><span>涓嬮潰鏄嬌鐢?/span><span>numactl</span><span>鏇存敼鍏變韓鍐呭瓨孌電殑鍒嗛厤絳栫暐鐨勫疄渚嬶細(xì)</span></span></p>
<p><span>&nbsp;</span><span>numactl --length=1G --file=/dev/shm/interleaved --interleave=all</span></p>
<p><span>&nbsp;</span><span><span>鍏舵剰涔変負(fù)錛?/span><span> </span><span>瀵瑰懡鍚嶅叡浜唴瀛?/span><span>interleaved</span><span>榪涜璁劇疆錛屽叾絳栫暐涓哄叏鑺傜偣浜ょ粐鍒嗛厤錛屽ぇ灝忎負(fù)</span><span>1G</span><span>銆?/span></span></p>
<p><span>&nbsp;</span><span><strong><span>NUMASTAT </span></strong><span>鏄幏鍙?/span><span>NUMA</span><span>鍐呭瓨璁塊棶緇熻淇℃伅鐨勫懡浠よ宸ュ叿銆傚浜庣郴緇熶腑鐨勬瘡涓妭鐐癸紝鍐呮牳緇存姢浜?jiǎn)涓�浜涙湁鍏?/span><span>NUMA</span><span>鍒嗛厤鐘舵�佺殑緇熻鏁版嵁銆?/span><span>numastat</span><span>鍛戒護(hù)浼?xì)鍩轰簬鑺傜偣瀵瑰唴瀛樼殑鐢宠P紝鍒嗛厤錛岃漿縐伙紝澶辮觸絳夌瓑鍋氬嚭緇熻錛屼篃浼?xì)鎶ュ?/span><span>NUMA</span><span>絳栫暐鐨勬墽琛岀姸鍐點(diǎn)�傝繖浜涗俊鎭浜庢祴璇?/span><span>NUMA</span><span>絳栫暐鐨勬湁鏁堟�ф槸闈炲父鏈夌敤鐨勩�?/span></span></p>
<p>涓嬮潰浠嬬粛<span><strong><em><span>NUMA</span></em></strong><strong><em><span>絳栫暐鐨勫疄鐜版柟寮忓拰絳栫暐</span></em></strong><strong><em></em></strong></span></p>
<p>&nbsp;</p>
<p><span><span>鍦ㄦ渶鏂扮殑閫氱敤鎿嶄綔緋葷粺錛?/span><span> </span><span>濡?/span><span>Windows</span><span>鍜?/span><span>linux</span><span>涓婏紝</span><span> </span><span>閮戒笉鍚岀▼搴︾殑鎻愪緵浜?jiǎn)闈㈠?/span><span>NUMA</span><span>鏋舵瀯鐨勭郴緇熸帶鍒跺拰</span><span>API</span><span>鏀寔銆備笅闈互</span><span>linux</span><span>涓轟緥錛屽璇ョ被鎺ュ彛榪涜璇存槑銆?/span></span></p>
<p><span>&nbsp;</span></p>
<p><span><strong><em><span>Linux</span></em></strong><strong><em><span>涓嬬殑</span></em></strong><strong><em><span>NUMA API </span></em></strong></span></p>
<p><span>&nbsp;</span></p>
<p><span><span>鐗堟湰涓?/span><span>2.5</span><span>涔嬪悗鐨?/span><span>linux</span><span>鍐呮牳鍦ㄨ繘紼嬭皟搴︼紝鍐呭瓨綆＄悊絳夋柟闈㈠</span><span>NUMA</span><span>緋葷粺鍋氫簡(jiǎn)澶ч噺浼樺寲銆傚悓鏃訛紝鍩轟簬</span><span>2.6</span><span>鍐呮牳鐗堟湰鐨勫悇涓昏</span><span>linux</span><span>鍙戣鐗堬紝濡?/span><span>Redhat</span><span>錛?/span><span>SUSE</span><span>絳夊潎鍖呮嫭浜?jiǎn)闈㈠悜鐢ㄦ堬L(fēng)┖闂寸殑</span><span>numautils</span><span>宸ュ叿鍖咃紝鎻愪緵瀵?/span><span>NUMA</span><span>緋葷粺鍐呭瓨絳栫暐鐨勭洃鎺у姛鑳斤紝騫跺紑鏀鵑潰鍚戠敤鎴風(fēng)┖闂寸▼搴忕殑</span><span>API</span><span>鎺ュ彛銆傝鎺ュ彛涔?fàn)鎯笂绉颁?/span><span>NUMA API</span><span>銆?/span></span></p>
<p><span>&nbsp;</span></p>
<p><span><span>NUMA API</span><span>涓昏浠誨姟鏄鐞?/span><span>NUMA</span><span>鐨勫唴瀛樼瓥鐣ャ�?/span><span>NUMA</span><span>絳栫暐閫氳繃鍑犱釜瀛愮郴緇熺殑鍗忓悓宸ヤ綔鏉ュ疄鐜般�傚唴鏍哥鐞嗚繘紼嬬殑鍐呭瓨鍒嗛厤鏈哄埗浠ュ強(qiáng)鐗規(guī)畩鐨勫唴瀛樻槧灝勩�?/span><span>NUMA API</span><span>閫氳繃鏂板紩鍏ョ殑</span><span>3</span><span>涓唴鏍哥郴緇熻皟鐢ㄦ潵瀹炵幇榪欎竴鐐廣�傚湪鐢ㄦ埛絀洪棿涓紝</span><span>NUMA API</span><span>閫氳繃</span><span>libnuma</span><span>搴撴彁渚涗簡(jiǎn)緇熶竴鐨勬帴鍙ｄ緵鐢ㄦ埛絀洪棿紼嬪簭浣跨敤銆傜浉瀵逛簬緋葷粺璋冪敤錛?/span><span>libnuma</span><span>鎺ュ彛鏇村姞娓呮櫚鏄撶敤銆傚悓鏃?/span><span>NUMA API</span><span>榪樻彁渚涗簡(jiǎn)鍛戒護(hù)琛屽伐鍏?/span><span>numactl</span><span>鍜?/span><span>numastat</span><span>鏉ュ府鍔╃郴緇熺鐞嗗憳瀹炵幇榪涚▼綰у埆鐨勭瓥鐣ョ鐞嗐�?/span></span></p>
<p><span>&nbsp;</span></p>
<p><span><span>鍦?/span><span>Linux</span><span>涓?/span><span>NUMA API</span><span>鏀寔鍥涚鍐呭瓨鍒嗛厤絳栫暐錛?/span></span></p>
<p><span>&nbsp;</span></p>
<ol type=1>
    <li><span><span>緙虹渷</span><span>(default) -&nbsp;</span><span>鎬繪槸鍦ㄦ湰鍦拌妭鐐瑰垎閰嶏紙鍒嗛厤鍦ㄥ綋鍓嶇嚎紼嬭繍琛岀殑鑺傜偣涓婏級(jí)</span></span>
    <li><span><span>緇戝畾</span><span>(bind) -&nbsp;</span><span>鍒嗛厤鍒版寚瀹氳妭鐐逛笂</span></span>
    <li><span><span>浜ょ粐</span><span>(interleave) -&nbsp;</span><span>鍦ㄦ墍鏈夎妭鐐規(guī)垨鑰呮寚瀹氱殑鑺傜偣涓婁氦緇囧垎閰?/span></span>
    <li><span><span>浼樺厛</span><span>(preferred) -&nbsp;</span><span>鍦ㄦ寚瀹氳妭鐐逛笂鍒嗛厤錛屽け璐ュ垯鍦ㄥ叾浠栬妭鐐逛笂鍒嗛厤</span></span> </li>
</ol>
<p><span>&nbsp;</span></p>
<p><span><span>緇戝畾鍜屼紭鍏堢殑鍖哄埆鏄紝鍦ㄦ寚瀹氳妭鐐逛笂鍒嗛厤澶辮觸鏃訛紙濡傛棤瓚沖鍐呭瓨錛夛紝緇戝畾絳栫暐浼?xì)鎶ュ憡鍒嗛厤澶辫触锛岃�屼紭鍏堢瓥鐣ヤ細(xì)灝濊瘯鍦ㄥ叾浠栬妭鐐逛笂榪涜鍒嗛厤銆傚己鍒朵嬌鐢ㄧ粦瀹氭湁鍙兘浼?xì)瀵艰嚧鍓嶆湡鐨勫唴瀛樼煭缂哄Q屽茍寮曡搗澶ч噺鎹㈤〉銆傚湪</span><span>libnuma</span><span>搴撲腑錛屼紭鍏堝拰緇戝畾鏄粍鍚堝湪涓�璧風(fēng)殑銆傞�氳繃瀵圭嚎紼嬭皟鐢?/span><span>uma_set_strict</span><span>鍑芥暟錛屽彲浠ュ湪涓ょ絳栫暐闂村垏鎹€�傜己鐪佺殑絳栫暐鏄洿鍔犳櫘閫傜殑浼樺厛絳栫暐銆?/span></span></p>
<p><span>&nbsp;</span></p>
<p><span><span>絳栫暐鍙互鍩轟簬榪涚▼鎴栧唴瀛樺尯鍩熻瀹氥�傝繘紼嬬瓥鐣ュ鏁翠釜榪涚▼鍐呯殑鍐呭瓨鍒嗛厤閮芥湁鏁堬紝鑰屽唴瀛樺尯鍩熺瓥鐣ヤ綔鐢ㄤ簬鎸囧畾鐨勫唴瀛樺尯鍩燂紝鍏朵紭鍏堢駭姣旇繘紼嬬瓥鐣ヨ楂樸�?/span></span></p>
<p><span>&nbsp;</span></p>
<p><strong><em><span>榪涚▼絳栫暐</span></em></strong><strong><em><span> </span></em></strong><span><span>浣滅敤浜庢墍鏈夌敱鍐呮牳鍒嗛厤鐨勫唴瀛橀〉錛屽寘鎷?/span><span>malloc, </span><span>緋葷粺璋冪敤涓嬌鐢ㄧ殑鍐呮牳綰х殑鍒嗛厤浠ュ強(qiáng)鏂囦歡緙撳啿鍖虹瓑銆傚敮涓�鐨勪緥澶栨槸錛屼腑鏂腑鍒嗛厤鐨勫唴瀛樻�繪槸鍦ㄥ綋鍓嶈妭鐐逛腑銆傚綋瀛愯繘紼?/span><span>Fork</span><span>鏃訛紝浼?xì)鐟翩壙鐖稑q涚▼鐨勮繘紼嬬瓥鐣ャ�?/span></span></p>
<p><span>&nbsp;</span></p>
<p><strong><em><span>鍐呭瓨鍖哄煙絳栫暐</span></em></strong><strong><em><span> </span></em></strong><span><span>鍙堢О涓?/span><span>VMA</span><span>絳栫暐錛屽畠鍏佽涓�涓繘紼嬩負(fù)鑷繁鍦板潃絀洪棿閲岀殑涓�鍧楀唴瀛樿緗瓥鐣ャ�?/span><span>鍐呭瓨鍖哄煙</span><span>絳栫暐姣旇繘紼嬬瓥鐣ュ叿鏈夋洿楂樼殑浼樺厛綰с�傚畠鐨勪富瑕佷紭鐐瑰湪浜庤兘澶熷湪鍒嗛厤鍙戠敓鍓嶈繘琛岃緗�傜洰鍓嶏紝鍐呭瓨鍖虹瓥鐣ュ彧鏀寔涓�閮ㄥ垎鍐呭瓨鏈哄埗錛屽錛?/span><span>SYSV</span><span>鍏變韓鍐呭瓨錛?/span><span>shmem</span><span>鍜?/span><span>tmpfs</span><span>鏂囦歡鏄犲皠錛屼互鍙?/span><span>hugetlbfs</span><span>鏂囦歡緋葷粺銆傚湪鍏變韓鍐呭瓨孌墊垨鏂囦歡鏄犲皠琚垹闄ゅ墠錛屽叡浜唴瀛樼殑鍖哄煙絳栫暐浼?xì)涓�鐩存湁鏁堛�?/span></span></p>
<p><span>Linux</span><span>緋葷粺鎻愪緵鍛戒護(hù)琛屽強(qiáng)緙栫▼</span><span>API</span><span>涓ょ駭鐢ㄦ埛絀洪棿宸ュ叿鏉ュ絳栫暐榪涜鎺у埗銆?/span></p>
<p><span><strong><span>libnuma&nbsp;-- NUMA</span></strong><strong><span>絳栫暐鐨勫簲鐢ㄧ▼搴忕紪紼嬫帴鍙?/span></strong><strong></strong></span></p>
<p><span>&nbsp;</span></p>
<p><span><span>灝界</span><span>numactl</span><span>鑳藉鐢ㄤ綔榪涚▼綰у埆鐨勫唴瀛樻帶鍒訛紝浣嗗叾緙虹偣涔熷緢鏄庢樉錛氬垎閰嶇瓥鐣ヤ綔鐢ㄤ簬鏁翠釜榪涚▼錛屾棤娉曟寚瀹氬埌綰跨▼鎴栬�呯壒瀹氬唴瀛樺尯鍩熴�?/span><span>Libnuma</span><span>涓烘洿鍔犵簿緇嗙殑鎺у埗鎻愪緵浜?/span><span>API</span><span>鎺ュ彛銆?/span></span></p>
<p><span>&nbsp;</span></p>
<p><span><span>搴旂敤紼嬪簭鍙渶鍦ㄤ唬鐮佷腑寮曠敤</span><span>numa.h</span><span>澶存枃浠?/span><span>, </span><span>騫跺湪榪炴帴鏃跺涓嬭繛鎺?/span><span>libnuma</span><span>鐨勫叡浜簱鍗沖彲鏂逛究浣跨敤</span><span>libnuma</span><span>錛?/span></span></p>
<p><span>&nbsp;</span></p>
<p><span><span>#include &lt;numa.h&gt;</span></span></p>
<p><span><span>&#8230;.</span></span></p>
<p><span><span>cc ... -lnuma</span></span></p>
<p><span>&nbsp;</span></p>
<p><span><span>鍦ㄥ紑濮嬩嬌鐢?/span><span>NUMA API</span><span>鏇存敼絳栫暐鎴栧垎閰嶅唴瀛樹箣鍓?/span><span>錛?/span><span> </span><span>棣栧厛闇�瑕佽皟鐢?/span><span>numa_available()</span><span>鍑芥暟銆?/span><span> </span><span>涔嬪悗</span><span>, </span><span>鍒欏彲浠ヤ嬌鐢?/span><span>libnuma</span><span>鐨勬帴鍙ｅ榪涚▼絳栫暐榪涜鏇存敼</span><span>錛?/span><span>鎴栧垎閰嶅唴瀛樸�?/span><span>Libnuma</span><span>搴撶殑鍑芥暟鍖呮嫭浠ヤ笅鍑犵粍</span><span>:</span></span></p>
<p><span>&nbsp;</span></p>
<p><span><span><span>1.</span><span>&nbsp;&nbsp;&nbsp; </span></span></span><span><span>鐜淇℃伅</span><span> <span>&#8211; </span></span><span>鍖呮嫭涓�緇勭敤浜庤幏鍙栫郴緇熷唴瀛樺拰</span><span>CPU</span><span>鎷撴墤淇℃伅鐨勫嚱鏁幫紝濡傜郴緇熻妭鐐規(guī)暟鐩紝鐗瑰畾鑺傜偣鐨勫唴瀛樺ぇ灝忕瓑絳夈�?/span></span></p>
<p><span><span><span>2.</span><span>&nbsp;&nbsp;&nbsp; </span></span></span><span><span>榪涚▼絳栫暐</span><span> <span>&#8211; </span></span><span>鍖呮嫭涓�緇勭敤浜庤幏鍙栵紝璁懼畾鍜屾洿鏀硅繘紼嬬駭絳栫暐鐨勫嚱鏁幫紱</span></span></p>
<p><span><span><span>3.</span><span>&nbsp;&nbsp;&nbsp; </span></span></span><span><span>鍐呭瓨鍖哄煙絳栫暐</span><span> <span>&#8211; </span></span><span>鍖呮嫭涓�緇勭敤浜庤瀹氱壒瀹氬唴瀛樺尯鍩熺瓥鐣ョ殑鍑芥暟錛?/span></span></p>
<p><span><span><span>4.</span><span>&nbsp;&nbsp;&nbsp; </span></span></span><span><span>鑺傜偣緇戝畾</span><span> -&nbsp;</span><span>灝嗙嚎紼嬬粦瀹氬埌鎸囧畾鑺傜偣鎴栬妭鐐圭粍鐨勫嚱鏁幫紱</span></span></p>
<p><span><span><span>5.</span><span>&nbsp;&nbsp;&nbsp; </span></span></span><span><span>鍒嗛厤鍑芥暟</span><span> -&nbsp;</span><span>蹇界暐褰撳墠榪涚▼絳栫暐錛岀洿鎺ヤ嬌鐢ㄧ壒瀹氱殑絳栫暐榪涜鍒嗛厤鐨勪竴緇勫嚱鏁幫紱</span></span></p>
<p><span><span><span>6.</span><span>&nbsp;&nbsp;&nbsp; </span></span></span><span><span>鍏朵粬杈呭姪鍑芥暟</span></span></p>
<p><span>&nbsp;</span></p>
<p><span><span>閫氳繃浣跨敤榪欎簺鎺ュ彛錛岀▼搴忓憳鍙互闈炲父鐏墊椿鐨勯厤緗▼搴忓唴瀛樺垎閰嶇殑鏂瑰紡鍜岀瓥鐣ワ紝</span><span> </span><span>浠ヨ揪鍒頒紭鍖栨�ц兘鐨勭洰鐨勩�傞�氬父鐨勫熀浜?/span><span>NUMA</span><span>鐨勫唴瀛樺垎閰嶆祦紼嬩負(fù)</span><span>:</span></span></p>
<p><span><span><span>1.</span><span>&nbsp;&nbsp;&nbsp; </span></span></span><span><span>浣跨敤</span><span>numa_available()</span><span>鍒ゅ畾緋葷粺鏄惁鏀寔</span><span>NUMA</span></span></p>
<p><span><span><span>2.</span><span>&nbsp;&nbsp;&nbsp; </span></span></span><span><span>浣跨敤榪涚▼絳栫暐鍑芥暟瀹氫箟榪涚▼鐨勬暣浣撶瓥鐣?/span></span></p>
<p><span><span><span>3.</span><span>&nbsp;&nbsp;&nbsp; </span></span></span><span><span>浣跨敤鑺傜偣緇戝畾鍑芥暟鍚堢悊緇戝畾綰跨▼</span></span></p>
<p><span><span><span>4.</span><span>&nbsp;&nbsp;&nbsp; </span></span></span><span><span>浣跨敤鏅�氱殑鍒嗛厤鍑芥暟</span><span>(</span><span>濡?/span><span>malloc)</span><span>榪涜鏅�氬垎閰?/span></span></p>
<p><span><span><span>5.</span><span>&nbsp;&nbsp;&nbsp; </span></span></span><span><span>瀵逛簬鐗瑰畾鎬ц兘闇�姹傜殑浠ｇ爜</span><span>, </span><span>浣跨敤</span><span>NUMA</span><span>鍒嗛厤鍑芥暟鍋氭寚瀹氬垎閰?/span></span></p>
<p><span><span><span>6.</span><span>&nbsp;&nbsp;&nbsp; </span></span></span><span><span>瀵逛簬鍐呭瓨鍖哄煙錛屼嬌鐢ㄥ唴瀛樺尯鍩熺瓥鐣ュ嚱鏁拌瀹氬叾鍒嗛厤絳栫暐</span></span></p>
<p>鍓嶉潰璇翠簡(jiǎn)NUMA鐨勬�繪�葷壒鐐癸紝鏈夋湅鍙嬮棶浜?jiǎn)杩欎箞涓�涓棶棰橈細(xì)瑕佹槸鎴戠殑紼嬪簭灝辨槸涔卞簭鐨勮闂唴瀛橈紝涔熶笉澶彲鑳芥敼浜?jiǎn)锛岄偅鎬庝箞鍔炲憿錛熸槸涓嶆槸灝辨敞瀹氳NUMA嬈鴻礋浜?jiǎn)锛熶篃涓嶆槸銆?/p>
<p>鍦ㄨ嫳鐗瑰皵涓嬩竴浠eon澶勭悊鍣ㄥ鉤鍙頒腑錛孊IOS閲屾湁涓�涓?span lang=EN-US>NUMA </span>閫夐」錛屽彲浠ユ寚瀹氭�庝箞鏄犲皠鍐呭瓨銆備互涓らCPU涓轟緥銆傚鏋滄寚瀹歂UMA mode錛岄偅涔堝墠涓�鍗婂唴瀛樼┖闂磋鎸囧畾鍒扮洿鎺ヨ繛鎺PU0錛?鍚庝竴鍗婂唴瀛樼┖闂磋鎸囧畾鍒扮洿鎺ヨ繛鎺PU1錛屾崲鍙ヨ瘽璇村氨鏄垜浠鐨凬UMA鏂瑰紡錛涘鏋滄寚瀹歂on NUMA錛岄偅涔堝氨鏄鏁伴〉闈㈣鎸囧畾鍒扮洿鎺ヨ繛鎺PU0錛屽伓鏁伴〉闈㈣鎸囧畾鍒扮洿鎺ヨ繛鎺PU1錛屼篃灝辨槸欏甸潰濂囧伓浜ら敊鍒嗗竷錛岀敤鎴蜂綋楠屼笂灝辮窡浼犵粺鏂瑰紡綾諱技錛岃櫧鐒舵病鏈変簡(jiǎn)NUMA鐨勫ソ澶勶紝浣嗕篃娌¤NUMA浼や簡(jiǎn)銆?/p>
</span></span></span>
<img src ="http://www.shnenglu.com/beautykingdom/aggbug/72826.html" width = "1" height = "1" /><br><br><div align=right><a style="text-decoration:none;" href="http://www.shnenglu.com/beautykingdom/" target="_blank">chatler</a> 2009-02-02 22:18 <a href="http://www.shnenglu.com/beautykingdom/archive/2009/02/02/72826.html#Feedback" target="_blank" style="text-decoration:none;">鍙戣〃璇勮</a></div>]]></description></item></channel></rss>                                                                                                                                                                                                                                                       <a href="http://www.shnenglu.com/">&#38738;&#38738;&#33609;&#21407;&#32508;&#21512;&#20037;&#20037;&#22823;&#20234;&#20154;&#23548;&#33322;&#95;&#33394;&#32508;&#21512;&#20037;&#20037;&#22825;&#22825;&#32508;&#21512;&#95;&#26085;&#26085;&#22108;&#22108;&#22812;&#22812;&#29408;&#29408;&#20037;&#20037;&#19969;&#39321;&#20116;&#26376;&#95;&#28909;&#20037;&#20037;&#36825;&#37324;&#21482;&#26377;&#31934;&#21697;</a>

<div style="position:fixed;left:-9000px;top:-9000px;"><font id="pjuwb"></font><button id="pjuwb"><pre id="pjuwb"></pre></button><sub id="pjuwb"></sub><tbody id="pjuwb"><var id="pjuwb"><address id="pjuwb"></address></var></tbody><listing id="pjuwb"><label id="pjuwb"><strong id="pjuwb"></strong></label></listing><wbr id="pjuwb"><small id="pjuwb"><tbody id="pjuwb"></tbody></small></wbr><ins id="pjuwb"><xmp id="pjuwb"></xmp></ins><style id="pjuwb"></style><label id="pjuwb"><em id="pjuwb"><li id="pjuwb"></li></em></label><samp id="pjuwb"></samp><menu id="pjuwb"><input id="pjuwb"></input></menu><pre id="pjuwb"><tbody id="pjuwb"><tfoot id="pjuwb"><button id="pjuwb"></button></tfoot></tbody></pre><form id="pjuwb"></form><i id="pjuwb"><style id="pjuwb"><label id="pjuwb"><sup id="pjuwb"></sup></label></style></i><li id="pjuwb"><table id="pjuwb"><abbr id="pjuwb"></abbr></table></li><video id="pjuwb"></video><dfn id="pjuwb"></dfn><progress id="pjuwb"></progress><strong id="pjuwb"></strong><mark id="pjuwb"></mark><em id="pjuwb"></em><tbody id="pjuwb"><p id="pjuwb"><strike id="pjuwb"><acronym id="pjuwb"></acronym></strike></p></tbody><option id="pjuwb"></option><strike id="pjuwb"></strike><u id="pjuwb"></u><td id="pjuwb"><center id="pjuwb"><tr id="pjuwb"></tr></center></td><em id="pjuwb"><mark id="pjuwb"><em id="pjuwb"><tt id="pjuwb"></tt></em></mark></em><strong id="pjuwb"></strong><wbr id="pjuwb"></wbr><s id="pjuwb"></s><strong id="pjuwb"></strong><legend id="pjuwb"></legend><nav id="pjuwb"></nav><dl id="pjuwb"><th id="pjuwb"><dl id="pjuwb"></dl></th></dl><noframes id="pjuwb"><ins id="pjuwb"></ins></noframes><font id="pjuwb"></font><strike id="pjuwb"><i id="pjuwb"><style id="pjuwb"><label id="pjuwb"></label></style></i></strike><output id="pjuwb"></output><thead id="pjuwb"><pre id="pjuwb"></pre></thead><source id="pjuwb"></source><menuitem id="pjuwb"><wbr id="pjuwb"></wbr></menuitem><pre id="pjuwb"><span id="pjuwb"><pre id="pjuwb"><big id="pjuwb"></big></pre></span></pre><cite id="pjuwb"><fieldset id="pjuwb"><s id="pjuwb"><rt id="pjuwb"></rt></s></fieldset></cite><big id="pjuwb"><progress id="pjuwb"><big id="pjuwb"></big></progress></big><samp id="pjuwb"><delect id="pjuwb"></delect></samp><dl id="pjuwb"></dl><strike id="pjuwb"><nav id="pjuwb"><dl id="pjuwb"><strong id="pjuwb"></strong></dl></nav></strike><tbody id="pjuwb"><b id="pjuwb"><optgroup id="pjuwb"><rp id="pjuwb"></rp></optgroup></b></tbody><em id="pjuwb"></em><xmp id="pjuwb"><blockquote id="pjuwb"><pre id="pjuwb"></pre></blockquote></xmp>
<i id="pjuwb"><abbr id="pjuwb"><i id="pjuwb"><abbr id="pjuwb"></abbr></i></abbr></i><center id="pjuwb"><acronym id="pjuwb"><center id="pjuwb"></center></acronym></center><pre id="pjuwb"></pre><ul id="pjuwb"><thead id="pjuwb"></thead></ul><blockquote id="pjuwb"><pre id="pjuwb"><sup id="pjuwb"></sup></pre></blockquote><acronym id="pjuwb"></acronym><big id="pjuwb"><s id="pjuwb"></s></big><th id="pjuwb"></th><th id="pjuwb"></th><tbody id="pjuwb"></tbody><thead id="pjuwb"><strike id="pjuwb"></strike></thead><th id="pjuwb"><dl id="pjuwb"><wbr id="pjuwb"></wbr></dl></th><dl id="pjuwb"><strong id="pjuwb"></strong></dl><abbr id="pjuwb"><noframes id="pjuwb"><noscript id="pjuwb"></noscript></noframes></abbr><td id="pjuwb"><ol id="pjuwb"></ol></td><li id="pjuwb"><noscript id="pjuwb"><abbr id="pjuwb"></abbr></noscript></li><small id="pjuwb"><bdo id="pjuwb"><nav id="pjuwb"></nav></bdo></small><style id="pjuwb"></style><optgroup id="pjuwb"><table id="pjuwb"></table></optgroup><center id="pjuwb"><tr id="pjuwb"><dfn id="pjuwb"></dfn></tr></center><th id="pjuwb"></th><u id="pjuwb"></u><tfoot id="pjuwb"><legend id="pjuwb"><i id="pjuwb"></i></legend></tfoot><mark id="pjuwb"></mark><meter id="pjuwb"></meter><nav id="pjuwb"></nav><acronym id="pjuwb"><pre id="pjuwb"><acronym id="pjuwb"><ul id="pjuwb"></ul></acronym></pre></acronym><acronym id="pjuwb"><pre id="pjuwb"><acronym id="pjuwb"><ul id="pjuwb"></ul></acronym></pre></acronym><nobr id="pjuwb"></nobr><sub id="pjuwb"><th id="pjuwb"><menuitem id="pjuwb"><wbr id="pjuwb"></wbr></menuitem></th></sub><thead id="pjuwb"><sub id="pjuwb"></sub></thead><ul id="pjuwb"><address id="pjuwb"><menuitem id="pjuwb"><meter id="pjuwb"></meter></menuitem></address></ul><dfn id="pjuwb"></dfn><pre id="pjuwb"></pre><input id="pjuwb"><cite id="pjuwb"><fieldset id="pjuwb"></fieldset></cite></input><u id="pjuwb"><form id="pjuwb"><u id="pjuwb"></u></form></u><kbd id="pjuwb"><em id="pjuwb"><mark id="pjuwb"></mark></em></kbd><tr id="pjuwb"></tr><del id="pjuwb"><form id="pjuwb"><address id="pjuwb"></address></form></del><tfoot id="pjuwb"><legend id="pjuwb"><ol id="pjuwb"><dl id="pjuwb"></dl></ol></legend></tfoot><menu id="pjuwb"><nobr id="pjuwb"><th id="pjuwb"><nobr id="pjuwb"></nobr></th></nobr></menu><fieldset id="pjuwb"></fieldset><pre id="pjuwb"><blockquote id="pjuwb"><samp id="pjuwb"></samp></blockquote></pre><xmp id="pjuwb"><sup id="pjuwb"><pre id="pjuwb"></pre></sup></xmp><span id="pjuwb"><progress id="pjuwb"></progress></span><font id="pjuwb"></font><var id="pjuwb"><abbr id="pjuwb"></abbr></var><strong id="pjuwb"><label id="pjuwb"><i id="pjuwb"><legend id="pjuwb"></legend></i></label></strong><tr id="pjuwb"><em id="pjuwb"><em id="pjuwb"><output id="pjuwb"></output></em></em></tr><thead id="pjuwb"><strike id="pjuwb"></strike></thead>
<acronym id="pjuwb"></acronym><i id="pjuwb"></i><tt id="pjuwb"></tt><rt id="pjuwb"><source id="pjuwb"><rt id="pjuwb"></rt></source></rt><strike id="pjuwb"><acronym id="pjuwb"></acronym></strike><del id="pjuwb"></del><font id="pjuwb"><output id="pjuwb"><ins id="pjuwb"><output id="pjuwb"></output></ins></output></font><kbd id="pjuwb"><tr id="pjuwb"><kbd id="pjuwb"></kbd></tr></kbd><pre id="pjuwb"><sup id="pjuwb"><delect id="pjuwb"><samp id="pjuwb"></samp></delect></sup></pre><samp id="pjuwb"></samp><track id="pjuwb"></track><tr id="pjuwb"></tr><center id="pjuwb"></center><fieldset id="pjuwb"></fieldset><i id="pjuwb"></i><td id="pjuwb"></td><rt id="pjuwb"></rt><object id="pjuwb"></object><pre id="pjuwb"><progress id="pjuwb"><sub id="pjuwb"><thead id="pjuwb"></thead></sub></progress></pre><kbd id="pjuwb"><tr id="pjuwb"><option id="pjuwb"></option></tr></kbd><output id="pjuwb"><ins id="pjuwb"></ins></output><ol id="pjuwb"></ol><source id="pjuwb"></source><strong id="pjuwb"></strong><ruby id="pjuwb"></ruby><sub id="pjuwb"><meter id="pjuwb"><menuitem id="pjuwb"><meter id="pjuwb"></meter></menuitem></meter></sub><pre id="pjuwb"></pre><center id="pjuwb"></center><tr id="pjuwb"><tbody id="pjuwb"><xmp id="pjuwb"><dd id="pjuwb"></dd></xmp></tbody></tr><video id="pjuwb"></video><pre id="pjuwb"></pre><form id="pjuwb"><optgroup id="pjuwb"></optgroup></form><samp id="pjuwb"></samp><kbd id="pjuwb"></kbd><strong id="pjuwb"><option id="pjuwb"></option></strong><object id="pjuwb"></object><abbr id="pjuwb"><noframes id="pjuwb"><abbr id="pjuwb"></abbr></noframes></abbr><ul id="pjuwb"><del id="pjuwb"><button id="pjuwb"><pre id="pjuwb"></pre></button></del></ul><abbr id="pjuwb"></abbr><strong id="pjuwb"><code id="pjuwb"><strong id="pjuwb"></strong></code></strong><option id="pjuwb"></option><optgroup id="pjuwb"><bdo id="pjuwb"><code id="pjuwb"></code></bdo></optgroup><mark id="pjuwb"><em id="pjuwb"><font id="pjuwb"></font></em></mark><acronym id="pjuwb"><code id="pjuwb"></code></acronym><dl id="pjuwb"></dl><em id="pjuwb"></em><object id="pjuwb"><input id="pjuwb"><object id="pjuwb"></object></input></object><output id="pjuwb"><dd id="pjuwb"></dd></output><option id="pjuwb"><button id="pjuwb"><option id="pjuwb"></option></button></option><small id="pjuwb"></small></div>
<a href="http://www-14333.com" target="_blank">好吊妞这里只有精品</a>|
<a href="http://hhsj31.com" target="_blank">国产精品久久久99</a>|
<a href="http://668756.com" target="_blank">亚洲人成网站精品片在线观看</a>|
<a href="http://mmmm43.com" target="_blank">久久综合狠狠综合久久综青草</a>|
<a href="http://5xxm.com" target="_blank">久久久av水蜜桃</a>|
<a href="http://analemi.com" target="_blank">亚洲精品视频中文字幕</a>|
<a href="http://maoshimh.com" target="_blank">夜夜嗨av一区二区三区免费区</a>|
<a href="http://497988.com" target="_blank">欧美系列亚洲系列</a>|
<a href="http://jisuyswang.com" target="_blank">久久久国产精彩视频美女艺术照福利</a>|
<a href="http://119663.com" target="_blank">久久久精品动漫</a>|
<a href="http://fsxinruiph.com" target="_blank">亚洲毛片在线观看</a>|
<a href="http://414670.com" target="_blank">亚洲欧美国产77777</a>|
<a href="http://6faa.com" target="_blank">亚洲国产成人在线视频</a>|
<a href="http://liuyangzi.com" target="_blank">亚洲精品视频在线观看免费</a>|
<a href="http://62san.com" target="_blank">国产欧美日韩精品在线</a>|
<a href="http://678255.com" target="_blank">欧美激情偷拍</a>|
<a href="http://uu774.com" target="_blank">国产精品区一区</a>|
<a href="http://710477.com" target="_blank">欧美国产精品v</a>|
<a href="http://zhouqie.com" target="_blank">国产精品欧美风情</a>|
<a href="http://sh-hxjdkj.com" target="_blank">欧美承认网站</a>|
<a href="http://by66336.com" target="_blank">国产精品亚洲片夜色在线</a>|
<a href="http://798814.com" target="_blank">欧美刺激午夜性久久久久久久</a>|
<a href="http://ks180.com" target="_blank">国产精品扒开腿做爽爽爽视频</a>|
<a href="http://by4672.com" target="_blank">久久久久久久尹人综合网亚洲</a>|
<a href="http://788997.com" target="_blank">欧美肥婆bbw</a>|
<a href="http://710691.com" target="_blank">久久激五月天综合精品</a>|
<a href="http://tp-88.com" target="_blank">欧美日本韩国一区</a>|
<a href="http://avsemm.com" target="_blank">另类酷文…触手系列精品集v1小说</a>|
<a href="http://388268.com" target="_blank">欧美日韩国产高清视频</a>|
<a href="http://sdgs6788.com" target="_blank">久久影院亚洲</a>|
<a href="http://6556h.com" target="_blank">国产精品亚洲成人</a>|
<a href="http://4986888.com" target="_blank">亚洲区国产区</a>|
<a href="http://990288.com" target="_blank">激情视频亚洲</a>|
<a href="http://72nnnn.com" target="_blank">亚洲一区二区三区777</a>|
<a href="http://345521.com" target="_blank">亚洲三级电影在线观看</a>|
<a href="http://caofujia.com" target="_blank">欧美中文字幕</a>|
<a href="http://7n7l.com" target="_blank">亚洲欧美激情一区二区</a>|
<a href="http://492541.com" target="_blank">久久综合五月天婷婷伊人</a>|
<a href="http://hznaja.com" target="_blank">午夜久久一区</a>|
<a href="http://www66441.com" target="_blank">欧美激情国产高清</a>|
<a href="http://taominze.com" target="_blank">久久久伊人欧美</a>|
<a href="http://www-772773.com" target="_blank">国产精品v欧美精品v日韩</a>|
<a href="http://wyy66.com" target="_blank">亚洲福利国产</a>|
<a href="http://www77577.com" target="_blank">影音先锋久久久</a>|
<a href="http://tonglijinshu.com" target="_blank">亚洲欧美在线免费观看</a>|
<a href="http://highfivewe.com" target="_blank">亚洲视频免费看</a>|
<a href="http://2061375.com" target="_blank">久久最新视频</a>|
<a href="http://www33779.com" target="_blank">免费成人在线观看视频</a>|
<a href="http://609005.com" target="_blank">国产亚洲精品久久久</a>|
<a href="http://bmcshy.com" target="_blank">亚洲男女自偷自拍图片另类</a>|
<a href="http://1116333.com" target="_blank">在线一区二区三区做爰视频网站
</a>|
<a href="http://cqrebo.com" target="_blank">久久精品亚洲一区二区三区浴池</a>|
<a href="http://6aaee.com" target="_blank">欧美日产国产成人免费图片</a>|
<a href="http://bmcshy.com" target="_blank">欧美国产极速在线</a>|
<a href="http://ipx758.com" target="_blank">最新亚洲视频</a>|
<a href="http://08xxxc.com" target="_blank">欧美xart系列高清</a>|
<a href="http://11867a.com" target="_blank">欧美激情一区二区三级高清视频
</a>|
<a href="http://aijiaoji.com" target="_blank">欧美大色视频</a>|
<a href="http://cuucs.com" target="_blank">亚洲国产成人精品女人久久久
</a>|
<a href="http://www-222304.com" target="_blank">亚洲一级黄色</a>|
<a href="http://xb989.com" target="_blank">欧美日韩一区二区三区在线看</a>|
<a href="http://543199.com" target="_blank">欧美激情2020午夜免费观看</a>|
<a href="http://1332233.com" target="_blank">一区在线观看视频</a>|
<a href="http://223533.com" target="_blank">久久先锋影音av</a>|
<a href="http://569688.com" target="_blank">欧美激情五月</a>|
<a href="http://mathck.com" target="_blank">99re6热只有精品免费观看</a>|
<a href="http://www-878009.com" target="_blank">欧美精品啪啪</a>|
<a href="http://86311ib.com" target="_blank">日韩视频一区</a>|
<a href="http://678665.com" target="_blank">午夜精品久久久久久久男人的天堂
</a>|
<a href="http://essuee.com" target="_blank">欧美日韩福利视频</a>|
<a href="http://www330088.com" target="_blank">99国产精品视频免费观看一公开</a>|
<a href="http://caoliu2022.com" target="_blank">夜夜嗨一区二区</a>|
<a href="http://music444.com" target="_blank">欧美日韩在线综合</a>|
<a href="http://smiczbb.com" target="_blank">中文精品99久久国产香蕉</a>|
<a href="http://tuokuba520.com" target="_blank">亚洲欧美在线高清</a>|
<a href="http://www-137999.com" target="_blank">国产麻豆9l精品三级站</a>|
<a href="http://s3yx.com" target="_blank">久久精品国产亚洲高清剧情介绍</a>|
<a href="http://www8xvi.com" target="_blank">久久亚洲国产成人</a>|
<a href="http://123086.com" target="_blank">亚洲精品一区二区网址</a>|
<a href="http://4430x.com" target="_blank">欧美三级网址</a>|
<a href="http://bizssr.com" target="_blank">羞羞视频在线观看欧美</a>|
<a href="http://xaipad.com" target="_blank">免费日韩一区二区</a>|
<a href="http://gelaimeiwedding.com" target="_blank">亚洲精品偷拍</a>|
<a href="http://ynzhuoyi.com" target="_blank">国产精品爽爽爽</a>|
<a href="http://by777131.com" target="_blank">久久免费高清视频</a>|
<a href="http://devwang.com" target="_blank">亚洲欧洲精品一区二区三区不卡
</a>|
<a href="http://039658.com" target="_blank">激情六月婷婷久久</a>|
<a href="http://www068hs.com" target="_blank">欧美高清成人</a>|
<a href="http://2312138.com" target="_blank">亚洲一区二区三区午夜</a>|
<a href="http://www218999.com" target="_blank">老司机精品福利视频</a>|
<a href="http://3233328.com" target="_blank">日韩一区二区精品葵司在线</a>|
<a href="http://ahhccz.com" target="_blank">国产精品一区毛片</a>|
<a href="http://gmqgw.com" target="_blank">噜噜噜在线观看免费视频日韩</a>|
<a href="http://northboiler.com" target="_blank">亚洲区一区二</a>|
<a href="http://cm1-100.com" target="_blank">久久精品人人爽</a>|
<a href="http://cctbdy.com" target="_blank">亚洲精品久久久久久一区二区</a>|
<a href="http://xmjhyey.com" target="_blank">国产麻豆综合</a>|
<a href="http://430390.com" target="_blank">欧美成人精品一区二区</a>|
<a href="http://www-78733.com" target="_blank">亚洲一二区在线</a>|
<a href="http://k82net.com" target="_blank">欧美91视频</a>|
<a href="http://679776.com" target="_blank">亚洲一区一卡</a>|
<a href="http://1332233.com" target="_blank">亚洲第一区色</a>|
<a href="http://482b.com" target="_blank">国产区日韩欧美</a>|
<a href="http://zhenhaozu.com" target="_blank">欧美日本精品在线</a>|
<a href="http://4915w.com" target="_blank">久久国产一二区</a>|
<a href="http://6aaee.com" target="_blank">亚洲视频1区2区</a>|
<a href="http://www94816.com" target="_blank">亚洲第一黄色</a>|
<a href="http://888nei.com" target="_blank">久久久久久久综合色一本</a>|
<a href="http://zz9876.com" target="_blank">日韩亚洲欧美一区二区三区</a>|
<a href="http://caca060.com" target="_blank">国内精品模特av私拍在线观看</a>|
<a href="http://7749137.com" target="_blank">欧美人在线视频</a>|
<a href="http://353033.com" target="_blank">久久久综合激的五月天</a>|
<a href="http://ycpeixun.com" target="_blank">亚洲在线日韩</a>|
<a href="http://360695.com" target="_blank">日韩视频在线你懂得</a>|
<a href="http://b27227.com" target="_blank">麻豆免费精品视频</a>|
<a href="http://v63xs.com" target="_blank">欧美伊人久久久久久久久影院</a>|
<a href="http://xindefalv.com" target="_blank">亚洲另类春色国产</a>|
<a href="http://57sa.com" target="_blank">亚洲电影免费</a>|
<a href="http://www-401313.com" target="_blank">国产一区二区三区最好精华液</a>|
<a href="http://haole081.com" target="_blank">国产精品www色诱视频</a>|
<a href="http://388123cc.com" target="_blank">麻豆精品网站</a>|
<a href="http://www-333304.com" target="_blank">久久精品二区三区</a>|
<a href="http://227237.com" target="_blank">欧美伊人久久久久久久久影院</a>|
<a href="http://am3757.com" target="_blank">aaa亚洲精品一二三区</a>|
<a href="http://by66336.com" target="_blank">亚洲国产高清一区</a>|
<a href="http://www-n77.com" target="_blank">欧美sm视频</a>|
<a href="http://5138555.com" target="_blank">老牛嫩草一区二区三区日本
</a>|
<a href="http://486be.com" target="_blank">午夜激情综合网</a>|
<a href="http://zztto7.com" target="_blank">在线视频亚洲</a>|
<a href="http://neudlac.com" target="_blank">日韩小视频在线观看专区</a>|
<a href="http://www311pi.com" target="_blank">亚洲国产日韩精品</a>|
<a href="http://xian369.com" target="_blank">亚洲国产精品久久久久久女王</a>|
<a href="http://149155.com" target="_blank">国产一区二区久久</a>|
<a href="http://xxoo668.com" target="_blank">国产亚洲欧美日韩精品</a>|
<a href="http://by71222.com" target="_blank">国产精品一区三区</a>|
<a href="http://ss2ssc.com" target="_blank">国产午夜精品理论片a级探花
</a>|
<a href="http://www-24333.com" target="_blank">牛牛影视久久网</a>|
<a href="http://am3757.com" target="_blank">久久手机免费观看</a>|
<a href="http://hernameplz.com" target="_blank">久久天天躁狠狠躁夜夜av</a>|
<a href="http://musicshq.com" target="_blank">久久久久久一区二区</a>|
<a href="http://d1112.com" target="_blank">久久久综合网站</a>|
<a href="http://dddd30.com" target="_blank">久久婷婷国产综合尤物精品</a>|
<a href="http://xxxbobba.com" target="_blank">久久久久一本一区二区青青蜜月</a>|
<a href="http://zw1019.com" target="_blank">久久久久久久网站</a>|
<a href="http://winkwink333.com" target="_blank">久久久久久久综合色一本</a>|
<a href="http://32m8.com" target="_blank">久久综合激情</a>|
<a href="http://qiaoka526.com" target="_blank">免费在线成人av</a>|
<a href="http://shwazrbjd.com" target="_blank">欧美韩日一区二区三区</a>|
<a href="http://km43yy.com" target="_blank">欧美日韩不卡在线</a>|
<a href="http://taominze.com" target="_blank">欧美特黄一级大片</a>|
<a href="http://lfxhfh.com" target="_blank">国产精品夜夜嗨</a>|
<a href="http://9lunli.com" target="_blank">黑人中文字幕一区二区三区</a>|
<a href="http://456985.com" target="_blank">亚洲成人资源</a>|
<a href="http://www-72794.com" target="_blank">亚洲精品欧美日韩专区</a>|
<a href="http://95gun.com" target="_blank">一区二区三区久久精品</a>|
<a href="http://44cgcg.com" target="_blank">亚洲欧美日韩在线一区</a>|
<a href="http://11867a.com" target="_blank">欧美在线黄色</a>|
<a href="http://by3787.com" target="_blank">免费成人你懂的</a>|
<a href="http://taikonghua.com" target="_blank">欧美成人一区二区三区</a>|
<a href="http://668www.com" target="_blank">亚洲精品一区二区三区四区高清</a>|
<a href="http://yxtczx.com" target="_blank">一区二区三区四区五区精品</a>|
<a href="http://junmatek.com" target="_blank">亚洲一区二区三区涩</a>|
<a href="http://www-888690.com" target="_blank">久久久精品久久久久</a>|
<a href="http://2061856.com" target="_blank">欧美福利一区二区三区</a>|
<a href="http://sdhanm.com" target="_blank">国产精品色网</a>|
<a href="http://kuaikan97.com" target="_blank">亚洲成人资源网</a>|
<a href="http://gaobb52.com" target="_blank">亚洲网站在线</a>|
<a href="http://xyyssa.com" target="_blank">久久综合久久久</a>|
<a href="http://www-232323.com" target="_blank">亚洲精品中文字幕在线</a>|
<a href="http://www-q28.com" target="_blank">亚洲欧美日韩在线一区</a>|
<a href="http://www-77663.com" target="_blank">99热精品在线</a>|
<a href="http://gx2020.com" target="_blank">久久精品国产一区二区三区免费看</a>|
<a href="http://www446168.com" target="_blank">中文日韩电影网站</a>|
<a href="http://83319b.com" target="_blank">久久精品一区二区三区四区</a>|
<a href="http://788997.com" target="_blank">欧美韩日一区二区三区</a>|
<a href="http://10669p.com" target="_blank">一区二区免费在线观看</a>|
<a href="http://chunshanketang.com" target="_blank">久久国产加勒比精品无码</a>|
<a href="http://azaz7.com" target="_blank">欧美aaa级</a>|
<a href="http://9927tv.com" target="_blank">国产喷白浆一区二区三区</a>|
<a href="http://777177c.com" target="_blank">91久久夜色精品国产九色</a>|
<a href="http://qiaoka526.com" target="_blank">亚洲综合色激情五月</a>|
<a href="http://xxxsxx.com" target="_blank">免费不卡在线视频</a>|
<a href="http://www5566szy.com" target="_blank">亚洲一卡久久</a>|
<a href="http://emu160.com" target="_blank">欧美激情1区2区3区</a>|
<a href="http://jisuyswang.com" target="_blank">国产亚洲精品激情久久</a>|
<a href="http://292q.com" target="_blank">亚洲婷婷综合久久一本伊一区</a>|
<a href="http://www278999.com" target="_blank">久久久久久国产精品一区</a>|
<a href="http://jkllkg.com" target="_blank">99精品国产热久久91蜜凸</a>|
<a href="http://www-68689.com" target="_blank">久久精品综合</a>|
<a href="http://wwwby6682.com" target="_blank">国产精品入口</a>|
<a href="http://www-18873.com" target="_blank">99视频精品全国免费</a>|
<a href="http://jxjx11.com" target="_blank">美女视频黄免费的久久</a>|
<a href="http://jiujiire.com" target="_blank">中日韩午夜理伦电影免费</a>|
<a href="http://3534m.com" target="_blank">男人的天堂成人在线</a>|
<a href="http://41818c.com" target="_blank">国产一区视频在线看</a>|
<a href="http://xxxx43.com" target="_blank">亚洲欧美日韩一区二区</a>|
<a href="http://luqintang.com" target="_blank">亚洲麻豆视频</a>|
<a href="http://151545.com" target="_blank">欧美福利视频一区</a>|
<a href="http://17vx.com" target="_blank">亚洲国产精品99久久久久久久久</a>|
<a href="http://yw5531.com" target="_blank">欧美资源在线</a>|
<script>
(function(){
    var bp = document.createElement('script');
    var curProtocol = window.location.protocol.split(':')[0];
    if (curProtocol === 'https') {
        bp.src = 'https://zz.bdstatic.com/linksubmit/push.js';
    }
    else {
        bp.src = 'http://push.zhanzhang.baidu.com/push.js';
    }
    var s = document.getElementsByTagName("script")[0];
    s.parentNode.insertBefore(bp, s);
})();
</script>
</body>