??xml version="1.0" encoding="utf-8" standalone="yes"?>91精品国产色综久久,性做久久久久久久久浪潮,99久久精品毛片免费播放http://www.shnenglu.com/skyscribe/zh-cnWed, 07 May 2025 13:56:14 GMTWed, 07 May 2025 13:56:14 GMT60转移到githubhttp://www.shnenglu.com/skyscribe/archive/2012/02/29/move-to-octopress.htmlskyscribeskyscribeWed, 29 Feb 2012 14:52:00 GMThttp://www.shnenglu.com/skyscribe/archive/2012/02/29/move-to-octopress.htmlhttp://www.shnenglu.com/skyscribe/comments/166817.htmlhttp://www.shnenglu.com/skyscribe/archive/2012/02/29/move-to-octopress.html#Feedback0http://www.shnenglu.com/skyscribe/comments/commentRss/166817.htmlhttp://www.shnenglu.com/skyscribe/services/trackbacks/166817.html
感兴的L? http://skyscribe.github.com

skyscribe 2012-02-29 22:52 发表评论
]]>
ruby学习W记《二?/title><link>http://www.shnenglu.com/skyscribe/archive/2012/02/19/ruby-study-1.html</link><dc:creator>skyscribe</dc:creator><author>skyscribe</author><pubDate>Sun, 19 Feb 2012 11:29:00 GMT</pubDate><guid>http://www.shnenglu.com/skyscribe/archive/2012/02/19/ruby-study-1.html</guid><wfw:comment>http://www.shnenglu.com/skyscribe/comments/165993.html</wfw:comment><comments>http://www.shnenglu.com/skyscribe/archive/2012/02/19/ruby-study-1.html#Feedback</comments><slash:comments>0</slash:comments><wfw:commentRss>http://www.shnenglu.com/skyscribe/comments/commentRss/165993.html</wfw:commentRss><trackback:ping>http://www.shnenglu.com/skyscribe/services/trackbacks/165993.html</trackback:ping><description><![CDATA[ruby的blocks和closureҎ明显有别于其它的语aQ其closure本n是real closureQ所l定的context是共享的而非copyQ其设计思\和lisp的相同;blocks本n则可以用于实现closure。二者的关系如下所q?Q?a >来源</a>Q?br /> <blockquote><br /> <div><strong>Yukihiro Matsumoto</strong>: You can reconvert a closure back into a block, so a closure can be used anywhere a block can be used. Often, closures are used to store the status of a block into an instance variable, because once you convert a block into a closure, it is an object that can by referenced by a variable. And of course closures can be used like they are used in other languages, such as passing around the object to customize behavior of methods. If you want to pass some code to customize a method, you can of course just pass a block. But if you want to pass the same code to more than two methods -- this is a very rare case, but if you really want to do that -- you can convert the block into a closure, and pass that same closure object to multiple methods.<br /> </div> </blockquote> <div> <ul> <li> <h2>7U结?/h2> </li> </ul> <blockquote> <ul> <li>block方式3U?/li> </ul> <blockquote> <ul> <li>隐式传入Q内部用yield调用</li> </ul> <blockquote>   <div style="background-color:#eeeeee;font-size:13px;border:1px solid #CCCCCC;padding-right: 5px;padding-bottom: 4px;padding-left: 4px;padding-top: 4px;width: 98%;word-break:break-all"><!--<br /> <br /> Code highlighting produced by Actipro CodeHighlighter (freeware)<br /> http://www.CodeHighlighter.com/<br /> <br /> --> <div style="background-color:#eeeeee;font-size:13px;border:1px solid #CCCCCC;padding-right: 5px;padding-bottom: 4px;padding-left: 4px;padding-top: 4px;width: 98%;word-break:break-all"><!--<br /> <br /> Code highlighting produced by Actipro CodeHighlighter (freeware)<br /> http://www.CodeHighlighter.com/<br /> <br /> --><span style="color: #0000FF; ">def</span><span style="color: #000000; "> thrice<br />   </span><span style="color: #0000FF; ">yield</span><span style="color: #000000; "><br />   </span><span style="color: #0000FF; ">yield</span><span style="color: #000000; "><br />   </span><span style="color: #0000FF; ">yield</span><span style="color: #000000; "><br /> end<br /> <br /> x</span><span style="color: #000000; ">=</span><span style="color: #000000; ">1</span><span style="color: #000000; "><br /> thrice {x</span><span style="color: #000000; ">+=</span><span style="color: #000000; ">2</span><span style="color: #000000; ">}</span></div> <span style="color: #000000; "></span></div> </blockquote> <ul> <li>&block参数传入Q内部直接操?amp;block</li> </ul> <blockquote> <div style="background-color:#eeeeee;font-size:13px;border:1px solid #CCCCCC;padding-right: 5px;padding-bottom: 4px;padding-left: 4px;padding-top: 4px;width: 98%;word-break:break-all"><!--<br /> <br /> Code highlighting produced by Actipro CodeHighlighter (freeware)<br /> http://www.CodeHighlighter.com/<br /> <br /> --><span style="color: #0000FF; ">def</span><span style="color: #000000; "> six_times(</span><span style="color: #000000; ">&</span><span style="color: #000000; ">block)<br />   thrice(</span><span style="color: #000000; ">&</span><span style="color: #000000; ">block)<br />   thrice(</span><span style="color: #000000; ">&</span><span style="color: #000000; ">block)<br /> end<br />  <br /> x </span><span style="color: #000000; ">=</span><span style="color: #000000; "> </span><span style="color: #000000; ">4</span><span style="color: #000000; "><br /> six_times { x </span><span style="color: #000000; ">+=</span><span style="color: #000000; "> </span><span style="color: #000000; ">10</span><span style="color: #000000; "> }</span></div> </blockquote> <ul> <li>&block传入Q保存block为变量,然后调用block.call</li> </ul> <blockquote> <div style="background-color: #eeeeee; font-size: 13px; border: 1px solid #cccccc; padding: 4px 5px 4px 4px; width: 98%;"><!--<br /> <br /> Code highlighting produced by Actipro CodeHighlighter (freeware)<br /> http://www.CodeHighlighter.com/<br /> <br /> --><span style="color: #0000FF; ">def</span><span style="color: #000000; "> save_for_later(</span><span style="color: #000000; ">&</span><span style="color: #000000; ">b)<br />   @saved </span><span style="color: #000000; ">=</span><span style="color: #000000; "> b  </span><span style="color: #008000; ">#</span><span style="color: #008000; "> Note: no ampersand! This turns a block into a closure of sorts.</span><span style="color: #008000; "><br /> </span><span style="color: #000000; ">end<br />  <br /> save_for_later { puts </span><span style="color: #800000; ">"</span><span style="color: #800000; ">Hello!</span><span style="color: #800000; ">"</span><span style="color: #000000; "> }<br /> puts </span><span style="color: #800000; ">"</span><span style="color: #800000; ">Deferred execution of a block:</span><span style="color: #800000; ">"</span><span style="color: #000000;"><br /> @saved.call<br /> @saved.call<br /> </span></div> </blockquote><blockquote>q里的saved保存为main对象的一个成员,后边实现延迟调用?br /> </blockquote></blockquote> <ul> <li>Proc.new/proc</li> </ul> <blockquote> <div style="background-color:#eeeeee;font-size:13px;border:1px solid #CCCCCC;padding-right: 5px;padding-bottom: 4px;padding-left: 4px;padding-top: 4px;width: 98%;word-break:break-all"><!--<br /> <br /> Code highlighting produced by Actipro CodeHighlighter (freeware)<br /> http://www.CodeHighlighter.com/<br /> <br /> --><span style="color: #000000; ">@saved_proc_new </span><span style="color: #000000; ">=</span><span style="color: #000000; "> Proc.new { puts </span><span style="color: #800000; ">"</span><span style="color: #800000; ">I'm declared with Proc.new.</span><span style="color: #800000; ">"</span><span style="color: #000000; "> }<br /> @saved_proc </span><span style="color: #000000; ">=</span><span style="color: #000000; "> proc { puts </span><span style="color: #800000; ">"</span><span style="color: #800000; ">I'm declared with proc.</span><span style="color: #800000; ">"</span><span style="color: #000000; "> }<br /> <br /> @saved_proc_new.call<br /> @saved_proc.call</span></div> </blockquote> <ul> <li>lambda</li> </ul> <blockquote> <div style="background-color:#eeeeee;font-size:13px;border:1px solid #CCCCCC;padding-right: 5px;padding-bottom: 4px;padding-left: 4px;padding-top: 4px;width: 98%;word-break:break-all"><!--<br /> <br /> Code highlighting produced by Actipro CodeHighlighter (freeware)<br /> http://www.CodeHighlighter.com/<br /> <br /> --><span style="color: #000000; ">@saved_lambda </span><span style="color: #000000; ">=</span><span style="color: #000000; "> </span><span style="color: #0000FF; ">lambda</span><span style="color: #000000; "> { puts </span><span style="color: #800000; ">"</span><span style="color: #800000; ">I'm declared with lambda.</span><span style="color: #800000; ">"</span><span style="color: #000000; "> }<br /> @saved_lambda.call</span></div> </blockquote> <ul> <li>method</li> </ul> <blockquote> <div style="background-color: #eeeeee; font-size: 13px; border: 1px solid #cccccc; padding: 4px 5px 4px 4px; width: 98%;"><!--<br /> <br /> Code highlighting produced by Actipro CodeHighlighter (freeware)<br /> http://www.CodeHighlighter.com/<br /> <br /> --><span style="color: #0000FF; ">def</span><span style="color: #000000; "> some_method<br />   puts </span><span style="color: #800000; ">"</span><span style="color: #800000; ">I'm declared as a method.</span><span style="color: #800000; ">"</span><span style="color: #000000; "><br /> end<br /> @method_as_closure </span><span style="color: #000000; ">=</span><span style="color: #000000;"> method(:some_method)<br /> <br /> </span></div> </blockquote></blockquote> <ul> <li> <h2>特点和差?/h2> </li> <ul> <li> <h2>return行ؓ</h2> </li> </ul> </ul> <blockquote><blockquote>当对应的block里边包含return的时候,上述7中方式有些许的不同:<br /> <ul> <li>lambda/method表现出真正的closure行ؓQ仅仅返回closure本nQ外部调用控制流不受影响Qlyield或者call的下一语句执行</li> <li>其它几种会蟩出外部调用者的控制,即return用者,yield/call之后的也不会再执行,直接跛_到最q的end?/li> </ul> <p><br /> </p> </blockquote> <ul> <li> <h3>arity - 参数个数校验</h3> </li> </ul> <blockquote>对于调用点的参数查,呈现如下行ؓQ?br /> <ul> <li>lambda/method严格校验参数的个敎ͼ如果不匹配回抛出异常</li> <li>其它几个不检查参C?/li> </ul> <p><br /> </p> </blockquote></blockquote> <ul> <li> <h2>ȝ</h2> </li> </ul> <blockquote>lambda/method方式呈现完备的closure行ؓQreturn之后l箋下一程Q对于实际传入参CC在调用点查;proc/blocks方式在return的时候直接返回了外部的函数或者blockQ对于传入的参数个数也没有执行检查?br /> <br /> </blockquote>参考:<a >http://innig.net/software/ruby/closures-in-ruby</a> <br />以上l论验证于最新的ruby1.9.3Q和链接中的l论有些怸同;ruby1.8中的proc函数可能有不同的行ؓ?br /> <div style="background-color: #eeeeee; font-size: 13px; border: 1px solid #cccccc; padding: 4px 5px 4px 4px; width: 98%;"><span style="color: #000000;"></span></div> </div><img src ="http://www.shnenglu.com/skyscribe/aggbug/165993.html" width = "1" height = "1" /><br><br><div align=right><a style="text-decoration:none;" href="http://www.shnenglu.com/skyscribe/" target="_blank">skyscribe</a> 2012-02-19 19:29 <a href="http://www.shnenglu.com/skyscribe/archive/2012/02/19/ruby-study-1.html#Feedback" target="_blank" style="text-decoration:none;">发表评论</a></div>]]></description></item><item><title>ruby学习W记《一?/title><link>http://www.shnenglu.com/skyscribe/archive/2012/02/19/ruby-study-0.html</link><dc:creator>skyscribe</dc:creator><author>skyscribe</author><pubDate>Sun, 19 Feb 2012 07:58:00 GMT</pubDate><guid>http://www.shnenglu.com/skyscribe/archive/2012/02/19/ruby-study-0.html</guid><wfw:comment>http://www.shnenglu.com/skyscribe/comments/165976.html</wfw:comment><comments>http://www.shnenglu.com/skyscribe/archive/2012/02/19/ruby-study-0.html#Feedback</comments><slash:comments>0</slash:comments><wfw:commentRss>http://www.shnenglu.com/skyscribe/comments/commentRss/165976.html</wfw:commentRss><trackback:ping>http://www.shnenglu.com/skyscribe/services/trackbacks/165976.html</trackback:ping><description><![CDATA[看得再多也不如自己动手试Q最q有闲就打算认真研究一下ruby语言了?a >Pragmatic programmer</a>中说Q需要一q学一门新语言一遍改造思想Q去q浅的学了javascript的皮毛,今年可以看看rubyq? 个有lisp之风的OO语言了?br /> <br /> <ul> <li>安装环境</li> </ul> W一个想到的是apt-get来下载了Q得到的是一个交互式解析器和~译器。和python的比较类|不过ruby的交互程序是个单独的E序叫做irb?br /> <blockquote> <div style="background-color:#eeeeee;font-size:13px;border:1px solid #CCCCCC;padding-right: 5px;padding-bottom: 4px;padding-left: 4px;padding-top: 4px;width: 98%;word-break:break-all"><!--<br /> <br /> Code highlighting produced by Actipro CodeHighlighter (freeware)<br /> http://www.CodeHighlighter.com/<br /> <br /> --><span style="color: #000000; ">skyscribe:</span><span style="color: #000000; ">~</span><span style="color: #000000; ">$ ruby </span><span style="color: #000000; ">--</span><span style="color: #000000; ">version<br /> ruby </span><span style="color: #000000; ">1.8</span><span style="color: #000000; ">.</span><span style="color: #000000; ">7</span><span style="color: #000000; "> (</span><span style="color: #000000; ">2011</span><span style="color: #000000; ">-</span><span style="color: #000000; ">06</span><span style="color: #000000; ">-</span><span style="color: #000000; ">30</span><span style="color: #000000; "> patchlevel </span><span style="color: #000000; ">352</span><span style="color: #000000; ">) [i686</span><span style="color: #000000; ">-</span><span style="color: #000000; ">linux]<br /> skyscribe:</span><span style="color: #000000; ">~</span><span style="color: #000000; ">$ irb<br /> irb(main):</span><span style="color: #000000; ">001</span><span style="color: #000000; ">:</span><span style="color: #000000; ">0</span><span style="color: #000000; ">></span><span style="color: #000000; "> puts </span><span style="color: #000000; ">"</span><span style="color: #000000; ">hello</span><span style="color: #000000; ">"</span><span style="color: #000000; "><br /> hello<br /> </span><span style="color: #000000; ">=></span><span style="color: #000000;"> nil<br /> <br /> </span></div> </blockquote>可惜得到的不是比较新的版本?br /> <br /> ? q很快想L?a > wiki</a>Q还是用rvm方便的多。教E比较简单,参考它?a >quick installation guide</a>可。第一ơ尝试的时候用apt-get安装了没有purgeQ导致L安装到root用户? ?#8220;permission denied"的问题?br /> <br /> 安装好之后,所有的东西都在$HOME/.rvm下边Q比较干脆?br /> <div style="background-color:#eeeeee;font-size:13px;border:1px solid #CCCCCC;padding-right: 5px;padding-bottom: 4px;padding-left: 4px;padding-top: 4px;width: 98%;word-break:break-all"><!--<br /> <br /> Code highlighting produced by Actipro CodeHighlighter (freeware)<br /> http://www.CodeHighlighter.com/<br /> <br /> --><span style="color: #000000;"><br /> </span><blockquote><span style="color: #000000; ">skyscribe:</span><span style="color: #000000; ">~</span><span style="color: #000000; ">$ rvm install </span><span style="color: #000000; ">1.9</span><span style="color: #000000; ">.</span><span style="color: #000000; ">3</span><span style="color: #000000;"></span><br /> <span style="color: #000000; ">skyscribe:</span><span style="color: #000000; ">~</span><span style="color: #000000;">$ rvm list</span><br /> <span style="color: #000000;"></span><br /> <span style="color: #000000;">rvm rubies</span><br /> <span style="color: #000000;"></span><br /> <span style="color: #000000; ">   ruby</span><span style="color: #000000; ">-</span><span style="color: #000000; ">1.9</span><span style="color: #000000; ">.</span><span style="color: #000000; ">3</span><span style="color: #000000; ">-</span><span style="color: #000000;">p125 [ i686 ]</span><br /> <span style="color: #000000;"></span><br /> <span style="color: #000000; "># Default ruby not </span><span style="color: #0000FF; ">set</span><span style="color: #000000; ">. Try </span><span style="color: #000000; ">'</span><span style="color: #000000; ">rvm alias create default <ruby></span><span style="color: #000000; ">'</span><span style="color: #000000;">.</span><br /> <span style="color: #000000;"></span><br /> <span style="color: #000000; "># </span><span style="color: #000000; ">=></span><span style="color: #000000; "> </span><span style="color: #000000; ">-</span><span style="color: #000000;"> current</span><br /> <span style="color: #000000; "># </span><span style="color: #000000; ">=*</span><span style="color: #000000; "> </span><span style="color: #000000; ">-</span><span style="color: #000000; "> current </span><span style="color: #000000; ">&&</span><span style="color: #000000; "> </span><span style="color: #0000FF; ">default</span><span style="color: #000000;"></span><br /> <span style="color: #000000; ">#  </span><span style="color: #000000; ">*</span><span style="color: #000000; "> </span><span style="color: #000000; ">-</span><span style="color: #000000; "> </span><span style="color: #0000FF; ">default</span><span style="color: #000000;"></span><br /> <span style="color: #000000;"></span><br /> <span style="color: #000000; ">skyscribe:</span><span style="color: #000000; ">~</span><span style="color: #000000; ">$ rvm alias create </span><span style="color: #0000FF; ">default</span><span style="color: #000000; "> ruby</span><span style="color: #000000; ">-</span><span style="color: #000000; ">1.9</span><span style="color: #000000; ">.</span><span style="color: #000000; ">3</span><span style="color: #000000; ">-</span><span style="color: #000000;">p125</span><br /> <span style="color: #000000; ">Creating alias </span><span style="color: #0000FF; ">default</span><span style="color: #000000; "> </span><span style="color: #0000FF; ">for</span><span style="color: #000000; "> ruby</span><span style="color: #000000; ">-</span><span style="color: #000000; ">1.9</span><span style="color: #000000; ">.</span><span style="color: #000000; ">3</span><span style="color: #000000; ">-</span><span style="color: #000000;">p125.</span><br /> <span style="color: #000000; ">Recording alias </span><span style="color: #0000FF; ">default</span><span style="color: #000000; "> </span><span style="color: #0000FF; ">for</span><span style="color: #000000; "> ruby</span><span style="color: #000000; ">-</span><span style="color: #000000; ">1.9</span><span style="color: #000000; ">.</span><span style="color: #000000; ">3</span><span style="color: #000000; ">-</span><span style="color: #000000;">p125.</span><br /> <span style="color: #000000; ">Creating </span><span style="color: #0000FF; ">default</span><span style="color: #000000; "> links</span><span style="color: #000000; ">/</span><span style="color: #000000;">files</span><br /><br /><div style="background-color:#eeeeee;font-size:13px;border:1px solid #CCCCCC;padding-right: 5px;padding-bottom: 4px;padding-left: 4px;padding-top: 4px;width: 98%;word-break:break-all"><!--<br /><br />Code highlighting produced by Actipro CodeHighlighter (freeware)<br />http://www.CodeHighlighter.com/<br /><br />--><span style="color: #000000; "></span><span style="color: #000000; "><div>skyscribe:~$ cat >> ~/.bashrc <br />[[ -s "$HOME/.rvm/scripts/rvm" ]] && . "$HOME/.rvm/scripts/rvm" # This loads RVM into a shell session.<br />^C<br />skyscribe:~$ bash<br />skyscribe:~$ rvm use 1.9.3<br />Using /home/skyscribe/.rvm/gems/ruby-1.9.3-p125<br />skyscribe:~$ ruby -v<br />ruby 1.9.3p125 (2012-02-16 revision 34643) [i686-linux]</div></span></div><br /> <span style="color: #000000; "></span></blockquote></div> <br /> <ul> <li>熟悉和上?/li> </ul> 官方的文档是最好的参考,推荐<a >ruby koans</a>,下蝲下来Q解压后Q是个典型的TDD学习材料Q不断运?br /> <blockquote> <div> <div> ruby path_to_enlightenment.rb</div></div></blockquote><div><div>koans 会遍历每一个test case直到全部完毕Q大概需?个小时以上的旉方可全部完工。中间的注释和THINK ABOUT的部分比较有意思,感悟比较深刻的是以下一些点Q?br /><blockquote><ul><li>完完全全的OOQ所有东西皆为对?/li><li>两种基本的collectionQhash和array基本对应于python的dict和array</li><li>函数调用可以不必d括号Q除非可能引发歧义或者解析错?/li><li>函数参数可以包含blockQ支持lambda和closure<br /></li><li>boolcd更简单,只有false和nil与false{hQ其余全部是true</li><li>控制l构有unless</li><li>cd义是开攑ּ的,便于非R入式设计Q当然也可以允许修改builtin</li><li>每一个对象都有object id</li><li>symbol和string可以互相转化构?/li><li>method的调用可以用send 的方法发送message - proxy变得极度Ҏ</li><li>module可以被class include从而包含方法, 便于mixin设计</li><li>instance variable和class variable 定义方便快捷</li><li>regular expresion的和python极ؓ怼<br /></li></ul></blockquote></div></div><blockquote><div> </div> </blockquote><img src ="http://www.shnenglu.com/skyscribe/aggbug/165976.html" width = "1" height = "1" /><br><br><div align=right><a style="text-decoration:none;" href="http://www.shnenglu.com/skyscribe/" target="_blank">skyscribe</a> 2012-02-19 15:58 <a href="http://www.shnenglu.com/skyscribe/archive/2012/02/19/ruby-study-0.html#Feedback" target="_blank" style="text-decoration:none;">发表评论</a></div>]]></description></item><item><title>Unix文本处理之利器-awk/gawkhttp://www.shnenglu.com/skyscribe/archive/2011/01/02/awk.htmlskyscribeskyscribeSun, 02 Jan 2011 12:32:00 GMThttp://www.shnenglu.com/skyscribe/archive/2011/01/02/awk.htmlhttp://www.shnenglu.com/skyscribe/comments/137894.htmlhttp://www.shnenglu.com/skyscribe/archive/2011/01/02/awk.html#Feedback0http://www.shnenglu.com/skyscribe/comments/commentRss/137894.htmlhttp://www.shnenglu.com/skyscribe/services/trackbacks/137894.htmlsince 1977Q。作者中最引h注目的当属鼎鼎大名的Brian W. Kernighan (即K的简U来源)。目前所用的版本大多是gawk或者nawk.

作ؓ一门微型且完整的编E语aQawk可以用数行代码就完成其他语言需要数倍的LOC才能完成的工作。其设计哲学也是比较Ҏ的,核心是dataQdriven的,q且采用了和CcM的语法来l织。它最核心的思想应该是如下两点:
  • pattern-action l构 借由强大的正则表辑ּ来匹配patternQ然后执行pattern对应的操?/li>
  • Record/Field 处理模型  所有的输入数据都根据制定的record separator 分割?record, 然后没一个record再根据field separator 分割为fields. POSIX 定义?field separator可以为正则表辑ּQ而gawk可以允许record separator同时为正则表辑ּ
引发我花Ҏ间来仔细研究awk的v因是q样的,我们的程序在做profiling的时候,发现原来用shell写的脚本分析一ơ话费的旉太长。初看了下那个脚本,大概的逻辑是要扫名所有的log文gQ按照时间戳关注的旉所耗费的时间提取出来,计算q_|波动{最l画出曲U图?br>
整体的脚本有几个部分QpythonQbashQ,处理一?0MB的log文g需要耗费40分钟?个小Ӟq显然超Z预期Q中间一个处理很长的部分是grep某个旉D늚信息然后按照报表格式写入C间文件中。在惌否优化这一节的时候,忽然想起了模式匚w来(学习Haskell的最深印象)Q于是大致翻了一下awkQ发现很Ҏ通过模式匚w使得按行处理Q同时记录中间的信息Q而一个时间段恰好和awk的record概念d?br>
׃2个小时研M下awk的函数语法,自定义自q旉截取函数Qgawk的strftime很有用,其我们发现记录有蟩跃要自动补全中间的数据记录时Q,通过三个pattern截取需要的信息Q?0分钟写出来awk的代码来?br>
所q的是,其它的shell脚本都不需要Q何改动,重新跑一ơ,3s处理完了原?0MB的文Ӟ看来q点旉投入q是相当值得的?br>
有兴的可参考:
     http://www.gnu.org/manual/gawk/gawk.html

PSQawk的另一作?a >Winberger 供职于google?br> 



skyscribe 2011-01-02 20:32 发表评论
]]>
GCC4.5的profile modehttp://www.shnenglu.com/skyscribe/archive/2010/05/09/GCC4_5ProfileMode.htmlskyscribeskyscribeSun, 09 May 2010 07:16:00 GMThttp://www.shnenglu.com/skyscribe/archive/2010/05/09/GCC4_5ProfileMode.htmlhttp://www.shnenglu.com/skyscribe/comments/114920.htmlhttp://www.shnenglu.com/skyscribe/archive/2010/05/09/GCC4_5ProfileMode.html#Feedback0http://www.shnenglu.com/skyscribe/comments/commentRss/114920.htmlhttp://www.shnenglu.com/skyscribe/services/trackbacks/114920.htmlq里Q,
q增加了一个新的profile模式Q尚处于试验阶段Q,可以ҎE序q行状态给出关于STL使用的一些优化徏议?br>
参看如下的描qͼ
  • An experimental profile mode has been added. This is an implementation of many C++ standard library constructs with an additional analysis layer that gives performance improvement advice based on recognition of suboptimal usage patterns. For example,

    #include <vector>
    int main()
    {
    std::vector<int> v;
    for (int k = 0; k < 1024; ++k)
    v.insert(v.begin(), k);
    }

    When instrumented via the profile mode, can return suggestions about the initial size and choice of the container used as follows:

    vector-to-list: improvement = 5: call stack = 0x804842c ...
    : advice = change std::vector to std::list
    vector-size: improvement = 3: call stack = 0x804842c ...
    : advice = change initial container size from 0 to 1024

    These constructs can be substituted for the normal libstdc++ constructs on a piecemeal basis, or all existing components can be transformed via the -D_GLIBCXX_PROFILE macro.

q个profile mode的主要作用就是根据代码实际运行状늻出关于STL的用优化徏议。有炚w憄是,该profileҎ是intrusive的,必须d-D_GLBCXX_PROFILE来重新编译所有的代码?br>
Profile mode的提出源?9qCGO的一?a >paperQ作者里边出C华h的名字(Ҏ拼音来判断);作者地址填的昄是Purdue大学的:
    Dept. of Comput. Sci., Purdue Univ., West?br>
GCC的Profiler对C++的支持一贯停留在和C同样的水qI׃C++模板机制和OO的存在得很多时候分析profilingl果的意义被大大削弱?br>q个针对STL的profile modeq是很值得期待的?br>



skyscribe 2010-05-09 15:16 发表评论
]]>
boost.cmake: 方便解决boost的升U问?/title><link>http://www.shnenglu.com/skyscribe/archive/2010/05/09/114919.html</link><dc:creator>skyscribe</dc:creator><author>skyscribe</author><pubDate>Sun, 09 May 2010 06:59:00 GMT</pubDate><guid>http://www.shnenglu.com/skyscribe/archive/2010/05/09/114919.html</guid><wfw:comment>http://www.shnenglu.com/skyscribe/comments/114919.html</wfw:comment><comments>http://www.shnenglu.com/skyscribe/archive/2010/05/09/114919.html#Feedback</comments><slash:comments>0</slash:comments><wfw:commentRss>http://www.shnenglu.com/skyscribe/comments/commentRss/114919.html</wfw:commentRss><trackback:ping>http://www.shnenglu.com/skyscribe/services/trackbacks/114919.html</trackback:ping><description><![CDATA[以前常用boost的一些库Q由于项目自w用cmakel织build?br>boost每次升之后Q重新用bjam~译一ơ都很是不便Q某些^収ͼ用默认选项~译有些问题Q基本安装不成功Q?br><br>最q才发现boost的cmake版本已经独立出来了,对于使用cmake的用戯言Q这里是个不错的选择?br>目<a >主页</a>的文档很是清晎ͼ最新的版本?.41 (版本号对应的基本是其upstream的boost版本??br>源代码是用gitl织的,对于Linux用户而言更加方便?br><br>引用其主上的一句话Q?br> <div style="margin-left: 40px;">Boost.<a class="reference external" >CMake</a> (or <a class="reference internal" ><em> alt.boost</em> </a>) is the boost distribution that all the cool kids are using. </div> <br>CMake + <a >GIT</a> +<a >Spinx</a> 实够酷了?br><br><span style="font-family: 微Y雅黑;">~译h可以充分利用强大的CMake了:</span><br> <div style="margin-left: 40px;"> <pre>git clone git://gitorious.org/boost/cmake.git src<br>cd src<br>git checkout <TAG>    //TAG==1.41.0.cmake0<br>mkdir bld<br>cd bld/<br>cmake ../<br></pre> </div> <pre>如果需要按需~译某些库,只需用make edit_cache修改cache卛_?br></pre> <div style="margin-left: 40px;"> </div><img src ="http://www.shnenglu.com/skyscribe/aggbug/114919.html" width = "1" height = "1" /><br><br><div align=right><a style="text-decoration:none;" href="http://www.shnenglu.com/skyscribe/" target="_blank">skyscribe</a> 2010-05-09 14:59 <a href="http://www.shnenglu.com/skyscribe/archive/2010/05/09/114919.html#Feedback" target="_blank" style="text-decoration:none;">发表评论</a></div>]]></description></item><item><title>VIM下实现对C++ STL的IntelliSensehttp://www.shnenglu.com/skyscribe/archive/2010/01/10/105346.htmlskyscribeskyscribeSun, 10 Jan 2010 10:28:00 GMThttp://www.shnenglu.com/skyscribe/archive/2010/01/10/105346.htmlhttp://www.shnenglu.com/skyscribe/comments/105346.htmlhttp://www.shnenglu.com/skyscribe/archive/2010/01/10/105346.html#Feedback0http://www.shnenglu.com/skyscribe/comments/commentRss/105346.htmlhttp://www.shnenglu.com/skyscribe/services/trackbacks/105346.html以前试q在vim下配|STL的Intellisense曄没有成功Q最q有I刚好仔l看了下vim的一些相寚wU的manualQ?/p>

ȝ?a >OmniCppComplete主页上的效果l弄了出来(q里Q?/p>

Ҏ其help提示一步一步来(装完了omnicppcomplete之后Q可?help omnicppcomplete)Q?/p>

  • 首先需要有ctagsQƈ且必LExuberant ctagsQ我的Ubuntu上已l是最新版了:

  • skyscribe@skyscribe:~/study/vim/pics$ ctags --version
    
    Exuberant Ctags 5.8, Copyright (C) 1996-2009 Darren Hiebert
    
      Compiled: Aug  6 2009, 17:06:22
    
      Addresses: <dhiebert@users.sourceforge.net>, http://ctags.sourceforge.net
    
      Optional compiled features: +wildcards, +regex
    
  • ?~/.vimrc 里边加入autoload taglist功能Qؓ了方便自动更新当前tagsQ设|一个快捷键Q?a >q里Q:

map <C-F12> :!ctags -R --c++-kinds=+p --fields=+iaS --extra=+q .<CR>
source ~/.vim/my.vim

 

q里的自动加载已有vim的功能,q一个vim脚步来实?(参?vim :help globl的例子)Q?/p>

cat ~/.vim/my.vim 
let tagfiles = glob("`find ~/.vim/tagfiles -name tags -print`")
let &tags = substitute(tagfiles, "\n", ",", "g")

 

以后需要新的taglistQ只需要放在~/.vim/tagfiles目录下就好了?/p>

vim启动的时候,会自动执行~/.vimrcQ从而调用my.vimQ将事先准备好的taglist更新q去Q这里一般放一些不太变化的静态头文gtag可以了?/p>

  • 要有STL的intelliSenseQ自然要有STL C++的tags database了,q里需要生成之?/h1>

Ҏ上一步的惯例Q需要生成一个tags文gQ放在~/.vim/tagfiles/的某个子目录下:

mkdir -p ~/.vim/tagfiles/gcc<ver>/
ls –l /usr/include/c++/

 

q里需要将ver换成当前pȝ的libstdc++版本QUbuntu 9.10上的?.4.1.

  • 用上边的命o生成对应的tags文gQ?/h1>

ctags -R --c++-kinds=+p --fields=+iaS --extra=+q . -o ~/.vim/tagfiles/gcc4.4/tags /usr/include/c++/4.4

写一个简单的c++E序Q在Insert Mode下,Ctrl+XQ?Ctrl+PQ发现ƈ不能工作Q什么提CZ没有Q初步怀疑是对应的tag文g不对?/p>

q好早有人尝试过了,l出了一U办?q是q里Q,可惜他的Ҏ我试了不行,不过已经可以借用他的思\了:

    • ?usr/include/c++/4.4.1/的内容全部拷贝到一个目录下Q?/li>
      mkdir gcc4.4
      cp -R /usr/include/c++/4.4 ./
    • 写一个脚步替换所有的NAMESPACE宏定?q里用sed完成宏替换,Z避免q于晦ӆQ还是放在一个时的脚步文g里边来,便于调试?Q?/li>
cat generate_sed.sh 
#!/bin/bash
if [ $# -eq 0 ];then
    dir=.
else
    dir=$1
fi
#Substitute scripts
cat > .edit.sed <<- EOF
s/\b_GLIBCXX_BEGIN_NAMESPACE\b\s*\(\s*(\w+)\s*\)/namespace \1{/g
s/_GLIBCXX_BEGIN_NESTED_NAMESPACE\s*\(\s*std\s*,\s*_GLIBCXX_STD_[DP]\s*\)/namespace std{/g
s/\b_GLIBCXX_END_NAMESPACE\b/}/g
s/\b_GLIBCXX_END_(NESTED_\?NAMESPACE\b/}/g
EOF
count=0
cols=`tput cols`
cols=`echo "$cols - 8"|bc`
for file in `find $dir -type f`;do
    if [ "T"$file != "T"$0 ];then
        sed -nf ".edit.sed" $file
        count=`echo "$count+1"|bc`
        echo -ne "="
        if [ `echo "$count%$cols"|bc` -eq 0 ];then
            echo " $count"
        fi
    fi
done
echo "Processed $count files!"
rm -fr .edit.sed
# Generate ctag file
ctags -R --c++-kinds=+p --fields=+iaS --extra=+q 
echo "generated tag file!"
    • 再生成tagQ?/li>
skyscribe@skyscribe:~/libstdc++/gcc4.4$ ./generate_sed.sh . 
==================================================================================================== 100
==================================================================================================== 200
==================================================================================================== 300
==================================================================================================== 400
==================================================================================================== 500
==================================================================================================== 600
===============================================================Processed 663 files!
generated tag file!
ls tags -lh

-rw-r--r-- 1 skyscribe skyscribe 4.2M 2010-01-10 18:21 tags

cp tags ~/.vim/tagfiles/gcc4.4/

 

  • 到这里效果终于出来了Q?/h1>

上边的reviewH口昄当前调用的函C?输入./->/:: 的时候会自动提示Q也可以用CTRL+X CTRL+O 来调出提C窗口,方向键则可以选择?/p>

omni_comp

vec_insert

vec_insert_sel

 

 

 
 
 
 
 


skyscribe 2010-01-10 18:28 发表评论
]]>
cmake的一些小l验http://www.shnenglu.com/skyscribe/archive/2009/12/14/103208.htmlskyscribeskyscribeMon, 14 Dec 2009 12:39:00 GMThttp://www.shnenglu.com/skyscribe/archive/2009/12/14/103208.htmlhttp://www.shnenglu.com/skyscribe/comments/103208.htmlhttp://www.shnenglu.com/skyscribe/archive/2009/12/14/103208.html#Feedback1http://www.shnenglu.com/skyscribe/comments/commentRss/103208.htmlhttp://www.shnenglu.com/skyscribe/services/trackbacks/103208.html初用CMake或者对其了解不太深的hQ可能经怼被\径包含、库搜烦路径、链接\径、RPathq些问题所l倒,因ؓq些东西在手工执行gcc或者编写makefile的时候是很轻而易丄d?/font>

其实我当初也有不疑惑,不过通过较长旉的实践和阅读manualQȝ有了个相对很清晰的认识?/font>

  • 如何使用其manual

cmake的帮助组l的q是很有规律的,了解了其规律Q找自己惌的东西就会很单,所以个得这一点可能是最重要的。其helppȝ大概是这么几c:

    • command

q个是实用过E中最长用到的Q相当于一般脚步语a中的基本语法Q包括定义变量,foreachQstringQifQbuiltin command都在q里?/font>

可以用如下这些命令获取帮助:

cmake --help-commands

q个命o给出所有cmake内置的命令的详细帮助Q一般不知道自己要找什么或者想随机ȝ得时候,可以用这个?/p>

我一般更常用的方法是其重定向到less里边Q然后在~辑器里Ҏ索关键字?/p>

 

另外也可以用如下的办法层层羃搜索范_

cmake --help-command-list

cmake --help-command-list | grep find

skyscribe@skyscribe:~/program/ltesim/bld$ cmake --help-command-list | grep find
find_file
find_library
find_package
find_path
find_program

cmake --help-command find_library

cmake version 2.6-patch 4
------------------------------------------------------------------------------
SingleItem

  find_library
       Find a library.

          find_library(<VAR> name1 [path1 path2 ...])

       This is the short-hand signature for the command that is sufficient in
       many cases.  It is the same as find_library(<VAR> name1 [PATHS path1
       path2 ...])

          find_library(
                    <VAR>
                    name | NAMES name1 [name2 ...]
                    [HINTS path1 [path2 ... ENV var]]
                    [PATHS path1 [path2 ... ENV var]]
                    [PATH_SUFFIXES suffix1 [suffix2 ...]]
                    [DOC "cache documentation string"]
                    [NO_DEFAULT_PATH]
                    [NO_CMAKE_ENVIRONMENT_PATH]
                    [NO_CMAKE_PATH]
                    [NO_SYSTEM_ENVIRONMENT_PATH]
                    [NO_CMAKE_SYSTEM_PATH]
                    [CMAKE_FIND_ROOT_PATH_BOTH |
                     ONLY_CMAKE_FIND_ROOT_PATH |
                     NO_CMAKE_FIND_ROOT_PATH]
                   )

    • variable

和command的帮助比较类|只不q这里可以查找cmake自己定义了那些变量你可以直接使用Q譬如OSNameQ是否是WindowsQUnix{?/font>

我最常用的一个例子:

cmake --help-variable-list  | grep CMAKE | grep HOST
CMAKE_HOST_APPLE
CMAKE_HOST_SYSTEM
CMAKE_HOST_SYSTEM_NAME
CMAKE_HOST_SYSTEM_PROCESSOR
CMAKE_HOST_SYSTEM_VERSION
CMAKE_HOST_UNIX
CMAKE_HOST_WIN32

q里查找所有CMake自己定义的builtin变量Q一般和pȝq_相关?/p>

如果希望所有生成的可执行文件、库攑֜同一的目录下Q可以如此做Q?/p>

q里的target_dir是一个实现设|好的绝对\径。(CMake里边l对路径比相对\径更出问题Q如果可能尽量用l对路径Q?/p>

# Targets directory
set(CMAKE_ARCHIVE_OUTPUT_DIRECTORY ${target_dir}/lib)
set(CMAKE_LIBRARY_OUTPUT_DIRECTORY ${target_dir}/lib)
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY ${target_dir}/bin)
    • property

Property一般很需要直接改动,除非你想修改一些默认的行ؓQ譬如修改生成的动态库文g的soname{?/font>

譬如需要在同一个目录下既生成动态库Q也生成静态库Q那么默认的情况下,cmakeҎ你提供的target名字自动生成cM的libtarget.so, libtarget.aQ但是同一个project只能同时有一个,因ؓtarget必须唯一?/font>

q时候,可以通过修改taget对应的文件名Q从而达到既生成动态库也生静态库的目的?/font>

譬如Q?/font>

cmake --help-property-list | grep NAME
GENERATOR_FILE_NAME
IMPORTED_SONAME
IMPORTED_SONAME_<CONFIG>
INSTALL_NAME_DIR
OUTPUT_NAME
VS_SCC_PROJECTNAME
skyscribe@skyscribe:~$ cmake --help-property OUTPUT_NAME
cmake version 2.6-patch 4
------------------------------------------------------------------------------
SingleItem
  OUTPUT_NAME
       Sets the real name of a target when it is built.
       Sets the real name of a target when it is built and can be used to
       help create two targets of the same name even though CMake requires
       unique logical target names.  There is also a <CONFIG>_OUTPUT_NAME
       that can set the output name on a per-configuration basis.
    • module

用于查找常用的模块,譬如boostQbzip2, python{。通过单的include命o包含预定义的模块Q就可以得到一些模块执行后定义好的变量Q非常方ѝ?/font>

譬如常用的boost库,可以通过如下方式Q?/font>

# Find boost 1.40
INCLUDE(FindBoost)
find_package(Boost 1.40.0 COMPONENTS thread unit_test_framework)
if(NOT Boost_FOUND)
    message(STATUS "BOOST not found, test will not succeed!")
endif()
一般开头部分的解释都相当有用,可满?0%需求:
cmake --help-module FindBoost | head -40
cmake version 2.6-patch 4
------------------------------------------------------------------------------
SingleItem
  FindBoost
       Try to find Boost include dirs and libraries
       Usage of this module as follows:
       == Using Header-Only libraries from within Boost: ==
          find_package( Boost 1.36.0 )
          if(Boost_FOUND)
             include_directories(${Boost_INCLUDE_DIRS})
             add_executable(foo foo.cc)
          endif()
       
       
       == Using actual libraries from within Boost: ==
          set(Boost_USE_STATIC_LIBS   ON)
          set(Boost_USE_MULTITHREADED ON)
          find_package( Boost 1.36.0 COMPONENTS date_time filesystem system ... )
       
          if(Boost_FOUND)
             include_directories(${Boost_INCLUDE_DIRS})
             add_executable(foo foo.cc)
             target_link_libraries(foo ${Boost_LIBRARIES})
          endif()
       
       
       The components list needs to contain actual names of boost libraries
  • 如何Ҏ其生成的中间文g查看一些关键信?/strong>

CMake相比较于autotools的一个优势就在于其生成的中间文gl织的很有序Qƈ且清晰易懂,不像autotools会生成天书一L庞然大物Q?0000+的不鲜见Q?/p>

一般CMake对应的Makefile都是有层U结构的Qƈ且会Ҏ你的CMakeLists.txt间的相对l构在binary directory里边生成相应的目录结构?/p>

譬如对于某一个targetQ一般binary tree下可以找C个文件夹:  CMakeFiles/<targentName>.dir/,比如Q?/p>

skyscribe@skyscribe:~/program/ltesim/bld/dev/simcluster/CMakeFiles/SIMCLUSTER.dir$ ls -l
total 84
-rw-r--r-- 1 skyscribe skyscribe 52533 2009-12-12 12:20 build.make
-rw-r--r-- 1 skyscribe skyscribe  1190 2009-12-12 12:20 cmake_clean.cmake
-rw-r--r-- 1 skyscribe skyscribe  4519 2009-12-12 12:20 DependInfo.cmake
-rw-r--r-- 1 skyscribe skyscribe    94 2009-12-12 12:20 depend.make
-rw-r--r-- 1 skyscribe skyscribe   573 2009-12-12 12:20 flags.make
-rw-r--r-- 1 skyscribe skyscribe  1310 2009-12-12 12:20 link.txt
-rw-r--r-- 1 skyscribe skyscribe   406 2009-12-12 12:20 progress.make
drwxr-xr-x 2 skyscribe skyscribe  4096 2009-12-12 12:20 src
q里Q每一个文仉是个很短的文本文gQ内容相当清晰明了。build.make一般包含中间生成文件的依赖规则QDependInfo.cmake一般包含源代码文g自n的依赖规则?/pre>
比较重要的是flags.make和link.txtQ前者一般包含了cM于GCC?I的相关信息,如搜索\径,宏定义等Q后者则包含了最l生成target时候的linkage信息Q库搜烦路径{?/pre>
q些信息在出现问题的时候是个很好的辅助调试手段?/pre>
  • 文g查找、\径相?/strong>
    • include

一般常用的是:

include_directoriesQ)用于d头文件的包含搜烦路径
cmake --help-command include_directories
cmake version 2.6-patch 4
------------------------------------------------------------------------------
SingleItem
  include_directories
       Add include directories to the build.
         include_directories([AFTER|BEFORE] [SYSTEM] dir1 dir2 ...)
       Add the given directories to those searched by the compiler for
       include files.  By default the directories are appended onto the
       current list of directories.  This default behavior can be changed by
       setting CMAKE_include_directories_BEFORE to ON.  By using BEFORE or
       AFTER you can select between appending and prepending, independent
       from the default.  If the SYSTEM option is given the compiler will be
       told that the directories are meant as system include directories on
       some platforms.
link_directories()用于d查找库文件的搜烦路径
cmake --help-command link_directories
cmake version 2.6-patch 4
------------------------------------------------------------------------------
SingleItem
  link_directories
       Specify directories in which the linker will look for libraries.
         link_directories(directory1 directory2 ...)
       Specify the paths in which the linker should search for libraries.
       The command will apply only to targets created after it is called.
       For historical reasons, relative paths given to this command are
       passed to the linker unchanged (unlike many CMake commands which
       interpret them relative to the current source directory).
    • library search

一般外部库的link方式可以通过两种Ҏ来做Q一U是昄d路径Q采用link_directories()Q?一U是通过find_library()L扑֯应的库的l对路径?/p>

后一U方法是更好的,因ؓ它可以减不潜在的冲突?/p>

        一般find_library会根据一些默认规则来搜烦文gQ如果找刎ͼ会set传入的第一个变量参数、否则,对应的参C被定义,q且有一个xxx-NOTFOUND被定义;可以通过q种方式来调试库搜烦是否成功?/p>

        对于库文件的名字而言Q动态库搜烦的时候会自动搜烦libxxx.so (xxx.dll),静态库则是libxxx.aQxxx.libQ,对于动态库和静态库L的情况,可能会出C些؜乱,需要格外小心;一般尽量做匚wq接?/p>

    • rpath

所谓的rpath是和动态库的加载运行相关的。我一般采用如下的方式取代默认d的rpathQ?/p>

# RPATH and library search setting
SET(CMAKE_SKIP_BUILD_RPATH  FALSE)
SET(CMAKE_BUILD_WITH_INSTALL_RPATH FALSE) 
SET(CMAKE_INSTALL_RPATH "${CMAKE_INSTALL_PREFIX}/nesim/lib")
SET(CMAKE_INSTALL_RPATH_USE_LINK_PATH TRUE) 

 



skyscribe 2009-12-14 20:39 发表评论
]]>
pipeline会启动多个q程Q?/title><link>http://www.shnenglu.com/skyscribe/archive/2009/12/14/103205.html</link><dc:creator>skyscribe</dc:creator><author>skyscribe</author><pubDate>Mon, 14 Dec 2009 11:46:00 GMT</pubDate><guid>http://www.shnenglu.com/skyscribe/archive/2009/12/14/103205.html</guid><wfw:comment>http://www.shnenglu.com/skyscribe/comments/103205.html</wfw:comment><comments>http://www.shnenglu.com/skyscribe/archive/2009/12/14/103205.html#Feedback</comments><slash:comments>0</slash:comments><wfw:commentRss>http://www.shnenglu.com/skyscribe/comments/commentRss/103205.html</wfw:commentRss><trackback:ping>http://www.shnenglu.com/skyscribe/services/trackbacks/103205.html</trackback:ping><description><![CDATA[<p>最q在TL?a >讨论?/a>忽然有h挑v了perl和pythonQ一场关于c++的讨论扯到脚步上q有不少的碰撞,倒是挺有意思)Q我则有感而发的想起了前几天面试的时候问别h的一个基本的shell问题Q?/p> <blockquote><pre><pre style="font-size: 12px; margin: 0em; width: 100%; font-family: consolas,'Courier New',courier,monospace; background-color: #ffffff"><p><font size="4">cat xxx.txt | grep "<span style="color: #8b0000">yyy</span>" | wc –l</font></p></pre></pre></blockquote> <p>问题是这个常见的pipeline操作一般最会起多个q程Q结果那位老兄倒是愣了半天然后目无表情?/p> <p>我只好l唠叨的解释了一下一般pipe的操作需要读取一个进E的输入Q然后将输出送给下一个进E;其实我希望对方干脆利落的回答是有3个,q个问题q是可以了Q我们主要不是用脚本开发,但是如果有这个技能是能得到额外的认可的?/p> <p> </p> <p>TL上的大虾们果然是x众多Q立马有人站出来问:我想知道{案是几个?直接让我怀疑是不是我的脑袋有问题。后来有人给Z可能?个的情ŞQ?/p> <p>      某个变态的shell可能内置了cat,使其成ؓ一个builtinQ然后自p俎代庖的d标准输入Qƈ且将内容文本输出Q那么进E就一个?/p> <p>起初我觉得这个解释ƈ不能成立Q但是经q几个老大的解释还是明白了他所说的情况是shell的builtin?/p> <p> </p> <p>中间又讨v那些可能是builtin的commandQD出的例子是cd/kill/timeQ但是我查了一下Solaris上的Q后两个都是executableQcd扑ֈ一?usr/bin/cd 的kshQ内容如下:</p><pre><pre style="font-size: 12px; margin: 0em; width: 100%; font-family: consolas,'Courier New',courier,monospace; background-color: #ffffff">#!/bin/ksh </pre><pre style="font-size: 12px; margin: 0em; width: 100%; font-family: consolas,'Courier New',courier,monospace; background-color: #ffffff"></pre><pre style="font-size: 12px; margin: 0em; width: 100%; font-family: consolas,'Courier New',courier,monospace; background-color: #ffffff">command = `basename $0` </pre><pre style="font-size: 12px; margin: 0em; width: 100%; font-family: consolas,'Courier New',courier,monospace; background-color: #ffffff">$command $@</pre></pre><pre>q个l果本来q是挺出乎我的意料的Q于是我也想当然的认为,shell里边不能直接调用syscallQ?/pre><pre>很快得证这个揣纯_Ҏ错误的;以前q真没想q这个问题,查了下wikipedia、google之后得到很多意料之外的收莗?/pre><pre> </pre><pre>最后居然有人搬Zbusyboxq个大旗Q做q嵌入式的大多都知道些)Qƈ声称它把vi也builtin了?/pre><pre>q下也很Z我的意料Q不我没有仔细研究q,没有什么发a权?/pre><pre>不过最后有人站出来_busyboxq没有内|这些想当然的viQ而是大部分也单独赯E了Q在Unix的哲学里边,做这些大而全的东西其实是不被鼓励的,因ؓ它违反unix的哲学?/pre><pre> </pre><pre>话说回来Q面试的时候,我之所以会问到q样的问题,也是有很真实的background的。曾l我们查q的一个很诡异的performance bottleneck是׃shell脚步的问题引L?/pre><pre>====================================================================================================</pre><pre>问题本n也是比较直观的(当然是“事后诸葛”了Q:</pre><pre> 某段E序的启动脚本用如下的东东来检环境:</pre><pre><pre style="font-size: 12px; margin: 0em; width: 100%; font-family: consolas,'Courier New',courier,monospace; background-color: #ffffff">exists=`netstat -rn | grep "<span style="color: #8b0000">xx.xx.xx.xx</span>" | wc -l` </pre><pre style="font-size: 12px; margin: 0em; width: 100%; font-family: consolas,'Courier New',courier,monospace; background-color: #ffffff"><span style="color: #0000ff">if</span> [ $exists -eq 0 ];then </pre><pre style="font-size: 12px; margin: 0em; width: 100%; font-family: consolas,'Courier New',courier,monospace; background-color: #ffffff"> idx=`ifconfig -an | grep bge0 | awk -F"<span style="color: #8b0000">:</span>" '{print $2}' | uniq | sort | tail`"<span style="color: #8b0000"> </span></pre><pre style="font-size: 12px; margin: 0em; width: 100%; font-family: consolas,'Courier New',courier,monospace; background-color: #ffffff"> ifconfig bge0:`echo $idx + 1 | bc` plumb up </pre><pre style="font-size: 12px; margin: 0em; width: 100%; font-family: consolas,'Courier New',courier,monospace; background-color: #ffffff"> ifconfig bge0:`echo $idx + 1 | bc` xx.xx.xx.xx netmask 255.255.255.0 </pre><pre style="font-size: 12px; margin: 0em; width: 100%; font-family: consolas,'Courier New',courier,monospace; background-color: #ffffff">fi </pre><pre style="font-size: 12px; margin: 0em; width: 100%; font-family: consolas,'Courier New',courier,monospace; background-color: #ffffff"> </pre></pre> <p>当有很多个同Lq程Q?gt;500)恰好于同一时刻跑到q个初始化点的时候,如果pȝ上已l存在的IP地址很多Q当时的场景大概?000+Q,那么netstat、ifconfig本n都变得非常耗时Q加上多个进E的原因Q系l中会有N多个q程在消耗着资源Q?/p> <p>后果的严重程度是Mshell都停止响应,数十分钟都陷入假死,不得不重启电源了事?/p> <p>当然的分析结果发玎ͼ真正占用的CPU都是处于kernel状态的Qƈ且用率过99%Q长长的pipeline带来的开销Q相当一部分可能来源于互相等待CPU的进E的互相抢占?/p> <p>解决的方法自然也很简单,q里不赘qC?/p> <p>=========================================================================</p> <p>当时以ؓ对这个问题搞得算是比较明白了Q结果拿出来一讨论Q发现自׃了解的还真不?/p><img src ="http://www.shnenglu.com/skyscribe/aggbug/103205.html" width = "1" height = "1" /><br><br><div align=right><a style="text-decoration:none;" href="http://www.shnenglu.com/skyscribe/" target="_blank">skyscribe</a> 2009-12-14 19:46 <a href="http://www.shnenglu.com/skyscribe/archive/2009/12/14/103205.html#Feedback" target="_blank" style="text-decoration:none;">发表评论</a></div>]]></description></item><item><title>Popen中奇怪的OSErrorQtoo many open fileshttp://www.shnenglu.com/skyscribe/archive/2009/10/19/98970.htmlskyscribeskyscribeMon, 19 Oct 2009 11:31:00 GMThttp://www.shnenglu.com/skyscribe/archive/2009/10/19/98970.htmlhttp://www.shnenglu.com/skyscribe/comments/98970.htmlhttp://www.shnenglu.com/skyscribe/archive/2009/10/19/98970.html#Feedback0http://www.shnenglu.com/skyscribe/comments/commentRss/98970.htmlhttp://www.shnenglu.com/skyscribe/services/trackbacks/98970.htmlq来用Python的unittest框架来做一个regression test 工具Q验证一些复杂的E序功能Q期望做到自动化试完成一部分需要繁琐的界面点击和等待才能完成的验证工作?/font>

其中核心的部分是通过python的subproces module中的Popen来调用shell脚步完成功能性请求,q抓取执行的l果?/font>

 

初一开始,test case比较的时候(100个以内)都没有Q何问题;但从某一刻vQ当过120个的时候,最后一个test case执行的时候,L报告

                OSErrorQ?too many open files

l出的stack trace指向了Popen的communicateҎ?/font>

 

最单的怀疑自然是打开文g数的限制了, ulimit  -a 发现默认的Shell的确只有256Q虽然感觉应该也够用了(因ؓ每一个case都是用单独的Popen来执行操?完毕应该被回收了)Q但q是自然其开大一些;

                      ulimit –n 1024

重新执行Q还是最后一个casep|Q错误依然?/font>

 

Google了一下,暂时没有头AQ只有待有时间再详细盘查一下原委了。不q在搜烦{案的过E中发现了一个有意思的现象Q?/font>

   有h在询问怎样python的字W串quote成ؓ一个可以再shell中安全用的字符Ԍ有人回复说Subprocess中有一个类似的Ҏ可以完成cM的工作ؓ什么不用;马上有牛人站出来_q个模块的作者大概不想维护过多的东西Q因而将光藏了hQ而不是放开了给大家用,因ؓ它的doc里边都没有直接说明,用的人是看了他的代码才知道有q个函数Q参?/font>q里Q?/font>



skyscribe 2009-10-19 19:31 发表评论
]]>
þþƷAVþþ| 99þùۺϾƷˮ| ŷ޹Ʒþ| պŷһþþþ| þþùҺ| ˾Ʒþ| þþþþþ| ޹þþþƷС˵| 99þ99þþƷѿ| Ʒþþþþ99| Ļþۺ| ۲ӰԺþþƷ| ĻþþƷ1| aëƬþѲ| þþþþþƷѿSSS| 99þ99þþƷ| þþƷձҰ| Ʒ99þaaaһëƬ| պƷþĻ| ˾þþƷ| þùƷƬ| պʮ˽һþ| 91Ʒþþþþù۲ | aëƬþѲ| þAvԴվ| þҹɫƷ| Ʒþþþ| þþžƷ| þþƷԭ| ɫþùƷ12p| ƷŮþþþavˬ| þþþþþ97| þþƷˬ97| þþƷavӰԺ| þþþþAVר| þ޾ƷAVӣ| þþþþþþþþþƷ| 99REþþƷﶼǾƷ | Ʒþþþù3d| ݺɫþþۺ| ij˾þþþӰԺѹۿ|