Recently I've come face-to-face with a significant processing task for a web application written in PHP.  I haven't worked with process control very much, so I started researching ways of distributing the calculations to multiple processes.  PHP offers several libraries for doing this (pcntl, POSIX), but it quickly became clear that if you're running Windows these libraries are not an option, and unfortunately at work I have a Windows machine.  After a lot more research, I came across Gearman.

Gearman is essentially a distributed processing framework, and seems to have community support for many programming languages.  It consists of two main components: the job server, and a Client and Worker API.  The Client and Worker API can be used in a wide variety of languages, but the job server is only available as a C library or a Perl library.  This makes it a bit tougher to get the server running on Windows, especially when you start running into some of the dependencies that it requires to build.  As well, the Client/Worker API for PHP can only be installed as a PECL extension, or a very-out-of-date PEAR extension called Net_Gearman.

Nonetheless, after yet more research I decided that I would give it a shot by using Cygwin to get the job server running (if you haven't used Cygwin before, be sure to read about it before attempting to install Gearman this way), and PEAR to use the API.  Pre-built PECL extensions aren't available for Windows anymore, and the build process for PHP extensions can be pretty painful, so it makes PEAR look good by comparison even if the code will be out of date.

I had a pretty frustrating time finally getting everything up and running due to various dependency issues, so I went back through the whole process and wrote it out step-by-step.  I used a Windows XP SP3 machine for this, but I also got it working on a Windows 7 machine as well.

Installing the Gearman job server (gearmand) on Windows with Cygwin

Installing Cygwin

1. If you don't have Cygwin already, you can get it from http://www.cygwin.com.  The setup file is located here, and the setup process is pretty straightforward; run it and follow the wizard.  Full installation instructions are available at the Cygwin site.
2. Keep the Cygwin setup.exe file handy after you've installed the default software packages, as you'll need it in the future to add packages, similar to apt-get, yum, and other Linux/UNIX package managers.
3. Cygwin installs with some basic packages, including a Cygwin Bash Shell that goes into your Start Menu.  I prefer the mintty emulator instead, as it has less of a DOS Command Prompt feel and better terminal features.  Feel free to use whatever shell you like of course.  You can get mintty by re-running the setup.exe, and at the package selection screen, type 'mintty' into the Search bar at the top left.  Expand the "Shells" category, and click on the word "Skip" under the "New" column beside the mintty package to select it before continuing the install process.

Installing Cygwin Package Dependencies needed for Gearman

If you're not already in the Cygwin setup, re-run the Cygwin setup.exe and go through to the package selection screen.  The following is a list of dependency packages you will need in order to build the Gearman job server (gearmand).  None of these packages were installed by default with Cygwin:

• gcc
• make
• libuuid1-devel
• libiconv

There's a good installation tutorial here that walks through getting gcc and make installed for people unfamiliar with Cygwin.  Finding the others is pretty straightforward, the Search bar in the package selector works well.

Installing libevent

Gearmand requires an event notification library called libevent that you cannot get as a Cygwin package, which means it has to be installed from source.  You can get the source here.

1. Download and unpack the latest libevent stable release.  At the time of this writing, I used libevent-1.4.14b-stable.
   NOTE: Download and unpack to a directory that does not contain spaces in the name, such as "C:/cygwin/home/Zombat/libevent-1.4.14b-stable".  If you unpack to something with spaces like "C:/Documents and Settings/Zombat/", the build process might not be able to install libevent correctly (libtool has a hard time with spaces)!
2. Open a Cygwin shell and cd to the unpacked libevent directory.
3. Run the following commands:

./configure
make
make install

libevent should now be installed and ready to be used when compiling the Gearman job server.

Installing the Gearman job server, gearmand.exe

1. Download and unpack the C implementation of gearmand from http://gearman.org/index.php?id=download
2. Open a cygwin shell and cd to your unpacked gearmand directory.  Same rules apply as before, make sure you've unpacked in a directory with no spaces in the path!  libtool hates that, and your build may fail.
3. Run the following commands:

./configure
make
make install

The Gearman job server should now be installed and ready to use!  Mine was installed at /usr/local/sbin/gearmand.exe, and running it with a "triple verbose" flag (-vvv) should produce the following:

That's it for the job server.  When you want to start it, simply open a Cygwin shell and run gearmand.exe.  Running it with the -d flag will cause the server to run as a daemon in the background, and running with --help will show you the full option list.

Installing the Gearman Client and Worker API (Net_Gearman)

I chose to install the PEAR Client and Worker API, as it is native PHP and doesn't involve compiling PECL extensions.  The PEAR package is called Net_Gearman, and was originally written by Joe Stump at Digg.com.  It is old and out of date now, although there appears to be a more recent fork at http://github.com/brianlmoon/net_gearman.  I stuck with the older version, as I suspect it will meet my needs, and was readily available as a PEAR package.

This also makes installation relatively painless.  Assuming you've previously set PEAR up, then all you have to do is open a command window (not a Cygwin shell) and run:

pear install Net_Gearman-alpha

The "-alpha" portion is necessary, as Net_Gearman apparently never made it to a stable release version.  That being said, it has functioned well for me so far.  Perhaps someone will pick the project up in the future.

I'll write more about getting started with the Client and Worker API in the next article, so we can actually use Gearman to get some work done.

Gearman是一個(gè)分布式的任務(wù)調(diào)度框架，它包括 a client，a worker ，a job server這三部分組成。

Gearman的執(zhí)行過程：客戶端通過客戶端API（PHP，C，Perl等）創(chuàng)建一個(gè)任務(wù)發(fā)送到j(luò)ob server上，Job Server 通過客戶端的function name 查找合適的worker，并分到該worker上，worker接收到任務(wù)后根據(jù)worker的規(guī)則執(zhí)行，并返回?cái)?shù)據(jù)到j(luò)ob Server，而Job Server則把數(shù)據(jù)返回給客戶端，這樣Gearman的執(zhí)行過程就結(jié)束了。

用戶可以根據(jù)不同的需求制定不同的worker來處理不同的任務(wù)，將這些worker存放到不同的服務(wù)器上，Job Server會(huì)根據(jù)不同的客戶端發(fā)送來的任務(wù)的function name尋找worker來執(zhí)行，從而達(dá)到為業(yè)務(wù)服務(wù)器減輕壓力；

Gearman的安裝：

下載http://launchpad.net/gearmand/trunk/0.12/+download/gearmand-0.12.tar.gz

運(yùn)行g(shù)earman 的job Server

運(yùn)行Job Server服務(wù)

[falcon@www-001  ~/gearmand]$ sbin/gearmand -d

判斷gearmand是否運(yùn)行

[falcon@www-001  ~/gearmand]$ ps -ef|grep gearmand

falcon    9083     1  0 02:46 ?        00:00:00 sbin/gearmand -d -vv

falcon    9112 28298  0 02:47 pts/1    00:00:00 grep gearmand

[falcon@www-001  ~/gearmand]$ netstat -an -t|grep 4730

tcp        0      0 0.0.0.0:4730                0.0.0.0:*                   LISTEN  

到此Job Server運(yùn)行正常，下面我們可以簡單的在本地上測試Worker和Client是否能夠正常接收任務(wù)

我們這里用gearman命令來測試

[falcon@www-001  ~/gearmand]$ bin/gearman --help

bin/gearman: invalid option -- -

Client mode: bin/gearman [options] [<data>]

Worker mode: bin/gearman -w [options] [<command> [<args> ...]]

公共參數(shù)區(qū)

Common options to both client and worker modes.  

        -f <function> - Function name to use for jobs (can give many)處理任務(wù)的函數(shù)名

        -h <host>     - Job server host  （Job Server主機(jī)，默認(rèn)是localhost）

        -H            - Print this help menu

        -p <port>     - Job server port （Job Server端口，默認(rèn)是4730）

        -t <timeout>  - Timeout in milliseconds  （執(zhí)行多長時(shí)間超時(shí)，微秒）

        -i <pidfile>  - Create a pidfile for the process （創(chuàng)建進(jìn)程的pid文件）

Client部分參數(shù)

Client options:

        -b            - Run jobs in the background （后臺(tái)運(yùn)行任務(wù)）

        -I            - Run jobs as high priority （高優(yōu)先級(jí)運(yùn)行任務(wù)）

        -L            - Run jobs as low priority （低優(yōu)先級(jí)運(yùn)行任務(wù)）

        -n            - Run one job per line （逐行執(zhí)行任務(wù)）

        -N            - Same as -n, but strip off the newline  

        -P            - Prefix all output lines with functions names （在輸入結(jié)果前面加處理的函數(shù)名）

        -s            - Send job without reading from standard input 執(zhí)行任務(wù)不返回結(jié)果

        -u <unique>   - Unique key to use for job 任務(wù)的唯一標(biāo)識(shí)

Worker部分參數(shù)

Worker options:

        -c <count>    - Number of jobs for worker to run before exiting （統(tǒng)計(jì)worker進(jìn)程處理多少個(gè)任務(wù)后中止）

        -n            - Send data packet for each line

        -N            - Same as -n, but strip off the newline

        -w            - Run in worker mode   以worker模式運(yùn)行

示例一、以命令行方式模擬worker 和 client來處理任務(wù)

開啟一個(gè)worker，以function name 為 tongji 來處理輸入的數(shù)據(jù)，統(tǒng)計(jì)行數(shù)并返回結(jié)果

[falcon@www-001  ~/gearmand]$ bin/gearman -w -f tongji -- wc -l  

模擬客戶端連接到Job Server，以tongji函數(shù)名來提交一個(gè)文件，并接收結(jié)果

[falcon@www-001  ~/gearmand]$ bin/gearman -f tongji  < /etc/profile

54

示例二、利用Gearman的php擴(kuò)展來測試Gearman

安裝PHP的Gearman擴(kuò)展模塊

[falcon@www-001  ~/src]$ wget  http://pecl.php.net/get/gearman-0.7.0.tgz

[falcon@www-001  ~/src]$ cd gearman-0.7.0

[falcon@www-001  ~/src/gearman-0.7.0]$ /home/falcon/php/bin/phpize

......

[falcon@www-001  ~/src/gearman-0.7.0]$ ./configure \

--with-php-config=/home/falcon/php/bin/php-config --with-gearman=/home/falcon/gearmand

[falcon@www-001  ~/src/gearman-0.7.0]$ make && make install

將gearman.so加入到php.ini配置文件使其生效

測試php是否加載gearman模塊

[falcon@www-001  ~/php/bin]$ php -m|grep gearman

官方示例：

將提交的字符串翻轉(zhuǎn)后返回

Worker ：worker_reverse.php

<?php

$worker= new GearmanWorker();

$worker->addServer();

$worker->addFunction("reverse", "my_reverse_function");

while ($worker->work());

function my_reverse_function($job)

{

  return strrev($job->workload());

}

?>

運(yùn)行worker

$php work_reverse.php &

Client：client_reverse.php

<?php

$client= new GearmanClient();

$client->addServer();

print $client->do("reverse", "Hello World!");

?>

執(zhí)行client_reverse.php

$ php client_reverse.php

!dlroW olleH

參考資料：

http://gearman.org/index.php?id=getting_started

http://pecl.php.net/package/gearman

Some times the core validation rules provided by Yii won't satisfy all your needs, so you'll need to create your very own validation rule.

Easy approach: inside-model rule

The easiest way to create a new validation rule is inside the model that is going to use it.

Let's say that you want to check if a user password is safe enough.
Usually you could achieve this result just by using the CRegularExpressionValidator but for the sake of this guide let's pretend that validator does not exist.

first of all in your model class you'll have to add two constants

const WEAK = 0; const STRONG = 1;

then in your rules method you'll have to set the rule

/**  * @return array validation rules for model attributes.  */ public function rules() {     return array(        array('password', 'passwordStrength', 'strength'=>self::STRONG),     ); }

make sure that you won't give the rule the name of an existing one, otherwise you are going to have some troubles later.

Now the only thing you need to do is create a new method inside the model, named after the validation rule you just declared.

/**  * check if the user password is strong enough  * check the password against the pattern requested  * by the strength parameter  * This is the 'passwordStrength' validator as declared in rules().  */ public function passwordStrength($attribute,$params) {     if ($params['strength'] === self::WEAK)         $pattern = '/^(?=.*[a-zA-Z0-9]).{5,}$/';       elseif ($params['strength'] === self::STRONG)         $pattern = '/^(?=.*\d(?=.*\d))(?=.*[a-zA-Z](?=.*[a-zA-Z])).{5,}$/';         if(!preg_match($pattern, $this->$attribute))       $this->addError($attribute, 'your password is not strong enough!'); }

The new method you just created accepts two arguments:

• $attribute = is the name of the attribute that the method is validating
• $params = additional parameters that you could define in the rules

In our rules method we used this rule on the password attribute, so the value of attribute inside our validation model will be password

In the rule we also setted an additional parameter named strength
the value of that parameter will be inside the $params array

As you can see inside the method we are making a call to CModel::addError().
Add Error accepts two parameters: the first one is the name of the attribute that you want to display the error in your form, the second one is the actual error string you want to be displayed.

Complete approach: extending the CValidator class

If you need your custom validation rule in more then one model the best thing to do is extending the CValidator class.
Extending this class you also can take advantage of other features, like CActiveForm::$enableClientValidation, first implemented with Yii 1.1.7 release.

Creating the class file

The first thing that you have to do is create your class file. The best thing is to always name it after your class name, to best use Yii lazy loading feature. Let's create a new directory inside your application extensions directory (which is located inside the protected directory).
Name this directory MyValidators.
Then we create our own file: passwordStrength.php

Inside this file create our CValidator class

class passwordStrength extends CValidator {       public $strength;       private $weak_pattern = '/^(?=.*[a-zA-Z0-9]).{5,}$/';     private $strong_pattern = '/^(?=.*\d(?=.*\d))(?=.*[a-zA-Z](?=.*[a-zA-Z])).{5,}$/'; ...

In the class file create one attribute for each additional parameter that you want to use inside your validation rule.
CValidator will take care to populate that attribute with the parameter value all by itself.
We also created two other attributes, each containing the patterns we want to use in our preg_match function.

Now we have to override the parent abstract method validateAttribute

/**  * Validates the attribute of the object.  * If there is any error, the error message is added to the object.  * @param CModel $object the object being validated  * @param string $attribute the attribute being validated  */ protected function validateAttribute($object,$attribute) {     // check the strength parameter used in the validation rule of our model     if ($this->strength == 'weak')       $pattern = $this->weak_pattern;     elseif ($this->strength == 'strong')       $pattern = $this->strong_pattern;       // extract the attribute value from it's model object     $value=$object->$attribute;     if(!preg_match($pattern, $value))     {         $this->addError($object,$attribute,'your password is too weak!');     } }

The method above is self explanatory i think.
Of course you could use constants in those IF, and I actually recommend it.

Implementing Client Validation

If you want to implement client validation you'll need to override another method inside your class: clientValidateAttribute

/**  * Returns the JavaScript needed for performing client-side validation.  * @param CModel $object the data object being validated  * @param string $attribute the name of the attribute to be validated.  * @return string the client-side validation script.  * @see CActiveForm::enableClientValidation  */ public function clientValidateAttribute($object,$attribute) {       // check the strength parameter used in the validation rule of our model     if ($this->strength == 'weak')       $pattern = $this->weak_pattern;     elseif ($this->strength == 'strong')       $pattern = $this->strong_pattern;            $condition="!value.match({$pattern})";       return " if(".$condition.") {     messages.push(".CJSON::encode('your password is too weak, you fool!')."); } "; }

As you can see this method simply returns the javascript that you need to use for your validation

Last step: how to use your validation class inside the module rules

There are several approach you can use here.

You could first use Yii::import in the rules method before returning the rules array, or you can just use Yii dot notation:

/**  * @return array validation rules for model attributes.  */ public function rules() {     return array(        array('password', 'ext.MyValidators.passwordStrength', 'strength'=>self::STRONG),     ); }