Gearman是一個(gè)分布式的任務(wù)調(diào)度框架,它包括 a client,a worker ,a job server這三部分組成。
Gearman的執(zhí)行過(guò)程:客戶端通過(guò)客戶端API(PHP,C,Perl等)創(chuàng)建一個(gè)任務(wù)發(fā)送到j(luò)ob server上,Job Server 通過(guò)客戶端的function name 查找合適的worker,并分到該worker上,worker接收到任務(wù)后根據(jù)worker的規(guī)則執(zhí)行,并返回?cái)?shù)據(jù)到j(luò)ob Server,而Job Server則把數(shù)據(jù)返回給客戶端,這樣Gearman的執(zhí)行過(guò)程就結(jié)束了。
用戶可以根據(jù)不同的需求制定不同的worker來(lái)處理不同的任務(wù),將這些worker存放到不同的服務(wù)器上,Job Server會(huì)根據(jù)不同的客戶端發(fā)送來(lái)的任務(wù)的function name尋找worker來(lái)執(zhí)行,從而達(dá)到為業(yè)務(wù)服務(wù)器減輕壓力;
Gearman的安裝:
下載http://launchpad.net/gearmand/trunk/0.12/+download/gearmand-0.12.tar.gz
[falcon@www-001 ~/src/]$ wget
http://launchpad.net/gearmand/tr ... earmand-0.12.tar.gz [falcon@www-001 ~/src/]$ cd gearmand-0.12
[falcon@www-001 ~/src/gearmand-0.12]$ ./configure --prefix=/home/falcon/gearmand
[falcon@www-001 ~/src/gearmand-0.12]$ make && make instal
運(yùn)行g(shù)earman 的job Server
[falcon@www-001 ~/src/gearmand-0.12]$ cd ~/gearmand
[falcon@www-001 ~/gearmand]$ ls
bin include lib sbin share
[falcon@www-001 ~/gearmand]$ sbin/gearmand --help
gearmand 0.12 -
https://launchpad.net/gearmand usage: sbin/gearmand
[OPTIONS]Main Options:
-b, --backlog=BACKLOG Number of backlog connections for listen.
-d, --daemon Daemon, detach and run in the background.
-f, --file-descriptors=FDS Number of file descriptors to allow for the process
(total connections will be slightly less). Default is max allowed for user.
-h, --help Print this help menu.
-j, --job-retries=RETRIES Number of attempts to run the job before the job server removes it. Thisis helpful to ensure a bad job does not crash all available workers. Default is no limit.
-l, --log-file=FILE Log file to write errors and information to. Turning this option on also forces the first verbose level to be enabled.
-L, --listen=ADDRESS Address the server should listen on. Default is INADDR_ANY.
-p, --port=PORT Port the server should listen on.
-P, --pid-file=FILE File to write process ID out to.
-r, --protocol=PROTOCOL Load protocol module.
-R, --round-robin Assign work in round-robin order per workerconnection. The default is to assign work in the order of functions added by the worker.
-q, --queue-type=QUEUE Persistent queue type to use.
-t, --threads=THREADS Number of I/O threads to use. Default=0.
-u, --user=USER Switch to given user after startup.
-v, --verbose Increase verbosity level by one.
-V, --version Display the version of gearmand and exit.
-w, --worker-wakeup=WORKERS Number of workers to wakeup for each job received. The default is to wakeup all available workers.
運(yùn)行Job Server服務(wù)
[falcon@www-001 ~/gearmand]$ sbin/gearmand -d
判斷gearmand是否運(yùn)行
[falcon@www-001 ~/gearmand]$ ps -ef|grep gearmand
falcon 9083 1 0 02:46 ? 00:00:00 sbin/gearmand -d -vv
falcon 9112 28298 0 02:47 pts/1 00:00:00 grep gearmand
[falcon@www-001 ~/gearmand]$ netstat -an -t|grep 4730
tcp 0 0 0.0.0.0:4730 0.0.0.0:* LISTEN
到此Job Server運(yùn)行正常,下面我們可以簡(jiǎn)單的在本地上測(cè)試Worker和Client是否能夠正常接收任務(wù)
我們這里用gearman命令來(lái)測(cè)試
[falcon@www-001 ~/gearmand]$ bin/gearman --help
bin/gearman: invalid option -- -
Client mode: bin/gearman [options] [<data>]
Worker mode: bin/gearman -w [options] [<command> [<args> ...]]
公共參數(shù)區(qū)
Common options to both client and worker modes.
-f <function> - Function name to use for jobs (can give many)處理任務(wù)的函數(shù)名
-h <host> - Job server host (Job Server主機(jī),默認(rèn)是localhost)
-H - Print this help menu
-p <port> - Job server port (Job Server端口,默認(rèn)是4730)
-t <timeout> - Timeout in milliseconds (執(zhí)行多長(zhǎng)時(shí)間超時(shí),微秒)
-i <pidfile> - Create a pidfile for the process (創(chuàng)建進(jìn)程的pid文件)
Client部分參數(shù)
Client options:
-b - Run jobs in the background (后臺(tái)運(yùn)行任務(wù))
-I - Run jobs as high priority (高優(yōu)先級(jí)運(yùn)行任務(wù))
-L - Run jobs as low priority (低優(yōu)先級(jí)運(yùn)行任務(wù))
-n - Run one job per line (逐行執(zhí)行任務(wù))
-N - Same as -n, but strip off the newline
-P - Prefix all output lines with functions names (在輸入結(jié)果前面加處理的函數(shù)名)
-s - Send job without reading from standard input 執(zhí)行任務(wù)不返回結(jié)果
-u <unique> - Unique key to use for job 任務(wù)的唯一標(biāo)識(shí)
Worker部分參數(shù)
Worker options:
-c <count> - Number of jobs for worker to run before exiting (統(tǒng)計(jì)worker進(jìn)程處理多少個(gè)任務(wù)后中止)
-n - Send data packet for each line
-N - Same as -n, but strip off the newline
-w - Run in worker mode 以worker模式運(yùn)行
示例一、以命令行方式模擬worker 和 client來(lái)處理任務(wù)
開啟一個(gè)worker,以function name 為 tongji 來(lái)處理輸入的數(shù)據(jù),統(tǒng)計(jì)行數(shù)并返回結(jié)果
[falcon@www-001 ~/gearmand]$ bin/gearman -w -f tongji -- wc -l
模擬客戶端連接到Job Server,以tongji函數(shù)名來(lái)提交一個(gè)文件,并接收結(jié)果
[falcon@www-001 ~/gearmand]$ bin/gearman -f tongji < /etc/profile
54
示例二、利用Gearman的php擴(kuò)展來(lái)測(cè)試Gearman
安裝PHP的Gearman擴(kuò)展模塊
[falcon@www-001 ~/src]$ wget http://pecl.php.net/get/gearman-0.7.0.tgz
[falcon@www-001 ~/src]$ cd gearman-0.7.0
[falcon@www-001 ~/src/gearman-0.7.0]$ /home/falcon/php/bin/phpize
......
[falcon@www-001 ~/src/gearman-0.7.0]$ ./configure \
--with-php-config=/home/falcon/php/bin/php-config --with-gearman=/home/falcon/gearmand
[falcon@www-001 ~/src/gearman-0.7.0]$ make && make install
將gearman.so加入到php.ini配置文件使其生效
測(cè)試php是否加載gearman模塊
[falcon@www-001 ~/php/bin]$ php -m|grep gearman
官方示例:
將提交的字符串翻轉(zhuǎn)后返回
Worker :worker_reverse.php
<?php
$worker= new GearmanWorker();
$worker->addServer();
$worker->addFunction("reverse", "my_reverse_function");
while ($worker->work());
function my_reverse_function($job)
{
return strrev($job->workload());
}
?>
運(yùn)行worker
$php work_reverse.php &
Client:client_reverse.php
<?php
$client= new GearmanClient();
$client->addServer();
print $client->do("reverse", "Hello World!");
?>
執(zhí)行client_reverse.php
$ php client_reverse.php
!dlroW olleH
參考資料:
http://gearman.org/index.php?id=getting_started
http://pecl.php.net/package/gearman
http://www.ibm.com/developerworks/cn/opensource/os-php-gearman/