Gearman是一個分布式的任務調度框架,它包括 a client,a worker ,a job server這三部分組成。
Gearman的執行過程:客戶端通過客戶端API(PHP,C,Perl等)創建一個任務發送到job server上,Job Server 通過客戶端的function name 查找合適的worker,并分到該worker上,worker接收到任務后根據worker的規則執行,并返回數據到job Server,而Job Server則把數據返回給客戶端,這樣Gearman的執行過程就結束了。
用戶可以根據不同的需求制定不同的worker來處理不同的任務,將這些worker存放到不同的服務器上,Job Server會根據不同的客戶端發送來的任務的function name尋找worker來執行,從而達到為業務服務器減輕壓力;
Gearman的安裝:
下載http://launchpad.net/gearmand/trunk/0.12/+download/gearmand-0.12.tar.gz
[falcon@www-001 ~/src/]$ wget http://launchpad.net/gearmand/tr ... earmand-0.12.tar.gz
[falcon@www-001 ~/src/]$ cd gearmand-0.12
[falcon@www-001 ~/src/gearmand-0.12]$ ./configure --prefix=/home/falcon/gearmand
[falcon@www-001 ~/src/gearmand-0.12]$ make && make instal
運行gearman 的job Server
[falcon@www-001 ~/src/gearmand-0.12]$ cd ~/gearmand
[falcon@www-001 ~/gearmand]$ ls
bin include lib sbin share
[falcon@www-001 ~/gearmand]$ sbin/gearmand --help
gearmand 0.12 - https://launchpad.net/gearmand
usage: sbin/gearmand
[OPTIONS]Main Options:
-b, --backlog=BACKLOG Number of backlog connections for listen.
-d, --daemon Daemon, detach and run in the background.
-f, --file-descriptors=FDS Number of file descriptors to allow for the process
(total connections will be slightly less). Default is max allowed for user.
-h, --help Print this help menu.
-j, --job-retries=RETRIES Number of attempts to run the job before the job server removes it. Thisis helpful to ensure a bad job does not crash all available workers. Default is no limit.
-l, --log-file=FILE Log file to write errors and information to. Turning this option on also forces the first verbose level to be enabled.
-L, --listen=ADDRESS Address the server should listen on. Default is INADDR_ANY.
-p, --port=PORT Port the server should listen on.
-P, --pid-file=FILE File to write process ID out to.
-r, --protocol=PROTOCOL Load protocol module.
-R, --round-robin Assign work in round-robin order per workerconnection. The default is to assign work in the order of functions added by the worker.
-q, --queue-type=QUEUE Persistent queue type to use.
-t, --threads=THREADS Number of I/O threads to use. Default=0.
-u, --user=USER Switch to given user after startup.
-v, --verbose Increase verbosity level by one.
-V, --version Display the version of gearmand and exit.
-w, --worker-wakeup=WORKERS Number of workers to wakeup for each job received. The default is to wakeup all available workers.
運行Job Server服務
[falcon@www-001 ~/gearmand]$ sbin/gearmand -d
判斷gearmand是否運行
[falcon@www-001 ~/gearmand]$ ps -ef|grep gearmand
falcon 9083 1 0 02:46 ? 00:00:00 sbin/gearmand -d -vv
falcon 9112 28298 0 02:47 pts/1 00:00:00 grep gearmand
[falcon@www-001 ~/gearmand]$ netstat -an -t|grep 4730
tcp 0 0 0.0.0.0:4730 0.0.0.0:* LISTEN
到此Job Server運行正常,下面我們可以簡單的在本地上測試Worker和Client是否能夠正常接收任務
我們這里用gearman命令來測試
[falcon@www-001 ~/gearmand]$ bin/gearman --help
bin/gearman: invalid option -- -
Client mode: bin/gearman [options] [<data>]
Worker mode: bin/gearman -w [options] [<command> [<args> ...]]
公共參數區
Common options to both client and worker modes.
-f <function> - Function name to use for jobs (can give many)處理任務的函數名
-h <host> - Job server host (Job Server主機,默認是localhost)
-H - Print this help menu
-p <port> - Job server port (Job Server端口,默認是4730)
-t <timeout> - Timeout in milliseconds (執行多長時間超時,微秒)
-i <pidfile> - Create a pidfile for the process (創建進程的pid文件)
Client部分參數
Client options:
-b - Run jobs in the background (后臺運行任務)
-I - Run jobs as high priority (高優先級運行任務)
-L - Run jobs as low priority (低優先級運行任務)
-n - Run one job per line (逐行執行任務)
-N - Same as -n, but strip off the newline
-P - Prefix all output lines with functions names (在輸入結果前面加處理的函數名)
-s - Send job without reading from standard input 執行任務不返回結果
-u <unique> - Unique key to use for job 任務的唯一標識
Worker部分參數
Worker options:
-c <count> - Number of jobs for worker to run before exiting (統計worker進程處理多少個任務后中止)
-n - Send data packet for each line
-N - Same as -n, but strip off the newline
-w - Run in worker mode 以worker模式運行
示例一、以命令行方式模擬worker 和 client來處理任務
開啟一個worker,以function name 為 tongji 來處理輸入的數據,統計行數并返回結果
[falcon@www-001 ~/gearmand]$ bin/gearman -w -f tongji -- wc -l
模擬客戶端連接到Job Server,以tongji函數名來提交一個文件,并接收結果
[falcon@www-001 ~/gearmand]$ bin/gearman -f tongji < /etc/profile
54
示例二、利用Gearman的php擴展來測試Gearman
安裝PHP的Gearman擴展模塊
[falcon@www-001 ~/src]$ wget http://pecl.php.net/get/gearman-0.7.0.tgz
[falcon@www-001 ~/src]$ cd gearman-0.7.0
[falcon@www-001 ~/src/gearman-0.7.0]$ /home/falcon/php/bin/phpize
......
[falcon@www-001 ~/src/gearman-0.7.0]$ ./configure \
--with-php-config=/home/falcon/php/bin/php-config --with-gearman=/home/falcon/gearmand
[falcon@www-001 ~/src/gearman-0.7.0]$ make && make install
將gearman.so加入到php.ini配置文件使其生效
測試php是否加載gearman模塊
[falcon@www-001 ~/php/bin]$ php -m|grep gearman
官方示例:
將提交的字符串翻轉后返回
Worker :worker_reverse.php
<?php
$worker= new GearmanWorker();
$worker->addServer();
$worker->addFunction("reverse", "my_reverse_function");
while ($worker->work());
function my_reverse_function($job)
{
return strrev($job->workload());
}
?>
運行worker
$php work_reverse.php &
Client:client_reverse.php
<?php
$client= new GearmanClient();
$client->addServer();
print $client->do("reverse", "Hello World!");
?>
執行client_reverse.php
$ php client_reverse.php
!dlroW olleH
參考資料:
http://gearman.org/index.php?id=getting_started
http://pecl.php.net/package/gearman
http://www.ibm.com/developerworks/cn/opensource/os-php-gearman/